So, in your world wealth is finite and if I have more then by definition it is because I, directly or indirectly, took it from someone that has less? What a dreary and depressing little world you live in.
Quite so, yes. If you're a billionaire, and you have yachts and diamonds and silks, you have those things and somebody else does not. It is binary. Those things didn't magically appear when you became rich; they are the product of (a very large number) of other people's labour. That labour was directed solely at making you happy, and not at making someone else happy- that too is binary.
Let's take the case of the yacht. One of those big billionaire's super-yachts probably took 10's of thousands of man hours to create. It also took factories, and education for the workers, and materials from the ground, and so forth. Perhaps you paid $100 million for it. Those factories could also have been used to manufacture other things- fishing boats, farm machinery, affordable family vehicles, whatever. Let's say that I took your $100 million and magically distributed it to some poor people- they could have spent the same money, used the same resources, and the same labour to create things that would be useful for them. You can't both have it; if the factories, labour and resources are making yachts, they are verifiably unable to make other things at the same time.
Let's take food. You're a billionaire, so you decide to host a great big feast for all of your swanky friends. You order 100 dishes, each made with a dazzling array of ingredients, enough food to feed 10 times as many guests as you have coming (but you just must do it that way to ensure you don't run out of the best bits!). At the end of the night, after you've all eaten far more than you could possibly need to eat, you throw out 80% of it. Farm land, farm labour, fertilizers, feeds, pesticides and so forth were all used to make that food- and not make food for others.If your wealth were more evenly distributed amongst hungrier people, the same amount of food from the same amount of farmland (et al) could have been more usefully distributed. As it is, you had the food and someone else didn't- food wasted is gone for good. It is binary- your excess is directly linked to other people's suffering.
And let's not even get on to the misery involved in the production of luxuries such as diamonds.
To pretend that wealth is not finite might be less "dreary and depressing", but it has consequences that you can't deny simply because you don't like the sound of them. If wealth were truly not finite, then presumably it would be possible for every single person alive to be a billionaire, living a billionaire's lifestyle. Everyone can have a super yacht! Diamonds for all! Obviously it's a nonsense. The things worth having in the world are not infinite, they are limited- if someone has a lot of "stuff", other people will, as a consequence, have less.
One day we might have increased the pool of "stuff" in the world (through those magical replicators and almost limitless sources of energy we keep hearing so much about) to such a point as everyone's wildest material desires can be fulfilled easily, with "stuff" to spare. But until that point, rich people are and will remain rich at the expense of the poor.