The reason why wealth itself is important is because wealth begets wealth. You're right that to some extent it's an under-consumption problem, but fundamentally, wealthy people use their wealth to accumulate more wealth.
And that sounds a bit tautological, but what I mean is: if you have a huge amount of wealth, the things you buy are things like property, stocks in other companies, natural resources, etc. Wealth accumulation accelerates over time. But there isn't an infinite amount of wealth to be had. There's a limit to how many homes there are, how many farms and oil wells and all those things. Governments are in debt and can't help their citizens because they no longer own big blocks of apartments, they no longer own productive land or natural resource companies. (Actually, some do, and you'll note that they're the ones that provide ongoing support to their citizens.)
Even with a little bit of wealth, you can anchor your whole life. Owning your own home is huge. Put enough money into investments, and there's a time at which contributions to your portfolio are less than the money the portfolio generates on its own. And of course, this wealth is both real and illusory; stocks have no value until you sell them, but they can serve as collateral for loans at lower interest rates, and you don't pay taxes on loans.
The rich DO compete with folks for real estate, they become landlords, which is an inherently parasitic position. Consider: a landlord buys a home and rents it out. They pay the mortgage and for upkeep, and they charge a tenant for their own costs, plus some amount of profit (or, occasionally, no profit, on the basis that land value goes up over time). If the tenant can pay this rent, that means they could own the property themselves, but there's an intermediary skimming profit off the top.
So more and more people are locked out of ownership, because the initial cost of ownership is too high, but we know factually that they could actually pay for and maintain that property themselves, because that's what the rent covers. Where does that extra money go? And what is it paying for? It's paying for nothing; the landlord is a parasite, plain and simple.
But anywhere that someone with tremendous wealth goes, the same story is allowed to play out. We've all seen the stories of these private investment firms buying up good businesses, saddling them with debt, taking all the good stuff, and then leaving the business to go bankrupt. It's wealth extraction that only rich people can do, and it destroys businesses big and small.
Ultimately, the accumulation of wealth is a de facto accumulation of power. Our politics is not SUPPOSED to be influenced by money, but it definitely is. Elon Musk doesn't need to be able to vote multiple times to influence the outcome of elections, he can just throw hundreds of millions of dollars at it--an amount he makes back in a trivial amount of time--and get the outcome that he wants.
You're right that the extremely rich use the same phones as I do, eat the same food as I do. What that tells us--what we all already know--is that they don't NEED to be that rich. They can live a life of extraordinary luxury with so much less. So my concern with wealth is that it is not created out of thin air, and there's a finite amount of it to go around. The billionaire class growing in number as well as their fortunes growing by huge amounts happening at the same time as governments running huge deficits and the middle class shrinking is not a random coincidence. One is the cause of the other. That's why when many of us talk about wealth, it's as a *transfer*, from one class to another. Billionaires are literally impoverishing the rest of us.