To all these "AI simultaneously sucks and will also destroy the planet through overuse" - exactly who do you think will be buying all this power and all these GPUs to use said power, if there's no economic value to it and a competitor could provide as good or better of a service without said insane expense?
The electricity generation market was valued at $1,6 trillion per year in 2023 (just generation, not distribution, grid services, etc). If you're going to be meaningfully increasing that, you're going to need to have some sizable percentage of that in added economic activity to justify it. And then atop that, the even more expensive aspect of said datacentres to consume said power. How many hundreds of billions or trillions of dollars are you positing that the market will decide is worth spending, while simultaneously having products that non-AI competitors could perform as well or better than? Like, adding a whole new "entire GDP of Russia"'s worth of electricity consumption and datacentres for something that consumers are indifferent to, is that what you think the market is going to pay for?
The reality is: you can run Phi-3 on a bloody smartphone without any sort of AI accelerator aboard it, and so long as you steer clear of trivia questions (or redirect them to RAG), it's excellent.. LLaMA 3 IMHO outperforms ChatGPT in most tasks and can run on a good gaming GPU. ~5 second generation = the power consumption of playing a video game for ~5 seconds. These are not world-eating levels of power consumption.
And the efficiency level for a given set of capabilities is growing exponentially - both exponentially on the hardware side and on the software side simultaneously, with a very fast doubling time. Now, we can of course offset this by using exponentially better models. And sure, in many places we will. But for any given task, you hit a point where its capabilities are good enough for the specific task it's given. Wherein, you're going to choose to apply those exponential gains to "exponentially cheaper and more efficient" rather than "exponentially more capable".
The other side of the coin is training new foundations (not finetunes, that's easy). But again, you and your investors have to believe that there's an economic case on the other end that will pay for the investment. Want a trillion dollars per year worth of training resources? Better have a clear path to tens of trillion dollars in revenue coming out the other side. That's just not happening. And again, training, too, becomes exponentially easier over time for a given level of capabilities.