Comment Re:The first hit is always free. (Score 1) 43
This will really become the problem for selling commercial access to frontier models, if it proves to be true. (I tend to believe it will ).
If the models get thousand-fold cheaper to run, than the hardware needed to do it will be something anyone interested in more than very occasional use will be able to justify. Even if it ends up not looking exactly like consumer GPU/NPU offerings today, it will land in PC and likely even SBCs soon enough.
So now the pure AI companies will have big problem, how to charge enough to pay to build and train their next model while not pricing people out of their cloud offerings in favor of a $200 expansion card - or even a $2000 expansion card - and some maybe not as good but very good free-as-in-beer models, which both academia, non-profits, and hobby groups probably can produce.
Which is why I don't companies like OpenAI and Anthropic being able to continue with an inferences as the product business-model. They are going to have to be acquired by the Alphabets and Microsoft's of the world who can eat the costs of leading edge model development and fund them with margin from other lines of business, and want to do so because they offer "better" inference as a feature in their other proprietary software tools and platform offerings.
Setting VC money on fire has never been a sustainable business-model, eventually the activity has pay for itself or it has be vertically integrated into something that does.