I remember when floating point was the luxuriously optional silicon. I try to be welcoming to new things even if I don't know how/if I'll use them, because I think they don't really inflate the cost of the processors much. (Am I right? I don't actually know.)
Long-term, I think there's widespread consensus that integrated floating point was a good idea. Even less controversial, integrated MMUs are a critically necessary part of our modern world. (It's hard to imagine that separate chips like the 68851 used to exist.) The vector stuff? Some code uses it. The cryptographic instructions? Oh hell yes! Maybe I'll get reamed for this, but I think the processor industry has a pretty good track record of making silicon that we eventually truly do light up.
This time, it's a little harder. The applications for LLMs seem so niche. Part of me thinks they're doing this several years too soon. But that said:
0. Neural networks have more applications than LLMs. However worthless you think LLMs are: if your computer is good at LLMs, what else might it be good at?
1. I strongly disagree with everyone who says the hypothetical applications for this should run "in the cloud" instead of on the user's own hardware. All my experience tells me that's definitely wrong. IF this "AI" stuff really isn't a bubble (I think it probably is), then getting coprocessors widely deployed for it out there, is a very good thing. "AI" is no different from non-"AI" logic, in that whatever you're doing, from the user's point of view it should be as local as possible, and with as few external dependencies as possible. You don't need to teach me that lesson again for the 100th time, dammit. Maybe a lot of laypeople will get stuck with OpenAI's (or whoever's) services, but we will want to run it on our machines.
2. Maybe the reason there are so few existing applications that use neural nets, is that the cheap hardware to make it practical isn't out there yet! Get the silicon out there and then developers will find uses for it. Back when I was stealing my employer's electricity (and coffee) at night, I ray-traced on a network of 80386s and 80486s, and the 486's floating point performance made me a lot more excited to work on my ray-tracer. Had it ran slower, I would have moved on to the next amateur time-waster sooner.
But I can see why consumers wouldn't care a bit, right now. By the time you have real use for this hardware, I think you'll have already retired the new machine that you're buying today. But wasn't that sort of the case with vector and crypto instructions too? Different people will check it out at a different pace. It's always been like that.