CPUs are already plenty fast. They have been for years.
Incorrect. CPUs are plenty fast and have been for years for doing many common tasks. The fact is that they aren't nearly fast enough (particularly for single-threaded items) and almost certainly won't be for another decade or more. There's a limit to what and how much you can multi-thread, and even then, you're still limited by single-thread performance x number of threads.
So yes, for grandma playing Blackjack on Yahoo, today's CPUs are plenty fast. For me and many others? The fastest stuff available is 100x slower than "fast enough".
Do you want one very powerful computer to run everything in your house? Or do you want everything in your house to have its own dedicated, highly efficient CPU that does just what that device needs?
I want computers (and servers, especially) which are able to perform their particular function without me having to wait on them. Ever. I want usable speech recognition feeding into a responsive AI that behaves as expected without delay (and God help you if you answer "Siri" to this). I want Eve Online to be able to stick 50,000 ships in one fight with full collision and damage physics modeling with zero lag. I want to be able to transcode, store, tag, and index 20 hours of home movies and a year worth of pictures without waiting. I want to run realtime and faster simulations of complex systems.
Are these common, everyday needs? Moreso than you might think. A lot of the back-end servers struggle to keep up with workloads that either expand or change over time. While much of what's right in front of your eyes seems pretty happy with the CPU that's there today, there's a lot of stuff happening behind the scenes that isn't. This causes server admins and developers to have to spend inordinate amounts of time, money, and cranial energy figuring out how to make it functional, giving the limited computing power available.
A lot of things need very little power, and they should have very little computers with very little CPUs to make them go. Some things - things you don't think about - need tons of power, either serially or just overall. I'd pay good money if Intel and AMD would stick with 4-12 cores and concentrate on making those cores enormously powerful. As it is, they're risking going the route of SPARC, and obviously that isn't working out well for SPARC. Interestingly enough, Oracle's trying to make SPARC more like x86 even as Intel and AMD are trying to make x86 more like SPARC.