Until you revise the whole way people write software, adding cores is useful to a very limited point. Today's software can be split at one core per thread, or one core per process. If you try to get two cores to work on the same thread, you just increase serial contention, not decrease it.
Even thread-happy Java is only working on maybe 3-5 threads at a time, the rest are sleeping until a device wakes it, or until a certain time has elapsed. A new compiler may be able to help a little bit, but it's just going to be creating very short-lived micro-threads when it detects those few opportunities for them.
Graphics hardware is great for many parallel cores, because it's the same tight problem with different data, endlessly repeated. Multiply these 4x4s please. Fill these pixels please. Endlessly. Same goes for encryption, and maybe a few bits of video game AI logic. Not many other software naturally fits to using many cores.