For most specific problems thrown at supercomputers, you can go 30 times faster with a custom hardware architecture baked into silicon
Perhaps that's what they should do. Make a robotic silicon wafer fabrication facility part of the computer. After being given a task requiring a new architecture, it creates the architecture it needs and augments itself. I'm sure for less than the cost of the F-35 program, a universally tasking self augmenting supercomputer could be made to happen.