There is a _big_ difference between clumsily optimized (or unoptimized) Java and carefully-optimized Java--more, in my experience, than the difference between clumsily optimized Java and clumsily optimized C or C++. So if you are already using Java for some reason (robustness to faults, ease of parallelism of certain kinds (w.r.t. C), library that does exactly what you need, etc.), you should figure out how to optimize it before bailing out and using a different language.
Only if you absolutely must get as much out of your hardware as physically possible should you start using C/C++, and at that point, don't expect to be using ANSI C; you should be issuing SSE4 instructions and such (basically writing targeted assembly, even if you are doing so in a way that looks like C functions) that have been cleverly crafted to do exactly what you need.
(And don't forget that while you are taking extra time to write all this low-level high-performance code, your computers _could_ have been running using the slower code, making progress towards a solution, or serving customers albeit with delays, etc..)