A good JIT actually performs very well compared to ahead of time compiling, and can take advantage of execution profiling (for things like branch prediction) by deferring compilation until the bytecode runs a few times.
The main advantages of gcj are improved startup times, porting to unusual architectures, and maybe for memory constrained targets.
When comparing C++ performance to Java, there are many things to keep in mind aside from the JIT compiler:
- Java also has GC overhead. You get a fairer comparison by adding garbage collection to your C++ app. The performance effects of garbage collection are complicated, but one of the difficult problems is that it tends to fill the processor data cache with useless entries.
- Java adds overhead required by the JLS such as array bounds checking or class initialization tests.
- Exception handling can also impact code generation. Java requires null pointer dereferencing (for example) to be trapped and handled, which adds abnormal edges to a CFG which places restrictions on e.g. instruction scheduling. Strict expression evaluation order, required by the JLS, has the same effect.
- Java has a more limited type system. In particular objects other than primitive types cannot be passed by value, which tends to result in more frequent memory allocations, more garbage collection, and more pressure on the memory cache than languages that can make more efficient use of the stack.
The biggest reason worked stopped on gcj was the release of OpenJDK, since one of the main reasons for gcj in the first place was to have a fast and free Java runtime. OpenJDK satisfies both.