Mono 2.2 was recently shipped and a new JIT code generator was the main selling point promoted by the Mono project because it provides the first significant performance improvement Mono has seen for many years. We used the F# version of the SciMark2 benchmark to benchmark this new version of Mono against Mono 2.0 and some competing VMs using their native SciMark2 implementations:
The composite results show that the new code generator in Mono is indeed substantially better at producing code that is 40% faster on average than the previous code generator from Mono 2.0 according to these results. However, Mono 2.2 is still a long way behind its competitors. Specifically, both LLVM and Sun's JVM are over 2.2× faster than the latest version of Mono.
SciMark2 is composed of five different tests: Fast Fourier transform, successive over-relaxation, Monte Carlo, sparse matrix-vector multiply and LU matrix decomposition. We can get more detailed comparison of the VMs by looking at the individual scores on each of these five tests:
We can see immediately that the Monte Carlo test from the SciMark2 benchmark had anomalous results. Specifically, Mono 2.0 was 12× slower than LLVM and Mono 2.2 is now only 3× slower than LLVM. The Monte Carlo test is an int intensive random number generator.
These performance improvements in Mono are compelling but we are still deterred by some fundamental design flaws that undermine Mono's reliability.