We received a lot of criticism for that post. The vast majority of the criticism was not constructive but two valid points did arise. Firstly, although our result was anomalous it would be more compelling to see the benchmark repeated across a wider variety of production languages using both RC and tracing GC because this would help to reduce the error caused by differences other than GC strategy. Secondly, it would be more compelling to repeat the study with different benchmark tasks in case this one problem (computing symbolic derivatives) is anomalous. This post addresses the former concern.
We implemented equivalent solutions to the original Swift and OCaml, this time in F# and Mathematica. The F# uses a tracing garbage that is inherited from its runtime (either Mono or .NET). Mathematica is a domain specific language (DSL) designed specifically for computer algebra that uses its own heavily-optimized RC implementation so it stands the best possible chance of being able to compete with the original OCaml. So these additional measurements should usefully extend the applicability of our findings.
The F# 4.0 code was run on Mono 4.6.2 directly on the Raspberry Pi 3 alongside the Swift and OCaml and also on .NET 4.7 on Windows 10 (i.e. different architecture and platform!). Our results for Mathematica represent excess memory consumption over the initial consumption (because Mathematica has a uniquely huge standard library) and were obtained running on Mathematica 11.2 in the cloud and 10.0.1 on the Raspberry Pi 3.
According to our measurements, the solutions using tracing GCs require less memory than the solutions using RC in every single case:
Just out of interest, here are the run times too (on Raspberry Pi 3 only):