Monday, 12 July 2010

Mono 2.4 still leaking like a sieve

How fast are hash tables in Mono 2.4 compared to .NET 4? An excellent question but one which led us to another trivial program that leaks memory on Mono but not on .NET (we previously gave a simple stack implementation written in F# that leaked memory on Mono 2.2).

We tried to use the following benchmark program to measure Mono's performance when filling a hash table from empty with float keys and values:

for i in 1..10 do let t = System.Diagnostics.Stopwatch.StartNew() let m = System.Collections.Generic.Dictionary() let mutable x = 0.0 for i=1 to 10000000 do m.[x] <- x x <- x + 1.0 printfn "m[42] = %g" m.[42.0] printfn "Took %gs\n" t.Elapsed.TotalSeconds

Here's the output of that program on Mono 2.4:

$ mono HashTableBenchmark.exe m[42] = 42 Took 3.45099s m[42] = 42 Took 3.2431s m[42] = 42 Took 3.39288s m[42] = 42 Took 27.2352s Unhandled Exception: System.OutOfMemoryException: Out of memory at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at System.Collections.Generic.Dictionary`2[System.Double,System.Double].Resize () [0x00000] at System.Collections.Generic.Dictionary`2[System.Double,System.Double].set_Item (Double key, Double value) [0x00000] at <StartupCode$HashTableBenchmark>.$Program.main@ () [0x00000]

Even more surprisingly, this program still dies with out of memory even if we explicitly Clear the hash table after every iteration!

12 comments:

Jonathan Shore said...

Has this been reported to the mono guys or shall I put in a bug report for you? I've tried this example on 2.6 and instead of running out of memory it simply hangs at 100% cpu after 5 iterations

Jonathan Shore said...

I filed this under the following bug report. I'm not yet using F# on mono because of fundamentals like this. Let's see where they go with this:

https://bugzilla.novell.com/show_bug.cgi?id=621899

Flying Frog Consultancy Ltd. said...

@Jonathan: Thanks. I gave up filing bugs with them after they threw the last one back in my face.

I really hope they start to take garbage collection seriously because bugs like this in the very foundation of a virtual machine will completely undermine anyone's confidence in using it for serious work.

Jonathan Shore said...

BTW are you running this test on Linux, OSX, or where? The test runs successfully on linux with mono 2.4.4, but fails for me in a different way on OSX with mono 2.6.

I'm running OSX in 64bit mode which is not well supported by mono yet. There is an early version in 2.7 pre-release builds.

Mark J. Miller said...

I don't use Mono, but I follow what they are working on. GC is a big reason why I haven't gone down that road yet. However, they have been working on a new GC (generational) for a while now (at least since 2.4). Today they announced that the 2.8 build will allow you to set a flag to make it easier to give the new GC a spin. I suggest you give it a try and see how it goes.

Flying Frog Consultancy Ltd. said...

@Mark: We'll certainly try it but the new GC the Mono team have announced for Mono 2.8 is still a conservative GC.

Frankly, I do not understand why accurate GC has not been given more attention by the Mono developers. This is an easy problem to solve and an incredibly important one for virtually all users...

Marek Sieradzki said...

Probably because it's not as easy as writing such comments on the internet. Mono 2.6.3 + amd64 + Linux works fine

Flying Frog Consultancy Ltd. said...

@Marek Sieradzki: I found it easy enough when I wrote the accurate GC in HLVM.

gelin yan said...

call

GC.Collect()

after the inner loop may alleviate the situation.

I implemented a C# version and suffered the same result: sgen error.however, add GC.Collect is a workaround though it is not a good idea...hope the next version mono can be better on that..

Justin said...

I just compiled and ran the F# program above on Linux Mint 11.

Using 'mono' it did four iterations and then output the string "Killed" and returned to the console.

Using 'mono-sgen' it ran through all 10 iterations successfully.

Very interesting. It looks like there has been some real progress.

Justin said...

Sorry, I did not mention the version of Mono I used at all in my previous comment:

Mono JIT compiler version 2.11 (master/e8807c5 Thu Jul 7 17:59:18 PDT 2011)

jay paul said...

Your post really cool and interesting. Thanks very much.

HP - Pavilion 17.3" Laptop - 4GB Memory - 500GB Hard Drive - Silver

HP - ENVY 15.6" Laptop - 8GB Memory - 750GB Hard Drive - Natural Silver (15-j011dx)