An effective way to compress Large Object Heap

[ygc369]

I think I've found an effective way to compress LOH:
If CLR always alloc every large object at the beginning of a RAM page(usually 4KB per page)，then the large object heap(LOH) can be compressed without much cost: CLR can compress LOH by modifying RAM page table and TLB instead of copying data. If so, small fragmentation maybe still exist (less then a memory page size per fragment), but there would be no large fragmentation, and compressing would be very fast because of no copying. To do this, OS support may be needed, fortunately Windows OS and Visual Studio are both Microsoft's softwares, so Microsoft can implement this at least on Windows.

[jkotas]

cc @Maoni0

[kangaroo]

To clairfy, by 'modify RAM page table', your idea here is that if the CLR could determine a page was 'free' in LOH, we could modify the PTE to point to a new physical page?

[Maoni0]

Thanks for your interest in the GC, @ygc369. The feature you are talking about is called VA remapping or swapping - it's something that needs to be implemented in the VMM. We talked to the OS guys about it a few years ago. We don't have it yet.

[ygc369]

@Maoni0 This is a great feature, not only GC, many other operations which need copy large amount of memory can also benefit from it. I think you should talk to the OS guys about it again.It will obviously improve the performance of many programs without modifying the source code.

[zgxnet]

I like the idea very much. Yes, LOH is likely to have fragments, and memory for large objects should be allocated directly, rather than have a managed heap for it. It is much better to do the address mapping in OS level than copy and move.

[omariom]

👍

[DemiMarie]

Also I would suggest filing feature requests with the Linux and *BSD kernel teams.

I believe (but could be wrong) that the last attempt to get this feature into Linux was tainted by association with the Azul Zing JVM, which – being proprietary and patent-encumbered – was looked down upon by the Linux team. The feature appeared to be only useful for a single, proprietary piece of software.

The FOSS *nix kernel developers might be much more interested if they saw that a free software VM would actually use address remapping.

[ygc369]

@drbo
Does Windows OS have this feature already?

[ygc369]

Is this feature very hard to realize?

[DemiMarie]

@ygc369 I don't know – I was referring to the now-defunct Managed Runtime Initiative, which (I believe) only ever managed to submit a patch to Linux. Azul has since resorted to shipping a custom proprietary kernel module.

[ygc369]

Virtual Machine softwares have already used this feature, I don't think it is too hard to realize.

[ygc369]

Nobody is interested in this?

[vancem]

I don't want to comment directly on the merits of pursuing this proposal, but I will mention that on a 64 bit machine, fragmentation is not as problematic as you might assume. The GC does not touch freed memory and since every object on the large object heap is > 85K (thus 42 pages), most of the pages simply drop out of the working set and don't 'hurt' real memory consumption (only address space consumption, which is significantly cheaper).

I don't want to make the statement that fragmentation of the large object heap does not matter, but my observation above suggests we need some data that suggests that large object heap fragmentation is a problem in interesting scenarios.

[ygc369]

@vancem
This proposal is not only for fragmentation problem, it can also collect garbage large objects earlier.
The cost of compressing LOH with traditional way is too much, so we can't do it frequently. But if we can compress LOH with the way I mentioned, the cost would be less. Thus we can collect garbage large objects in time.

[Maoni0]

Not compacting LOH does not mean we do not collect garbage on LOH. We just don't compact (unless you specifically tell us to). We can collect LOH as often as we need if we think it's productive.

As I mentioned above, we already talked to the OS group a few years ago about doing this and the OS has yet to implement the VA remapping feature. I will talk to them again but feel free to bring this up with the Windows group and other OS groups as @drbo mentioned.