Waste not, want not: Resource-based garbage collection in a shared environment

To achieve optimal performance, garbage-collected applications must balance the sizes of their heaps dynamically. Sizing the heap too small can reduce throughput by increasing the number of garbage collections that must be performed. Too large a heap, however, can cause the system to page and drag down the overall throughput. In today's multicore, multiprocessor machines, multiple garbage-collected applications may run simultaneously. As a result, each virtual machine (VM) must adjust its memory demands to reflect not only the behavior of the application it is running, but also the behavior of the peer applications running on the system. We present a memory management system that enables VMs to react to memory demands dynamically. Our approach allows the applications' heaps to remain small enough to avoid the negative impacts of paging, while still taking advantage of any memory that is available within the system. This memory manager, which we call Poor Richard's Memory Manager, focuses on optimizing overall system performance by allowing applications to share data and make system-wide decisions. We describe the design of our memory management system, show how it can be added to existing VMs with little effort, and document that it has almost no impact on performance when memory is plentiful. Using both homogenous and heterogenous Java workloads, we then show that Poor Richard's memory manager improves average performance by up to a factor 5.5 when the system is paging. We further show that this result is not specific to any garbage collection algorithm, but that this improvement is observed for every garbage collector on which we test it. We finally demonstrate the versatility of our memory manager by using it to improve the performance of a conservative whole-heap garbage collector used in executing .Net applications.