Optimizing dynamic memory management in a multithreaded application executing on a multiprocessor

The Billing Gateway (BGw) is a large multithreaded object oriented C++ application running on Sun Solaris. Due to frequent allocation and deallocation of dynamic memory, the initial implementation of this system suffered from poor performance when executed on a multiprocessor. We compare two approaches for improving the performance of BGw. First we replace the standard Solaris heap with a parallel heap. In the second approach we optimize the application code by removing a number of heap allocations/deallocations. In order to do this, we introduce memory pools for commonly used object types and replace some heap variables with stack variables. The parallel heap approach resulted in a dramatic speedup improvement. The optimization of the application code did also result in a dramatic speedup improvement. For this approach the performance using a single processor computer was also increased by a factor of eight. The optimizations took approximately one week to implement