The effects of cache architecture on the performance of operating systems in multithreaded processors

Multithreading is well accepted as a technique to hide latency in non-blocking cache architectures. By switching execution from a blocked thread to another, the CPU can perform useful work, while waiting for long requests to be processed by the memory hierarchy. In this paper we study four memory hierarchy organizations, and we find that the performance of the operating system, in terms of response time, greatly depends on the cache configuration and memory hierarchy. For a simple cache design, the OS response time can be below 50%, which can only be improved by increasing the cache size or associativity. We find that the best way to guarantee OS response time in multithreaded processors is to use pipelined memory, which reduces the miss penalty and thus the uncertainty in response time.