Seeds of SEED: Characterizing Enclave-level Parallelism in Secure Multicore Processors

Secure processor technologies incorporating some form of enclave-based isolation are being deployed in remote cloud computing environments. However, commercial enclave-based systems, such as Intel SGX, incur performance penalties due to architectural limitations arising from enclave interactions with the operating system (OS), encryption and attestation checks for data accesses to main memory, and limitations on the enclave memory size. Enclave software development frameworks like Graphene-SGX aim to improve these limitations with performance enhancements such as exitless calling that offset the latency of expensive enclave interactions with the OS. However, to the best of our knowledge, prior works have not presented a thorough characterization of enclave performance in the presence of increased enclave-level parallelism. In this paper, we characterize how enclave overheads trade off exploitable parallelism on an Intel SGX-enabled multicore CPU for a set of parallelized workloads. We develop a microbenchmark to study the effects of threading as a function of application characteristics, such as the intensity of memory operations and system calls to the OS. We extend our characterization to realistic parallelized enclave workloads from the database and web server domains. We find that application performance scaling with threading is tightly correlated to system call and memory-bound activities in applications. The real world applications stress these constraints, while the underlying system calling implementations deliver competing performance at different thread counts.