Performance modeling of multiprocessor implementations of protocols

Two major performance bottlenecks in multiprocessor execution of protocols are contention for shared memory and for locks. Locks are used to protect shared messages and/or shared protocol state in a memory shared by competing processors. Mutual exclusion by locking can be costly, in terms of both lock contention and memory contention, if the parallel protocol code frequently accesses shared state and data. This paper presents a queueing network model for performance predictions of shared-memory multiprocessor protocol executions. Predictions from this model are compared to performance measurements from a multiprocessor implementation of two commonly used communication protocol stacks, transmission control protocol/Internet protocol (TCP/IP)/Ethernet and user datagram protocol/Internet protocol (UDP/IP)/Ethernet. These stacks are implemented on a parallelized version of the x-kernel protocol environment from the University of Arizona. A "processor-per-message" paradigm is used to partition the load among the processors. The measured speedups for the parallel implementations relative to the sequential ones are more than 11 times for UDP (using 20 processors) and three times for TCP (using five processors) on a sequent symmetry. We show that the model accurately captures the effects of lock and memory contention in our shared-memory multiprocessor and predicts the performance with a discrepancy of less than 10%.