Enhancements to 4.4 BSD UNIX for efficient networked multimedia in project MARS

Cluster based architectures that employ inexpensive personal computers (PCs) interconnected by high speed commodity interconnect have been recognized as a cost effective way of building high performance scalable multimedia-on-demand (MOD) storage servers (W. Bolosky et al., 1996; M. Buddhikot et al., 1994). Typically, the PCs in these architectures run operating systems such as UNIX that have traditionally been optimized for interactive computing and lack fast disk-to-network data paths and support for guaranteed CPU and storage access. We report design, implementation and performance measurements of innovative enhancements to 4.4 BSD UNIX carried out to rectify these limitations in the context of our Massively-parallel And Real-time Storage (MARS) project (M. Buddhikot et al., 1994). We have proposed and implemented the following enhancements to a 4.4 BSD compliant public domain NetBSD UNIX operating system: (1) a new kernel buffer management system called Multimedia M-buf (mmbuf) which shortens the data path from a storage device to network interface; (2) fair queueing within the SCSI driver for equitable resource sharing between real time and non real time streams; and (3) integration of these new OS services with a CPU scheduling mechanism called Real Time Upcall (R. Gopal, 1996) and a software disk striping driver called Concatenated Disk (CCD). Our experimental results demonstrate that these enhancements provide throughput improvements and QOS guarantees on the data path from the disk to network.