Transparent Distributed Programming under Linux

Developing parallel and distributed programs is usually considered a hard task. One has to have a good understanding of the problem domain, as well as the target hardware, and map the problem to the available hardware resources. The resulting program is often hard to port to another system. The development and maintenance process may thus be costly and time-consuming. In this paper we propose giving priority to hiding the details of distributed programming behind normal data sharing and synchronisation mechanisms. The programs are thus written as if they are meant to be run on a single, parallel computer. The developers still have to divide the task and make sure that the problem is solved in parallel, but the details of data transfer and synchronisation over a network are hidden. The resulting programs can thus be developed on nondistributed, non-parallel environments, and then be run on a variety of distributed and/or parallel platforms. Though such programs may not be as optimised as programs written specifically for a distributed computer, the speedups in programming time and costs may offset the losses. In this paper we introduce Distributed Inter-Process Communication (DIPC), a heterogeneous distributed programming system that hides inside Linux’s kernel, traps access requests to System V IPC mechanisms (messages, semaphores and shared memories) and delegates them for execution on other computers as needed. The results are then handed back through the kernel to the calling process, which is unaware of any distributed activity. DIPC currently supports Intel, PowerPC, ALPHA, MIPS, SPARC and Motorola 68k processors.