Performance evaluation of underwater MAC protocols: From simulation to at-sea testing

Many MAC protocols have been proposed for underwater sensor networks, usually variants of well-known terrestrial approaches. Although performance comparisons among different MAC protocols have been estimated by simulations, e.g., no extensive comparison has yet been performed by means of at-sea experiments. Simulations can only capture a subset of the total environmental variability, resulting in an approximate and generally simplified model of the acoustic channel and its dynamics. Moreover, they do not generally capture constraints introduced by the actual hardware and this can significantly impact the overall protocol performance (e.g., limitations in the packet format and size, latency introduced by the hardware, control overhead associated with a given acoustic modem operation, etc.). For these reasons, at-sea experiments are needed to validate not only the relative performance of different classes of MAC protocols but also the validity of the simulation process itself. We have developed a framework to seamlessly simulate, emulate and test (at-sea) a variety of communication protocols. Three candidate underwater MAC protocols (CSMA, T-Lohi and DACAP) have been implemented on our framework (representing simple, intermediate and fully negotiated protocols). We conducted various tests in the waters surrounding Pianosa island during September 2010, comprising both single and muti-hop messaging, under various types of application loads. We then simulated exactly the same scenarios and settings in order to compare simulation and at-sea trial results. We show that if an inadequate acoustic channel model is used, or (even more importantly) if the overheads and delays due to the specific hardware are not included, there is a significant gap between actual at-sea and simulation results. Once accurate models for the channel and the modem are introduced into the simulator, a significant reduction in the gap is achieved. Moreover, we show how overcoming some of the limitations of commercial acoustic modems is expected to result in much better system performance in terms of throughput efficiency and packet latency.