What details are needed for wireless simulations? - A study of a site-specific indoor wireless model

The wireless networking community continuously questions the accuracy and validity of simulation-based performance evaluations. The main reason is the lack of widely-accepted models that represent the real wireless characteristics, especially at the physical (PHY) layer. Hence, the trend in wireless networking is to rely more and more on testbeds, which on one hand bring more realism to network and protocol evaluation, but on the other hand present a high implementation barrier before an idea is ready to be tested. Therefore, realistic network simulators are still very much needed to reduce the time and effort for “concept testing” of novel ideas. In this case, the main question is how detailed should wireless simulators be to evaluate network and protocol performance. In this paper, we attempt a first answer to this question by using the Berlin Open Wireless Lab (BOWL) indoor model (BIM) in the ns-3 simulator. BIM includes several measurement-based models to characterize wireless communication such as frame detection ratio (FDR), frame error ratio (FER), capture and interference models. Through extensive measurements, we analyze the accuracy that we obtain with these PHY-layer models. Our experiments also show whether the detailed models at the PHY layer play an important role to represent transport layer performance in simulations.