Metroflux: A high performance system for analysing flow at very fine-grain

Researches in network traffic analysis embrace a large diversity of goals and are based on a variety of methodologies and tools. To have a better insight on the real nature and on the evolution of network traffic we argue that fine-grain analysis of real traffic traces have to complement simulations studies as well as coarse grain measurement performed by classical flow measurement systems. In particular, packet level measurements and analysis are needed. However, such methodologies are resource consuming and require very high performance devices to be operational in real high speed networks. In this paper we present the Metroflux system which aims at providing researchers and network operators with a very flexible and accurate packet-level traffic analysis toolkit configured for 1 Gbps and 10 Gbps speed links. This system is based on the GtrcNet FPGA-based device technology and on specific statistical analysis tools. We show that the association of the unique fully reconfigurable grid testbed Grid5000 (up to 5000 independant high speed fully controllable sources) with a highly flexible packet capture tool allows us to achieve a unique and innovative large scale experimental potential. We also present the Network eXperiment Engine (NXE) we have developed to automate the excecution of experiments in real networks. We illustrate the application of the Metroflux system with the practical validation of the theoretical prediction relating self-similarity and heavy tails given by Taqqu theorem. We also illustrate several usages of this toolset, such as the investigation of conditions under which several traffic theories apply, as well as studies on traffic, protocols and systems interactions.