Challenges in the capture and dissemination of measurements from high-speed networks

The production of a large-scale monitoring system for a high-speed network leads to a number of challenges. These challenges are not purely technical but also socio-political and legal. The number of stakeholders in such monitoring activity is large including the network operators, the users, the equipment manufacturers and, of course, the monitoring researchers. The MASTS project (measurement at all scales in time and space) was created to instrument the high-speed JANET lightpath network and has been extended to incorporate other paths supported by JANET(UK). Challenges the project has faced included: simple access to the network; legal issues involved in the storage and dissemination of the captured information, which may be personal; the volume of data captured and the rate at which these data appear at store. To this end, the MASTS system will have established four monitoring points each capturing packets on a high-speed link. Traffic header data will be continuously collected, anonymised, indexed, stored and made available to the research community. A legal framework for the capture and storage of network measurement data has been developed which allows the anonymised IP traces to be used for research purposes.