From Visualisation to Data Mining with Large Data Sets

In 3D particle simulations, the generated 6D phase space data can be very large due to the need for accurate statistics, sufficient noise attenuation in the field solver and tracking of many turns in ring machines or accelerators. There is a need for distributed applications that allow users to peruse these extremely large remotely located datasets with the same ease as locally downloaded data. This paper will show concepts and a prototype tool to extract useful physical information out of 6D raw phase space data. PartView allows the user to project 6D data into 3D space by selecting which dimensions are represented spatially and which dimensions are represented as particle attributes, and the construction of complex transfer functions for representing the particle attributes. It also allows management of time-series data. An HDF5-based parallel-I/O library, with C++, C and Fortran bindings simplifies the interface with a variety of codes. A number of hooks in PartView will allow it to connect with a parallel back-end that is able to provide remote file access, progressive streaming, and even parallel rendering of particle sets in excess of 1Billion particles.