An Integrated Software System for Supporting Real-Time Near-Infrared Spectral Big Data Analysis and Management

Near-infrared spectroscopy (NIRS) is a rapid, chemical-free, easy to use, and non-destructive analytical technique that has been widely applied to a diverse range of fields. NIRS analyzes the investigated samples through their NIR spectra. However, NIR spectral data are complex and multivariate, so multivariate data analysis methods (chemometrics) are used to interpret and predict the spectra's chemical and physical information. The analysis process is also very complex, involving both data processing and modeling. This paper first introduces basic concepts of NIRS analysis with the aim to show its complexity. The paper then characterizes the NIR spectral data using the "3H" of scientific big data, with the aim to show their challenges. Finally, the paper describes our initial effort on the development of an integrated software system to support efficient real-time NIRS data analysis and management. The paper claims that this development is an important contribution to tackling the challenges of scientific big data.