SciDB: An array-native computational database for heterogeneous, multi-dimensional data sets

As an array-native computational DBMS, SciDB is purpose-built for rapid access and scalable advanced analytics on heterogeneous, multi-dimensional data sets. NASA earth scientists use SciDB to combine and analyze 30 years of satellite image data geospatially and temporally aligned with data from many types of ground-based sensors to understand changes in extreme weather events. The Global Biobank Engine uses SciDB for large-scale joint analysis of large genotype-phenotype datasets like the UK Biobank. This paper show how SciDB's multi-dimensional arrays and in-situ distributed array processing enable pharmaceutical, medical, and materials science research that uses imaging mass spectrometry (MS), a new technology for quantifying and visualizing the spatial distribution of molecules by their molecular masses. Materials scientists use imaging mass spec to understand the interactions of substrates and coatings using various delivery methods such as sputtering and masking. Drug developers use imaging mass spec to determine where in the tissues and cells of an organ a drug compound is distributed. Drug developers can also look for indications of toxicity and efficacy by observing the redistribution of naturally occurring molecules such as lipids. The principle use of imaging mass spectrometry is to test for differences between drug candidate molecules before they are tested in humans and to explain toxicity that is observed in humans. As the use of MS scales from small experiments to automated workflows involving typical studies of animal subjects treated with different drugs, dosages, and methods of administration over longitudinal time periods, there is a critical need for software like SciDB to handle organizing and signal processing the 100s of TBs of data generated so that scientists get timely results.