A Prototype Naval Oceanography Uncrewed Systems (UxS) Data Pipeline

Naval Research Laboratory (NRL), Stennis Space Center (SSC), has partnered with the Naval Meteorology and Oceanography Command (NMOC) and the National Oceanic and Atmospheric Administration (NOAA) to develop an Uncrewed Systems (UxS) Data Pipeline, prototyped and tested as part of the Naval Live Virtual Constructive (LVC) Environment (NILE). Each NILE exercise (NILEX) involves a systematic process to capture metrics, which provides quantifiable feedback to the developers. The process for each NILEX includes end user training and at least two dry runs prior to the run for record. Incremental improvements are made after each dry run, and a final report is written after the run for record, with recommendations for the next NILEX. The UxS Data Pipeline was implemented within the NRL-developed Meteorology and Oceanography (METOC) Server, and moves raw sensor field collections through an automated process, whose results are easily discoverable by end users via standard formats and services. The long-term goal of the UxS Data Pipeline is dissemination of critical environmental data through an automated process in near real time that follows standardized guidelines, from upload to download, in order to enhance operational decision making for Department of Defense (DOD), as well as other federal agencies and foreign partners. The development of the UxS Data Pipeline prototype began in October 2022, with testing and evaluation going through July 2023 as part of two NILE exercises, and ends in September 2023. The pipeline starts with a user uploading their raw side scan or synthetic aperture sonar (SAS) data to an external drop zone via a web page with a simple graphical user interface (GUI). This web page also includes a resumable upload capability, in case network connection is lost during the upload. Automated processing starts with data scanning, extraction, and standard metadata tagging. Status of data in the pipeline can be monitored by the user on a web page, which also includes a geospatial map to display the mosaic upon completion. The data is converted to several industry standard formats for compatibility with multiple server types, such as Geoserver, Environmental Systems Research Institute (ESRI) ArcServer, Meteorology and Oceanography (METOC) Server, and Environmental Research Division's Data Access Program (ERDDAP). Downloads are discoverable from a RESTful API and available as the original raw format, Keyhole Markup Language (KML) Zipped (KMZ), NetCDF mosaic, XML formatted meta data, and a JSON file with quality control metrics derived from the raw data during processing. A web mapping service (WMS) is available to clients such as ArcGIS Pro. Testing done as part of NILE has already yielded improvements, as well as future capabilities needed. Future development includes adding more robust uploading tools to more seamlessly manage larger raw datasets (>8Gb), increased user testing outside the continental United States, a new standardized data format that can support a wider array of analysis needs, automated feature extraction, new dissemination controls to support different user groups, and duplication to other internal networks. With proper curation and machine-to-machine availability of the data at the appropriate level, the ability to apply new artificial intelligence / machine learning (AI/ML) technology will be greatly improved.