Offshore Unexploded Ordnance Detection and Data Quality Control—A Guideline

The detection of old submerged ammunition that is located on or below the seabed is an important preparatory work for offshore projects. Multibeam echosounder, side-scan sonar, sub-bottom profiler, and magnetometers are the sensor types that are most commonly used for the task. Survey design decisions center around both detection campaign efficiency and sufficient data quality to find specific reference objects. This article presents a comprehensive workflow for unexploded ordnance surveys in the sea and focuses on aspects of quantitative data quality control. For this purpose, data quality factors and corresponding threshold values were developed for each of the main sensor types. The authors designed and moderated a workshop-based stakeholder engagement process to establish expert consensus on suitable data quality factors and appropriate calculation of thresholds that define which data are sufficient and which are not. This approach was accompanied by a literature review of existing guidelines, standards, and survey recommendations. A quantitative description of data quality eases analyzing and comparing newly acquired and existing data. This article presents the results of this process, thereby providing guidance for the planning, execution, and quality control of a technical investigation with the aim of finding unexploded ordnance in the sea. The defined data quality factors derived from the literature review and collective expert knowledge can be used in the suggested workflow.


I. INTRODUCTION
O LD ammunition or unexploded ordnance (UXO) in the sea poses a threat during offshore work, such as pipeline laying or platform construction, and to the marine environment. If the UXO detection and clearance activities, which take place prior to an offshore project, are executed erroneously, managed poorly, or even overall omitted, UXO threaten the lives of construction workers, the construction schedule, the marine fauna, and the public image of the involved parties. The increase in knowledge about the potential UXO impacts [1], [2], [3], [4] has created an urge to address the challenge on a strategic level. Manuscript  Consequently, a "Quality Guideline for Offshore Explosive Ordnance Disposal" (EOD) was released [5]. It provides advice on how to conduct EOD, which is subdivided into the four phases: (I) desk-based preinvestigation, (II) technical survey, (III) investigation of suspected UXO sites, and (IV) clearance and disposal of present UXO.
In this article, we focus on the technical survey taking place in phase II. During that phase, mostly geophysical survey methods, such as different hydroacoustic and magnetic sensors, are used. Conducting these surveys is a complex task as the definition of some of the target object (munition object) parameters is challenging. Furthermore, deploying a combination of different sensors is necessary, which complicates survey planning. Therefore, knowledge about munition types, geophysical survey methods (including computer and electronic skills), data processing, and management is required to successfully conduct a UXO project. In addition, performing a technical multisenor survey can lead to the acquisition of large amounts of data. The data volume can strongly vary depending on the required resolution and, hence, the survey line spacing and required data point density, as well as the used data formats for storage. Especially, detecting small UXO items in the survey data is a demanding and time-consuming task. A commonly agreed practical guideline for the workflow and the technical requirements in phase II could support all involved personnel and save processing time. In addition, the definition of quantitative data quality factors for the conventionally used sensors would simplify communication between different parties involved in UXO detection projects.
The document written by Frey [5] is the basis for the workflow that is used here. The guide describes the entire workflow for an EOD campaign from the desktop study to the removal of UXO items. In addition, there are several alternative guiding documents available, which were reviewed for the generation of this article. One document that describes a workflow for UXO surveys for the specific cases of offshore cable installation is given in [6]. The guideline focuses on the risk assessment for a cable installation project and describes the technical survey in some detail. Simms et al. [7] describe a workflow for UXO detection surveys on land. The authors discuss UXO characteristics, different sensors that are applicable, and how the sensors are influenced by the environmental setting. A description of geophysical survey practices for marine surveys with a focus on archeological applications is given in [8], which to some extent overlap with UXO surveys. Further documents that describe the processes of marine UXO surveys and clearance are [9], [10], [11]. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ However, there are only very few guidelines and standards that provide quantitative data requirements for marine surveys. Those that are partly relevant for marine UXO surveys are: 1) IHO S-44 Standards for Hydrographic Surveys [12]; 2) OWA Guidance for geophysical surveying for UXO and boulders supporting cable installation [6]; 3) NOAA Hydrographic Survey Specifications and Deliverables [13]; 4) DNVGL Subsea power cables in shallow water [14]; 5) IOGP Guidelines for the conduct of offshore drilling hazard site surveys [15]; 6) BSH Standard Ground Investigations [16]; 7) CIRIA C754 Assessment and management of UXO risk in the marine environment [17]; 8) ISO 19901-10:21 Petroleum and natural gas industries-Specific requirements for offshore structures-Part 10: Marine geophysical investigations [18]. The main purpose of this article is to provide a detailed workflow for the execution of UXO surveys and to close the existing gap in the quantification of data quality. Therefore, we outline the workflow that describes which data and metadata are required for a UXO survey and explain how this workflow was developed. This article presents the different sensors and describes the data and metadata that are relevant for each sensor. We suggest quality factors for the data that are acquired during marine UXO surveys that partly depend on the defined reference object.
We emphasize that this tutorial article is meant as a guideline and not as an ultimate standard, as it would be impossible to define a norm for all scenarios and circumstances that can be met in the marine environment [8]. It represents the well informed and moderated joint view of the involved experts, the authors, and the existing literature and theories. The guideline shall facilitate the understanding of marine UXO surveys, support survey design, simplify data quality checks, and enable the quantitative comparison of new and existing datasets, where no such capability has previously been available. It should be noted that the suggested workflow and data quality factors could, in general, be useful for marine geophysical surveys with the aim of object detections (e.g., archeology, wreck search). Section II explains how these data quality factors and thresholds were developed. Section III introduced the workflow that includes the most important data quality factors that can be quantitatively evaluated by the defined thresholds.

II. METHODS
To generate data quality factors and thresholds for technical UXO surveys, a combined approach of a literature review and expert inquiries was used. We define data quality factors as measurable properties of survey data that determine 1) whether survey data are fit for the purpose of detecting a specified reference object and 2) whether the accuracy of the position of a detected target object is sufficient. The reference object is the smallest object that should be detectable in the data. Thresholds are values or formulas that separate acceptable data from data that are not up to the task of detecting the reference object. Therefore, the first step was defining a list with all parameters of the reference object that are required for the technical phase II. The second step was defining the data quality factors and associated thresholds for the four commonly used sensors such as multibeam echosounder (MBES), side-scan sonar (SSS), magnetometers (MAG), and sub-bottom profiler (SBP). The resulting data quality factors and thresholds are listed in Tables II-V, while in the following, the process of defining them is explained in detail.
In an initial step, documents that provide relevant guidance on offshore UXO surveys and on offshore surveys in general were reviewed (see Section I). The aim was to compile suggestions for measurable factors that determine the quality of survey data and for corresponding minimum acceptable thresholds. During the literature review, six munition parameters for the reference object were identified. Furthermore, the review resulted in 9 data quality factors for SSS, 8 data quality factors for MBES, 14 data quality factors for MAG, and 10 data quality factors for SBP. For each quality factor, a threshold was either adopted from the literature or proposed by the authors based on practical and theoretical considerations.
Next, a questionnaire was developed to allow experts to vote on whether each of the quality factors was relevant, to propose quality factors that were missing in the list, and to vote whether thresholds were suitable. The questionnaire was distributed to 125 experts, 10 of which returned with answers. The questionnaires were evaluated, and consequently, the list of munition parameters was extended to 8. The assessment furthermore resulted in a total of 15 data quality factors for SSS, 14 for MBES, 19 for MAG, and 15 for SBP. The expert vote on thresholds was rejected due to the small number of participants and due to conflicting expert views, which did not allow to decide on the further development of data quality factors.
To encourage stronger expert engagement, an alternative approach was developed. The solution was a series of moderated expert workshops. In a first tier, four remote video workshops were organized, two each on magnetics (MAG) and on hydroacoustics (SSS, MBES, SBP). This allowed participation without restraints by travel restrictions. However, limitations to participation may have arisen from the time difference between Europe and, e.g., North America. Overall, 26 experts attended the virtual workshops. The magnetics workshops were attended by a total of 17 experts, and the hydroacoustics workshops by 18 experts. Some experts attended both types of workshops. Experts were again asked to vote on the relevance of the data quality factors and the suitability of the proposed thresholds. In all workshops, experts were asked to vote on munition parameters. Participants were furthermore polled regarding the data quality factors and thresholds for sensors that were relevant for the respective workshop (hydroacoustics or magnetics). The distinguishing advantage of the workshops over the questionnaire lay in the possibility of answering participant questions prior to and during the voting process and for immediately discussing polling results for which no consensus was reached. Consensus was defined as a result according to which less than two experts deviated from the majority of the vote. Additionally, experts seem to prefer the possibility of a more direct exchange with the moderators and their peers. The polling results from the workshops were evaluated to determine which data quality factors and thresholds were accepted by the experts. A quality factor or threshold was labeled "accepted" if at least 70% of answers were positive. Quality factors were rejected when expert contributions during the discussion after the polling indicated their lack of relevance. Note that rejection of a quality factor meant that its threshold was automatically rejected as well. Of the munition parameters, four were accepted, four remained under discussion, and two were added, one of which was immediately accepted. For SSS, eight data quality factors were accepted, six were left under discussion, and one was rejected. Furthermore, six thresholds were accepted and eight were left under discussion. For MBES, seven data quality factors were accepted, six were left under discussion, and one was rejected. Further, 1 threshold was accepted and 12 were left under discussion. For MAG, 12 data quality factors were accepted, 4 were left under discussion and 3 were rejected. Further, 4 thresholds were accepted and 12 were left under discussion. For SBP, six data quality factors were accepted, seven were left under discussion, and two were rejected. All 13 thresholds were left under discussion. Discussions during the workshops demonstrated that the opinions of experts on the relevance of data quality factors for specific processes throughout the EOD workflow varied significantly. Accordingly, a workflow diagram was designed to better illustrate the application of data quality factors within a UXO workflow (see Fig. 1).
Since not all items were accepted during the first tier of virtual workshops, an additional on-site workshop was organized to focus on the remaining quality factors and thresholds. The workshop covered all four sensors and was attended by ten experts. This time, acceptance or rejection of data quality factors and thresholds was determined by simple majority. Again, the rejection of a quality factor led to the immediate rejection of the respective threshold. Due to time constraints, not all thresholds could be polled and some of them remained under discussion. The workshop's design was altered. Polling was conducted after the discussion and not the other way around. Of the munition parameters, the remaining five were accepted. For SSS, one additional data quality factor was accepted and five were rejected. No additional thresholds were accepted and, thus, three were left under discussion. For MBES, no additional data quality factors were accepted and all six were rejected. Further, four additional thresholds were accepted and two were left under discussion. For MAG, no additional data quality factors were accepted and four were rejected. Consequentially, no thresholds were accepted but none were left under discussion. Furthermore, after the expert discussion, five of the previously accepted MAG data quality factors and respective thresholds were rejected. For SBP, one additional data quality factor was accepted and six were rejected. No additional threshold was accepted and all seven were left under discussion. Following the workshop, the EOD workflow was updated.
Finally, to involve the largest possible share of the expert community for offshore UXO detection and to account for the fact that possibly not all experts had the capacity to attend the on-site workshop, all resulting data quality factors, thresholds, and the EOD workflow were distributed among the same 125 experts, who had received the initial questionnaire and were invited to the workshops. The request to the experts was to comment on or suggest changes to the proposed data quality factors and thresholds. Seven responses were received which agreed with the proposed tables. Some comments were related to changes of the proposed EOD workflow. These were implemented accordingly. The remaining thresholds, which were still under discussion after the expert involvement, were chosen based on the literature review, theoretical, and best practice considerations.

III. WORKFLOW AND SENSORS
In this section, we present the EOD workflow. This workflow was not designed for the clearance of a munitions dump site but for the management of UXO in preparation of area utilization, for example, in connection with an offshore construction project. Parts of the EOD workflow that are relevant for this article are illustrated in Fig. 1. Processes that immediately concern the parameters of the reference object, the data quality factors, and the threshold values are highlighted with bold lines. It shows that for UXO surveys, phases I and II of the EOD workflow (see Section I) are relevant. Each phase is subdivided into processes. Fig. 1 only includes processes that are relevant for the UXO survey. Phase I is targeted toward assessing, whether UXO are present in the area of interest, and toward deciding which UXO object should be the reference object. First, the processes "Documentation of Site Conditions" and "Historical Survey" need to be conducted. Only the munition parameters in Table I are the relevant output for this phase. They are generated in the process "Threat Assessment." In Phase II, survey preparations take place and the threshold values for the data quality factors need to be defined during the "Definition of Survey Methods" and monitored during the Survey Process. After Data Processing, the thresholds are verified during the Data Quality Check. If the data pass the quality check, the Data Analysis and Data Annotation take place. Ultimately, a target list is generated to continue with EOD workflow phases III and IV. Fig. 1 lists process outputs that need to be generated because they are relevant for the UXO survey. It furthermore displays for which of the subsequent processes these outputs are required as inputs. Processes and outputs that are specifically addressed in this article are highlighted in gray.
The defined data quality factors with the given formulas and thresholds (see Tables II-V) were derived from the literature review and collective expert knowledge. While some thresholds and formulas were developed by the authors, others were found in the literature and yet other were suggested by experts. All of them were approved by the experts as outlined in Section II. They are the principle suggested enhancements to the EOD workflow (see Fig. 1).

A. Phase I: Preliminary Survey
The preliminary survey is a desk-based investigation of the natural site conditions and the area's history regarding a possible entry of munitions. The aim is to assess whether one must expect to find UXO in a given area and to propose measures on how Fig. 1. Different phases and processes of the EOD workflow. The sequence of the process is displayed horizontally. The process names and their inputs and outputs are arranged vertically. Details are given for phases I and II and processes, inputs, and outputs that are relevant for the content of this publication. Processes and outputs for which the defined parameters of the reference object, data quality factors, and thresholds are important are highlighted in gray and bold boxes.

TABLE I MOST IMPORTANT PARAMETERS FOR THE REFERENCE OBJECT AND SITE CONDITIONS DERIVED FROM THE LITERATURE REVIEW AND EXPERT SURVEYS AND DISCUSSIONS
(see Fig. 1).
to manage the risk that is connected to its presence. If UXO is present, it is necessary to propose a reference object, which is the smallest munition object that must be detectable with the means applied in phase II. In addition, natural site conditions that are relevant for the preparation of a technical survey must be documented [5].
Information on the natural conditions can usually be acquired during preliminary site investigations that take place prior to offshore construction projects. Historic information on the presence of UXO, on the other hand, must be collected by scrutinizing the different causal scenarios for munitions entry (mine laying, dumping, battles, exercises, transport losses, indirect entry, e.g., by bottom trawling) [19]. Such information must be acquired through archive work. Additional clues can be obtained by reviewing past technical surveys or accidental UXO finds in adjacent areas [9], [20].
Due to the immense effort that is connected to EOD, it is usually not the intention to clear an area of every single UXO item. Instead, a lower threshold is introduced by ways of defining a reference object. UXO similar to or larger than the reference object should either be cleared before construction launches or their location must be known to avoid contact during the planned use of the area. The reference object should be the result of thoroughly executed threat and risk assessments that consider the impact of a detonation of potentially present UXO on all relevant subjects of protection. It should also take the extent and intensity of sediment intrusion of the planned use of the area into account [17]. Finally, the reference object definition must consider the technical limitations of the available survey methods. These are described in Section III-B (phase II). Table I lists the defined parameters of the reference object and the site conditions that are relevant for the subsequent work and data quality factors in phase II. The values in the third column are examples for a British 155 mm shell BL Mark VII. It should be noted that these are example values and some of the parameters could be difficult to define for specific munition types. Since the selected example object does not contain chemical warfare agents, the value of the net chemical mass is 0 kg. However, for areas in which chemical UXO is present, this is a relevant parameter for the risk assessment. In areas in which other chemical waste [21] or other hazardous materials are present, a more comprehensive risk assessment may be in order.

B. Phase II: Technical Survey
The aim of the technical survey is to generate a list of target points that can be investigated and cleared in the subsequent phases. A target point is a location at which UXO may be present  II  SUGGESTED MBES DATA AND METADATA THAT SHOULD BE RECORDED FOR UXO SURVEYS. IN ADDITION, THE MOST IMPORTANT DATA QUALITY FACTORS AND  THRESHOLDS DERIVED FROM THE LITERATURE REVIEW, THEORY, AND EXPERT SURVEYS AND DISCUSSIONS ARE LISTED. THE DATA QUALITY FACTORS PARTLY DEPEND ON THE REFERENCE OBJECT [5]. For the technical survey, numerous sensor technologies are available. The suitability of the available sensors under the conditions of a specific survey project is evaluated based on the output of phase I. This section describes the most commonly used sensors in marine UXO surveys. It covers the sensors' main functionalities and their main output (data and metadata) which are required for data processing and interpretation. In addition, the section describes the suggested data quality factors for each sensor, which can be used to evaluate whether the survey data are fit for the purpose of detecting the specified reference object.
1) Positioning Systems: Every geophysical survey requires the provision of the geolocation of each sensor measurement. An accurate positioning of the acquired data is crucial as the locations of detected target objects need to be known for the identification and potential clearance operations (phases III and IV) or for their future avoidance. Marine positioning systems can be divided into surface (above water) and underwater positioning. For the measurement of the surface position, global navigation satellite systems (GNSSs) are used. Three major methods exist, which differ in the accuracy of the measured position. These are, with increasing accuracy, 1) GNSS, 2) differential GNSS, and 3) real-time kinematic positioning (RTK). More detailed information about surface positioning systems and their accuracies can be found in [23], [24], and [25]. Sensors that are attached to the hull of the survey vessel can then be positioned relative to the GNSS antenna on the vessel. Some sensors can be towed behind a vessel, or they are installed on underwater vehicles that are remotely operated (ROVs) or autonomous (AUVs). As the electromagnetic (EM) signals from satellite systems do not propagate well in seawater, additional systems need to be used to determine the position underwater. Two major techniques can be distinguished, which differ in their functionality. These are acoustic positioning systems installed on the underwater sensor platform and a reference point, such as ultrashort baseline (USBL) positioning, and inertial navigation systems (INS). The USBL positioning uses a transmitter, usually referred to as beacon, mounted on the underwater survey platform and a receiver at a known position, for example installed underwater on the vessel. It, therefore, determines the underwater location relative to the location of the vessel. The INS consists of different motion sensors, such as accelerometers and gyroscopes, and additional accompanying acoustic sensors, such as a Doppler velocity log or an altimeter. It receives an initial GNSS position when the sensor is at the sea surface and, based on the motion of the platform, computes the position relative to the initial value for the duration the survey platform is submerged. More detailed information about underwater systems and their accuracies can be found in [26], [27], [28], and [29].
2) Motion Reference Unit (MRU): Due to the impact of waves, wind, and currents, the orientations of both the vessel and underwater systems are dynamic. For the different geophysical measurements, the orientation of the sensor (MBES, SSS, MAG, SBP) relative to a given reference frame needs to be known. This is important for proper data processing of the acoustic wave propagation and the orientation of magnetic field components. Therefore, an MRU is required. The MRU measures the angle offsets between the reference frame and the sensor orientation which are referred to as pitch, roll, and yaw. In addition, the lift or drop from the horizontal reference frame is measured, which is referred to as heave. More detailed information about motion sensors can be found in [30] and [31].
3) Multibeam Echosounder: The MBES is a hydroacoustic sensor that transmits a wide narrow swath of sound toward the seabed, perpendicular to the direction of movement of the survey platform. Multiple hydrophones receive the acoustic signal that is reflected and scattered from the seafloor or objects in the water column. Via beam forming processing steps, the system computes the time, direction, and strength of the returned signal. Therefore, one transmitted signal, referred to as ping, leads to numerous measured beams distributed across the swath, as illustrated in Fig. 2(a). Each beam contains information about the traveltime and the returned amplitude of the acoustic signal. From this information, the water depth can be calculated and information on sediment or object properties can be retrieved. Modern MBES systems can receive up to 1024 beams or more with across-track and along-track beam opening angles of about  III  SUGGESTED SSS DATA AND METADATA THAT SHOULD BE RECORDED FOR UXO SURVEYS. IN ADDITION, THE MOST IMPORTANT DATA QUALITY FACTORS AND  THRESHOLDS DERIVED FROM THE LITERATURE REVIEW, THEORY, AND EXPERT SURVEYS AND DISCUSSIONS ARE LISTED. THE DATA QUALITY FACTORS PARTLY  DEPEND ON THE REFERENCE OBJECT 0.5°and the technologies are steadily advancing. It is also possible to record several amplitude values within one beam which is referred to as snippets. More detailed information about MBES and their functionalities can be found in [32], [33], [34], and [35]. Table II lists the important data and metadata that should be acquired during the MBES survey. In addition, the table describes the recommended data quality factors, their mode of computation, and suggested thresholds. The data quality factors need to be computed on the multibeam point cloud data, related to the sounding's spatial x and y coordinates. Fig. 2 illustrates the survey setup and acquired data of the MBES. The suggested data quality factors are denoted by bold letters. The data quality factors such as data point spacing, beam footprint (along track), beam footprint (across track), and coverage allow determining whether the specified reference object can be detected in the data. Their calculation, thus, directly depends on properties of the reference object (see Table I). All four data quality factors need to be jointly evaluated as the violation of one threshold can result in objects being missed even though the thresholds of the other three data quality factors were met. The footprint threshold seems rather conservative, but in practice, the detectability within the beam also depends on the signal-to-noise ratio and the used detection algorithm (amplitude or phase detection) [37]. Coverage refers to the area between the two outermost beam footprints (soundings) of a ping and, hence, the area of the swath. The positioning accuracies (vertical and horizontal) are important for the analysis of the data and for target point investigation (phase III) and clearance (phase IV), during which the detected objects need to be relocated. It should be noted that the threshold for the horizontal positioning accuracy is mainly applicable to hull-mounted systems and shallow waters (down to ∼40-50 m [37]), e.g., areas in which offshore wind farms are developed. In deeper waters or during extended INS-positioned AUV operations, the suggested thresholds could be challenging to achieve. Also note that positioning refers to the location of the beam footprint (sounding) and not of the vessel or the sensor platform. The suggested threshold for the acoustic center frequency is provided as general guidance as the required resolution is already defined by the beam footprint (along and across track). Further suggestions regarding MBES data quality checks for the survey can be found in [34], [36], [37], [38], [39], [40], and [41].

4) Side-Scan Sonar:
The SSS is a hydroacoustic sensor that transmits two wide narrow sound beams toward the seafloor, perpendicular to the direction of movement of the survey platform, one to either side, as illustrated in Fig. 3. The same sensor that transmits the signal also records the reflected and scattered amplitudes. Conventionally, the angle of the returned signal is not known, and hence, the exact across-track position of the returned signal origin is only estimated. Modern SSS systems can also transmit different frequencies within each beam and ping. This is beneficial as there is always a tradeoff between transmission range and resolution. Higher frequencies increase the spatial resolution while the transmission range is decreased and vice versa. This way it is possible to reach high spatial resolution for areas closer to the transmitting sensor while being able to survey wider areas at a lower resolution. More detailed information about SSS and their functionalities can be found in [35], [42], [43], [44], and [45]. Table III lists the important data and metadata that should be acquired during the SSS survey. In addition, the table describes the recommended data quality factors, their mode of computation, and suggested thresholds. The data quality factors need to be computed on the SSS point cloud data, related to the sounding's spatial x and y coordinates. Fig. 3 illustrates the survey setup and acquired data of the SSS. The suggested data quality factors are denoted by bold letters. The data quality factors such as data point spacing, beam footprint (along track), signal footprint (across track), and coverage allow determining whether the specified reference object can be detected in the data. Their calculation, thus, directly depends on the properties of the reference object (see Table I). All four quality factors need to be jointly evaluated as the violation of one threshold can result in objects being missed even though the thresholds of the other three data quality factors were met. The way the signal footprint (across track) is calculated depends on the time sampling and the projection from the slant range to the ground range on the seafloor (see Fig. 3). Therefore, it can be considered a synthetic footprint. Coverage refers to the area within the swath created by the beams. The positioning accuracies (vertical and horizontal)  IV  SUGGESTED MAG DATA AND METADATA THAT SHOULD BE RECORDED FOR UXO SURVEYS. IN ADDITION, THE MOST IMPORTANT DATA QUALITY FACTORS AND  THRESHOLDS DERIVED FROM THE LITERATURE REVIEW, THEORY, AND EXPERT SURVEYS AND DISCUSSIONS ARE LISTED. THE DATA QUALITY FACTORS PARTLY  DEPEND ON THE REFERENCE OBJECT are important for the analysis of the data and for target point investigation (phase III) and clearance (phase IV). It should be noted that the threshold for the horizontal positioning accuracy is mainly applicable to USBL positioning of towed systems in shallow waters (down to ∼40-50 m). In deeper waters or during extended INS-positioned AUV operations, the suggested thresholds could be challenging to achieve. In contrast to MBES, where positioning of the beam footprint (sounding) was considered, positioning of SSS addresses the location of sensor platform in the water column. For the signal-to-noise ratio, the backscattered signals from the seafloor are the information of interest and all signals that are returned from the water column are considered noise [59]. If the recorded data only have positive values, e.g., if the intensity is recorded, the SNR in Table III could also be computed using the arithmetic mean instead of the root-mean-square (rms) for the noise. The suggested threshold for the acoustic center frequency is provided as general guidance as the required resolution is already defined by the beam footprint (along track). Further suggestions regarding SSS data quality checks for the survey can be found in [46]. 5) Magnetometers: MAG are potential field sensors that measure the existing magnetic field surrounding the sensor that is within its sensing range (see Fig. 4). This field is mainly the Earth's magnetic field. However, it can be disturbed when magnetic objects or magnetic geological structures are present. It is the aim of magnetic measurements to detect these anomalies in the magnetic field. A magnetometer measurement must therefore be processed while accounting for the Earth's magnetic field strength and other temporal disturbances (e.g., magnetic storms) in the survey area. Two types of magnetometers can be distinguished. These are scalar magnetometers that only measure the scalar magnitude of the magnetic field and threecomponent magnetometers that measure the magnetic field in three orthogonal directions. More detailed information about the detection of magnetic anomalies of UXO can be found in [47], [48], [49], [50], and [51]. Table IV lists the important data and metadata that should be acquired during the MAG survey. In addition, the table describes the recommended data quality factors, their mode of computation, and suggested thresholds. The data quality factors need to be computed on the magnetometer point cloud data related to the sensor's spatial x and y coordinates. Fig. 4 illustrates the survey setup and acquired data of the magnetometers. The suggested data quality factors are denoted by bold letters. The data quality factors such as data point spacing (along track), data point spacing (across track), and altitude allow estimating whether the specified reference object can be detected in the data. Their calculation, thus, directly depends on properties of the reference object (see Table I). All three quality factors need to be jointly evaluated as the violation of one threshold can result in objects being missed even though the thresholds of the other two data quality factors were met. The data point spacings along and across track are related to the shortest wavelength of a magnetic anomaly that could be covered by the measurements [6], [52], [53], and hence, it is not related to the reference object dimensions. The data point spacing (across track) accounts for the number of sensors in a fixed frame and, hence, is equal to the line spacing if more than one sensor in the frame is used. It is recommended that the sensor spacing within the frame follows the same threshold D ac ≈ h m 3 as smaller or larger spacings would have a negative impact on the line spacing threshold in Table IV when multiplied by the number of sensors. The positioning accuracies (vertical and horizontal) are important for the analysis of the data and for target point investigation (phase III) and clearance (phase IV). It should be noted that the threshold for the horizontal positioning accuracy is mainly applicable to USBL positioning of towed systems in shallow waters (down to ∼40-50 m). In deeper waters or during extended INS-positioned AUV operations, the suggested thresholds could be challenging to achieve. Like SSS, positioning of MAG addresses the location of the sensor in the water column. The signal-to-noise ratio requires a defined noise floor (see Fig. 4), which could be achieved by a field test in a magnetically quiet area. Further suggestions regarding MAG data quality checks for the survey can be found in [53]. The detectability of magnetic munition objects cannot be verified in the same way as for hydroacoustic methods. This is because the orientation of the object relative to the Earth's magnetic field, the shape of the object (anisotropic effects), and remanent magnetization could significantly reduce the magnetic anomaly of the object [49], [54].
6) Sub-Bottom Profiler: SBP can be divided into several different system types, which differ significantly between their acquisition setup and the applied data processing techniques [55]. Here, the SBP is assumed to be a parametric echosounder, which is currently one of the most commonly used systems for UXO surveys, if an SBP system is applied. The parametric SBP is a hydroacoustic sensor that transmits a narrow sound beam downward from the sensor. The reflected and scattered amplitudes are received at the same sensor [56], [57], [58] (see Fig. 5). Compared to MBES and SSS, the transmitted  V  SUGGESTED SBP DATA AND METADATA THAT SHOULD BE RECORDED FOR UXO SURVEYS. IN ADDITION, THE MOST IMPORTANT DATA QUALITY FACTORS AND  THRESHOLDS DERIVED FROM THE LITERATURE REVIEW, THEORY, AND EXPERT SURVEYS AND DISCUSSIONS ARE LISTED. THE DATA QUALITY FACTORS PARTLY  DEPEND ON THE REFERENCE OBJECT frequencies from the SBP are lower. This allows the acoustic signal to penetrate the seafloor. The signal is, therefore, not only returned from the seabed but also from buried objects and other changes in geological structures. The traveltime of the returned signal can be used to estimate the depth of an object and the amplitude can be used to interpret the difference in impedance between the sediment and the object. SBP systems that acquire data in 3-D are briefly discussed in Section III-B7). Table V lists the important data and metadata that should be acquired during the SBP survey. In addition, the table describes the recommended data quality factors, their mode of computation, and suggested thresholds. The data quality factors need to be computed on the SBP point cloud data, related to the footprint's spatial x and y coordinates. Fig. 5 illustrates the survey setup and acquired data of the SBP. The suggested data quality factors are denoted by bold letters. The data quality factors such as data point spacing (along track), data point spacing (across track), and beam footprint could allow estimating whether the specified reference object can be detected in the data. Their calculation, thus, directly depends on properties of the reference object (see Table I). All three quality factors need to be jointly evaluated as the violation of one threshold can result in objects being missed even though the thresholds of the other two data quality factors were met. It should be noted that the data point spacing is defined as the spacing between different acquired survey lines. The positioning accuracies (vertical and horizontal) are important for the analysis of the data and for target point investigation (phase III) and clearance (phase IV). It should be noted that the threshold for the horizontal positioning accuracy is mainly applicable to hull-mounted systems and shallow waters (down to ∼40-50 m), e.g., areas in which offshore wind farms are developed. In deeper waters or during extended INS-positioned AUV operations, the suggested thresholds could be challenging to achieve. For the signal-to-noise ratio, the backscattered and reflected signals from the seafloor and below are the information of interest and all signals that are returned from the water column are treated as noise [59]. If the recorded data only have positive values, e.g., if the intensity is recorded, the SNR in Table V can also be computed using the arithmetic mean instead of the rms for the noise. The suggested threshold for the acoustic frequency accounts for the minimum vertical resolution required to detect the reference object in the sediment, and hence, a realistic sound velocity estimate for the sediment type (see Table I) in the survey areas is required.
7) Advanced Sensor Technologies: In addition to the aforementioned commonly used sensors, other measurement techniques are being developed or are already in use on a smaller or experimental scale. These are briefly described here and references for more details are given. The systems should not be seen as completely different techniques as they have physical similarities to the ones presented above, especially the hydroacoustic methods. They could be added to the commonly used sensors later once proven to be feasible and to work reliably on large scale field applications.
EM systems are, in contrast to passive magnetometers, active geophysical systems. The associated transmitter is alternately turned-ON and turned-OFF and generates a primary magnetic field that induces eddy currents in conductive objects close by. These induced eddy currents diffusively decay over time and produce secondary magnetic fields that can be measured in corresponding electric or magnetic receiver devices. In comparison to magnetometers, EM systems usually have lower detection ranges. However, the advantage of EM systems is their detection potential of objects that are nonferrous but electrically conductive, e.g., munition made of aluminum or austenitic steel. In addition, magnetic objects that are not made from conductive materials will not be detected, which allows discriminating geogenic magnetic anomalies from a magnetometer dataset. However, a lot of energy is required for the generation of the primary field as the EM signal is strongly attenuated in saltwater. EM systems are already commonly used in phase III for the investigation of target points on a smaller scale. For more details, the reader is referred to [60], [61], [62], and [63].
Synthetic aperture sonars (SASs) are similar to SSSs. However, also due to more advanced processing techniques, the resolution and positioning accuracy of the measured returned signals is higher than for conventional SSS systems. To do this, the SAS ensonifies the same location with numerous pings. For the processing, highly accurate motion sensors are required for the SAS. For more details, the reader is referred to [64], [65], [66], and [67].
Low-frequency SASs are similar to SAS systems but they transmit lower frequencies. Therefore, the signal can penetrate the seafloor and buried objects could be detected. For more details, the reader is referred to [68] and [69].
The 3-D SBP are hydroacoustic systems that acquire data using multiple receivers and potentially multiple sources, simultaneously. This has the benefit that buried objects are ensonified from different angles at the same time, which allows enhancing the object detectability and accuracy of the positioning of the object during processing. The acquisition setup is similar to the acquisition of 3-D seismic data [70] but with higher frequencies. For more details, the reader is referred to [71], [72], [73], [74], and [75].

C. Phase III: Investigation of Target Points
In phase III, target points that were identified during phase II are investigated in detail. The aim is to check whether a UXO item is present at the target point. If this is the case, phase IV commences. If this is not the case, the target point can be signedoff as free of UXO.
For the investigation of target points, usually a combination of either magnetics or EMs with a camera mounted on an ROV is used. When visibility is limited, high-frequency acoustic scanners, such as ARIS or BlueView, can be utilized. In difficult to navigate or shallow waters, companies may employ divers instead of an ROV [5], [17]. In one recent project, the use of a crawler was reported to work under a particularly challenging water current regime [76].

D. Phase IV: Clearance and Disposal
If a target point turns out to be contaminated with UXO, it is cleared and disposed of in phase IV. Depending on whether the object is safe to transport, safe to handle or neither it can be lifted aboard the vessel, moved underwater, or must be detonated in situ, respectively. Once all UXO is cleared from a target point, it can be signed-off as free of UXO [5], [17].
Work can be performed by divers or with the help of an ROV or crawler. Optical cameras or high-frequency acoustic scanners are used to supervise the work from the vessel. After clearance, an as-left survey is conducted with magnetics or EMs to verify that no UXO is still buried in the sediment at the same location [5].

IV. DISCUSSION
This guideline should facilitate the work within a marine UXO project by describing a generalized workflow for the preliminary (phase I) and technical survey (phase II). However, the guideline may contain some content that is debatable for practical applications, of which some of the points are discussed in the following paragraphs.
The method described in Section II has some shortcomings that should be noted. During the first tier of workshops, one voting process for SBP was interrupted due to technical difficulties and was neither recorded nor could it be repeated or reproduced. Furthermore, during workshops, it happened that individual experts had to drop out of the discussion for a limited amount of time, which led to dynamic changes in the sample size during the voting process. It is also possible that experts accidentally voted differently than intended or that they accidentally voted when they did not mean to cast a vote for a specific question at all and did not notify the moderators. These are among the usual challenges when conducting polls by means of remote video conferences under time constraints. From one workshop to the next, the moderators became more experienced, both methodologically as well as regarding the discussed content. Therefore, it can be assumed that the discussion during later workshops was moderated more clearly and in a more structured manner than the first one.
The thresholds were developed under the paradigm that the reference object should be theoretically detectable when adhering to them. In applied technical surveying, there are, however, additional factors that can make the reliable detection of objects more difficult or even impossible. Unsuitable weather conditions can decrease GNSS positioning accuracy as it can lead to strong vessel and sensor motion, which in turn lead to larger errors in motion correction. Through mixing of water layers, a storm can result in changes of the sound velocity in the water column during a survey, which leads to errors in the calculation of traveltime, direction, and amplitudes of acoustic signals. These are just some examples, which demonstrate that surveyors need to consider the limitations of the used equipment that is deployed and cannot automatically assume a survey to be of sufficient quality when thresholds are met. To further verify the suggested thresholds, experimental field test could be conducted by placing known UXO objects on and below the seabed which are then surveyed by the different sensor types. There exist a few testbeds that could serve as test sites for the execution of these surveys (e.g., [77]). For each sensor type (MBES, SSS, MAG, SBP), different sensor models should be tested to get an overall threshold value for different systems and experienced marine surveyors are required to conduct the experiments. In general, it should be noted that the threshold values are guideline values that depend on environmental conditions and could evolve with new technical developments.
It should also be noted that the computation of the data quality factors is performed on the point cloud data and not on data that is interpolated onto a grid. This could lead to enormous amounts of data which need to be processed, especially for MBES and SSS systems. Current advances in high performance, distributed computing, and data management could solve the problem of big data handling [78], [79]. That large amount of point cloud data that can be processed is also demonstrated for laser-based measurements [80] and MBES surveys [81]. This might still be a limitation as commercial software tools may not allow to include these computations yet. However, more tailor-made solutions are evolving and can lead to time and cost savings if the data processing and interpretation is improved.
Another potential benefit of quantitative data quality factors is for the application of machine learning algorithms on the survey data. The quantitative data quality factors would allow to filter the data before an algorithm, e.g., for object detection, is applied to the data. In addition, the data quality factors can also be used as data features for machine learning algorithms.

V. CONCLUSION
A workflow for marine UXO surveys is presented with a focus on quantitative data quality control. The defined data quality factors are considered for general usage on marine survey data. The defined thresholds, especially for the positioning accuracies, might vary for different environmental settings (e.g., deep water). The workflow and the suggested data quality factors are based on expert knowledge (questionnaire, workshops), existing literature (primarily guidelines, standards), and theory (research articles). The overall number of involved experts was 39, while 125 were initially contacted. The experts are employed by survey companies, UXO companies, clients, sensor manufacturers, and research institutes. Therefore, we consider the data quality factors and thresholds generated here to be representative for the general industry practice and for a wide range of commercially available sensor systems. The best results from the expert engagement were received through the workshops during which direct exchange between the involved people was possible. Hence, for similar future endeavors, immediate direct engagement with experts is recommended. Furthermore, to involve experts from different time zones, the virtual workshops should be conducted at convenient times. While it would have been desirable to involve even more experts in this study, one must consider that the group of individuals possessing sufficient expertise to support the development of UXO survey data quality factors and thresholds is rather small. In addition, there exists no comprehensive publicly available expert list on the issue. Thus, the authors relied on the pool of experts they had access to from previous stakeholder engagement activities.
The workflow and data quality factors should support the understanding of marine UXO surveys for all involved people, ranging from the project manager to the surveyor and the responsible authorities. A general and more standardized way of how data are acquired and managed could save time and cost, especially in the marine environment. Therefore, this guideline can be one step toward the goal of more standardized workflows in UXO surveys.

ACKNOWLEDGMENT
The authors would like to thank the support of the experts, who filled questionnaires, attended workshops, and provided additional comments for the data quality factors, thresholds, and the workflow. For a list of all involved experts, please visit. 1 The link also contains more detailed information about the workshop results. Furthermore, they would like to thank their colleagues at GEOMAR and north.io. They also like to thank the two unknown reviewers for their constructive feedback that did improve the article's content.