Analysis on Effective UAS Survey Conditions for Classification of Coastal Sediments

This study aims to introduce effective unmanned aerial system (UAS) survey conditions for coastal sediment classification, including muddy sand, sand, gravel, and shells in a tidal flat area. UAS images with resolutions ranging from 2 to 60 mm are used as an implication of survey altitudes. The UAS images are used for sediment classification using random forest (RF) and support vector machine (SVM) methods. The results showed that RF is more effective in sediment classification while the general accuracy pattern was similar. The accuracy decreased with lower spatial resolutions. Notably, there is a significant drop of accuracy with a resolution coarser than 40 mm. Considering the training data selection, classification accuracy, and survey efficiency, it is suggested that 40 mm UAS images would provide optimal condition with acceptable accuracy for coastal sediment classification using RF model. To gain higher accuracy, a lower flight altitude is required, which will elongate the survey time significantly. Given the fact that this study is the first approach to test various UAS survey conditions for coastal sediment classifications in a field condition; the methodology and findings of this study can serve as a guideline framework for future coastal UAS sediment mapping.

SECTION I. Introduction

GRAIN size of coastal sediments is a key component for understanding coastal topography, habitat of marine organisms, and geological processes [1]–​[3]. Worldwide, researchers have been actively using remote sensing for grain size analysis of coastal sediments to provide low-cost and an efficient survey approach over traditional methods [1]–​[4].

There is a recent trend in coastal sediment mapping of a vigorous participation of unmanned aerial systems (UAS) by taking the advantages of their high flexibility, accuracy, and efficiency for survey [1], [3]–​[8]. Vázquez-Tarrío et al. [5] analyzed grain roughness and size distribution in a braided, gravel bed river at the 2 cm resolution for grain sizes ranging from sand to gravel by using UAS optical imagery and structure from motion photogrammetry. Woodget and Austrums [4] predicted grain size of gravel sediments in a river channel based on the point roughness acquired from UAS survey at 1 cm resolution with an R² of 0.7554. Arif et al. [7] classified river bed sediments for grain size from silt to boulder based on object-based classification at 1 cm resolution UAS image. Kim et al. [1] used UAS images at 26 cm resolution, tidal channel network, 50 cm DEM, and tidal channel density data for classification of tidal flat sediments ranging from silt to gravelly sand with an accuracy of 72.8%.

However, the UAS survey also has limitations. To acquire the target image with an optimal quality, flight condition including flight altitude associated with spatial resolution, and flight time associated with illumination condition have to be properly assigned [8], [9]. Inappropriately configured flight altitude of a UAS survey may lead to low image quality or unnecessary cost of survey time and labors. However, previous studies [1], [4], [5], [7] only dealt with single spatial resolution for sediment classification, and thus, the UAS flight condition for optimal data acquisition is not well discussed.

There were several approaches to test spatial resolution and classification efficiency of a UAS survey. Sturdivant et al. [6] classified land cover of a coastal area using 2.5, 15, 35, and 50 cm UAS imagery with an accuracy of 85% for the classes of shrub, vegetation, marsh, sand, and water. Flores-de-Santiago et al. [8] analyzed a UAS survey condition, which was applicable for coastal biodiversity at flight altitudes ranging from 60 to 120 m (2.5 to 5 cm resolution). Bae et al. [3] experimented with the applicability of UAS images at various survey altitudes for six classes of sand and gravel for UAS survey control for sand classification. However, these studies are not focused on coastal sediments [6], [8] or experimental setting [3]. Although Bae et al. [3] reported the impact of survey altitude on the accuracy of grain size classification, their work was based on a controlled laboratory condition, and there is larger uncertainty in a field condition.

The goal of this research is to evaluate UAS survey conditions for coastal sediment classification. We used UAS images at various resolutions for a tidal flat and classified sediment types using machine learning classification methods, such as random forest (RF) and support vector machine (SVM). The optimal UAS survey conditions in term of spatial resolution and flight time are suggested.

SECTION II. Methodology

A. Study Area

This study selected a tidal flat on Hansan island, Korea (34°47’52”, 128°29’2”) (see Fig. 1). This island is in the southern sea of Korea and belongs to Hallyeohaesang National Park. The tidal flat has a complicated sedimentary environment composed of sediments ranging in size from mud to boulder, because a freshwater channel is developed in a relatively small area. We surveyed the tidal flat for two sectors with sizes of 6600 and 10300 m². The field survey confirmed the alternating sediment distribution of fine-grained and coarse-grained sediments, which were composed of muddy sand, sand, gravel, and shells at the surface. The complicated distribution of different sediments makes this study site an ideal test site for grain size mapping using UAS survey.

Fig. 1.

Study area and location map.

B. UAS Data Acquisition and Processing

The UAS survey for the study site was carried out from September 9–13th, 2020 during low tide time for one or two times per day depending on the tidal condition (see Table I). A total of 54 ground control points (GCPs) were collected over the survey area for geometric calibration (see Fig. 2). The GCP positions were measured with Trimble R2 GNSS receiver. The UAS image was acquired by DJI Phantom 4 pro V2.0 with 1”CMOS sensor. The sensor's configuration is field of view (FOV) 84°, focal length 8.8 mm, and image size of 5472 × 3648 pixels, respectively. The flight path was managed with DJI GS Pro application for the total survey distance of 19 829 m. The UAS survey was conducted eight times with front overlap 80%, side overlap 70%, and 5 m flight altitude with Hover and Caper mode to generate high resolution ortho-rectified images.

TABLE I Details of UAS Survey Schedule for This Study

Fig. 2.

Location of ground truth and GCP locations overlaid with flight sectors (red box).

An additional UAS survey to acquire a full scene of the study area was conducted with Matrice 100 (DJI) with Zenmuse X3 sensor. The sensor has specifications of 1/2.3”CMOS, FOV 94°, focal length 3.61 mm, and image size of 4000 × 3000 pixels, respectively. A total of 548 UAS images were acquired at 100 m altitude for the total survey distance of 8648 m with 90% front overlap and 71% side overlap. Different from high resolution UAS survey, this UAS survey provides a relatively consistent illumination condition for image acquisition as the survey was conducted within a short period.

A total of 338 to 2111 UAS images for each survey sector were collected. Each survey session was processed with metashape professional software (Agisoft) to produce ortho-rectified images at a 2 mm spatial resolution [4], [10].

C. Ground Truth Data

A total of 40 ground truth samples were acquired for training data selection and validation where the sample sites are identical to the GCP locations for coregistration between field survey and UAS survey (see Fig. 3). Because UAS images can only detect surface condition of the tidal flat, the ground truth samples were only selected from top 3 cm sediments for amount of 500 g∼1000 g. The samples were carried to a laboratory and air dried for three days. An additional drying session was applied to the samples in an oven at 60 °C for 36 h to achieve a fully dried condition. The dried samples were sieved and sorted to mud (<0.0625 mm), sand (0.0625–2 mm), and gravel (>2 mm). The sorted field samples were classified by Folk's sediment classification diagram [11] based on the proportion of mud, sand, and gravel. The sediment classification was then grouped to represent gravel dominated, sand dominated, and mud mixed sand to represent the three main types of sediments; gravel, sand, and mud. The final classification results were used as a training data selection and validation.

Fig. 3.

Snap photos of sediments in the tidal flat. (a) Muddy sands. (b) Sands. (c) and (d) Gravels. (e) and (f) Shells.

D. UAS Image Classification of Coastal Sediment

To figure out the optimal UAS flight conditions for grain size mapping of coastal sediments, this study used RF algorithm and SVM for image classification at different image resolutions. RF and SVM are the most commonly used machine learning algorithms in image classification of remote sensing field with satisfiable accuracy and reliability [12]–​[15].

Many previous studies compared various kinds of machine learning algorithms and concluded that the RF was one of the most powerful classification algorithm in remote sensing application [16], [17]. The RF algorithm was developed by Breiman [18] based on an ensemble method. This method has relative advantages in minimal parameter requirements [13], and provides the variable importance inferring the most useful bands for the classification. Furthermore, previous studies have reported the RF as one of the most efficient machine learning-based classification methods [16], [17]. The RF classification requires two input parameters, the maximum number of tree (Ntree) and the user-defined number of features (Mtry). This study set Ntree to 500 following previous studies [19], [20], and Mtry as a root of number of input variable [21]. We used R statistics language for RF classification [22], [23].

The SVM is widely used in a remote sensing field because it manages a relatively small number of training datasets efficiently and is effective in high dimensional space, such as remote sensing images [14], [15], [24], [25]. The SVM is a kernel -based algorithm introduced by Vapnik [26]–​[28]. The SVM is to define a hypothetical hyperplane, which then separates the input dataset into predefined classes. The closest input vectors to the hyperplane are called support vectors, and the distance between the hyperplane and support vectors is called the margin. This study used radial basis function kernel (RBF), the commonly used kernel for remote sensing [14], [15]. The RBF-based SVM requires setting two parameters, c and gamma. C is a penalty parameter where high c values form hard margins prohibiting errors and low c values form soft margins allowing errors. Gamma is the parameter that determines the spread of the RBF kernel where high values may cause overfitting and low values may cause underfitting [14]. We assigned 100 for c and 0.003 for gamma with a classification probability threshold as 0.5 for image classification. The ENVI 5.6 is used for SVM classification.

To figure out the optimal spatial image resolution, which requires minimum flight efforts with maximum classification accuracy, this study resampled the original ortho-rectified UAS images at 2 mm resolution to 4, 10, 20, 30, 40, 50, and 60 mm, respectively (see Fig. 5). The spatial resolution can be converted to UAS flight altitude to 7.5, 15, 37, 75, 113, 150, 188, and 226 m based on the conversion equation [29]. The images at each resolution consist of three visible bands (R, G, B), and a median filter was applied to remove noise pixels showing salt and pepper textures. The noise removed images were converted to an intensity image for texture filter application following the equation below [30]:

$$\begin{equation*} {\mathrm{Y\ }} = {\mathrm{\ }}0.3{\mathrm{R}} + 0.59{\mathrm{G}} + 0.11{\mathrm{B}} \tag{1} \end{equation*}$$ where Y is the intensity of image pixel in digital number, and R, G, and B are digital numbers of the Red, Green, and Blue band, respectively. The intensity image is, then, converted to texture images derived from grey level cooccurrence matrix, including contrast, homogeneity, dissimilarity, and second moment bands [3], [30]–​[32]. The composite of eight bands including R, G, B, intensity, contrast, homogeneity, dissimilarity, and second moment images at each resolution are used for grain size classification of coastal sediments based on RF and SVM.

Training and validation datasets were constructed based on ground truth data, ortho-rectified images, and field survey snapshot pictures. The field sample and snapshot pictures at 40 locations were first classified to each grain size based on the grain size analysis at a laboratory. Then, the image pixels of ortho-mosaic images corresponding to the sampling sites were selected as training data representing each grain size. At that point, image pixels showing the same characteristics are additionally selected for training and validation pixels based on manual selection using iterative self-organizing unsupervised classification results as an additional reference. So, 70% of the selected image pixels were used as training data and the remaining 30% pixels were used as validation data (see Table II). Finally, the accuracy assessment at each resolution was analyzed using the confusion matrix to figure out the most efficient image resolution, and efficient flight altitude for the UAS survey was defined. The data processing was conducted by ArcGIS 10.5 and ENVI 5.6 software.

TABLE II Number of Pixels Used for Training and Validation Data for Grain Size Classification at Various Spatial Resolution

SECTION III. Results and Discussion

The grain size analysis of 40 ground truth samples revealed that the sediments of the study area consist of muddy sandy gravel (msG), sandy gravel (sG), gravelly sand (gS), and gravelly muddy sand (gmS) by Folk's classification (Fig. 4). The gravels commonly have larger particle sizes with large variations up to meters. Consequently, the image classification often classifies individual gravel as an independent sediment. This phenomenon often causes confusion in the classification results. Therefore, the classes of sediments for the image classification are reclassified based on the dominant sediment class. The gavel dominated classes, including msG and sG are defined as gravel. The sand dominated gS is defined as sand, and the mud mixed sand gmS is defined as muddy sand. In addition, the oyster and clam shells were classified as shells based on visual observation of the high resolution image (see Table III).

Fig. 4.

Sediment classification of training samples plotted in Folk's sediment classification diagram [11].

TABLE Ⅲ Redefinition of Sediment Classes Used for This Study From the Grain Size Analysis

The original ortho-rectified image and the resampled images were visually inspected for grain recognition in the images [see Fig. 5(a)] for selection of training data. The highest resolution image at 2 mm resolution can distinguish texture differences for all kinds of grain sizes with the naked eyes. At 10 mm resolution, the textures of different grain sizes can hardly be recognized while the shape of oyster shells can be distinguished. From 40 mm resolution, the grain texture cannot be detected by ortho-rectified image while the image texture is recognizable with the help from the ground truth data. The images with coarser resolution than 40 mm cannot detect the texture differences even with the ground truth data.

Fig. 5.

(a) Ortho-rectified image clips of the coastal sediment acquired by UAS survey at each spatial resolution. (b) The results of classification using random forest.

The grain size classification by RF and SVM was carried out for ortho-rectified images at different resolution and the classification accuracy was assessed to figure out the most efficient UAS survey conditions for grain size analysis of coastal sediments [see Table IV and Fig. 5(b)]. The results of RF classification showed that the best accuracy of the training data was derived from 2 or 5 mtry; the important variables for grain size analysis were Blue, intensity, Red, and contrast bands. The overall accuracy of training data ranged from 68.5% to 86.8% with kappa coefficient 0.56 to 0.81 where the overall accuracy and kappa coefficient decreased with a decrease in spatial resolution. The highest accuracy was achieved at the spatial resolution finer than 10 mm with overall accuracy higher than 80% and kappa coefficient higher than 0.78. At 40 mm resolution, the validation accuracy was 80%, and the accuracy dropped below 70% for images with 50 mm resolution or coarser. Excluding the shell class, the classification accuracy was the highest for gravels followed by muddy sands and sands [see Table IV and Fig. 5(b)].

TABLE IV Accuracy Assessment of Tidal Sediment Classification and Important Variables Derived From RF

The SVM classification showed similar results, while the RF performed better. The overall accuracy of the validation data ranged from 52.1% to 86.9% with kappa coefficient 0.42 to 0.82. The accuracy decreased with a decrease in spatial resolution and the accuracy dropped significantly for images with 50 mm or coarser showing below 55% (see Table V). Both results indicate that the spatial resolution of 40 mm would be a threshold for grain size analysis. Comparing the two machine learning methods, RF showed significantly better accuracy for spatial resolution smaller than 40 mm ranging from 83.2% to 85.7% compared to SVM range from 60.8% to 86.9% (see Tables IV and V). Therefore, we chose the RF for further analysis.

TABLE V Accuracy Assessment of Tidal Sediment Classification Derived From SVM Using Validation Set

Based on the machine learning classification results, we think that the ortho-rectified image at 40 mm resolution would provide the most efficient UAS survey for grain size analysis of coastal sediments because of the capability of training data selection. Although the images coarser than 40 mm resolution showed an acceptable classification accuracy with 70% or higher, the image texture of different grain size can hardly be recognized. It infers that the classification cannot be operated with trusted training data because the training data cannot be appropriately selected even with the ground truth data for coarse UAS images. Therefore, we suggest 40 mm UAS images for grain size analysis and tested the grain size mapping at 40 mm resolution with two additional rounds of UAS surveys at a flight altitude of 100 m to cover the entire study area.

Based on the assessment of UAS survey efficiency with an acceptable classification accuracy, the grain size analysis for the whole study area was carried out using 40 mm ortho-rectified image acquired at 100 m altitude using Matrice 100 with Zenmuze X3 optical camera (see Table VI and Fig. 6). Because the new images contain vegetation and water areas, they were removed first by using an RF classification with an accuracy of 91.9% and 0.89 kappa. Then, the remaining areas were classified to the four types of sediments, including muddy sand, sand, gravel, and shell. The sediment classification showed an overall accuracy of 89.0% with 0.83 kappa coefficient based on 15570 training and 6671 validation pixels (see Table VI). The RF model picked up Red and Blue bands in the grain size analysis. The results showed that the study area is covered by 32.3% of muddy sand, 14.3% of sand, 45.4% of gravel, and 5.7% of shells, respectively. This result agrees with sediment environment composed of tidal flat intersected by stream channels merging into the ocean, resulting high energy environment along the channels mixed with lower energy environment at an intertidal zone [33], [34].

TABLE VI Confusion Matrix for Sediment Classification of the Study Area Using 40 mm Ortho-Rectified Image

Fig. 6.

Sediment classification map of the study area derived from RF using the 40 mm ortho-rectified image.

UAS-based grain size analysis provides sufficient classification accuracy contingent upon the survey condition. The spatial resolution of UAS image should have sufficient details to detect grain size. If the pixel size is larger than a certain level (40 mm in this study), the accuracy drops quickly because the sediment texture became blurred and less useful. On the other hand, the efficiency of UAS survey should also be considered along with resolution. Although the high spatial resolution has advantages for higher classification accuracy, the common optical sensors require low survey altitude; this results in extended survey time and flight distance. Given the fact that the coastal survey must take consideration of the tidal period, the extended flight time can induce changes in illumination conditions at each flight and different acquisition date for the same area. This case can cause inconsistency in DN values for the same type of sediment, and classification discrepancies might occur (see Fig. 7). Therefore, it is important to maintain the consistent illumination conditions within the short survey time covering larger area without losing the details to detect sediment distribution. In this study, we initially flew the UAS with a 5-m altitude to get high resolution images for experiments. With the findings from the experiments, we adjusted the flight altitude to 100 m to obtain the 40 mm images, which still provided sufficient accuracy. The DJI's phantom 4 Pro can acquire 40-mm image for an area of 500 × 500 m in 46 min 35 s-flight time at 150-m altitude with front overlap 80% and side overlap 70%. The 150-m altitude can obtain images 100 times faster than the 5-m altitude to cover the same area. If higher classification accuracy is required, 10-mm spatial resolution may be an option, while it requires around a 4.6-times longer flight time.

Fig. 7.

Example of problems caused by changes in illumination condition at each flight and different acquisition date for the same area. (a) Inconsistency in DN value. (b) Classification discrepancy.

SECTION IV. Conclusion

This study was designed to identify the optimal resolution of UAS surveys for coastal sediment mapping. The various spatial resolutions of UAS image, including 2, 4, 10, 20, 30, 40, 50, and 60 mm were used as an implication of survey altitude. These UAS images were used for sediment classification, including muddy sand, sand, gravel, and shell in a tidal flat. The image classification was derived from RF and SVM methods using ground truth data as a training set. Classification accuracy was assessed for derivation of optimal survey altitude in terms of classification accuracy and UAS survey efficiency in terms of flight time. The results showed that RF is more efficient for sediment classification than SVM while the general accuracy pattern was similar. The accuracies decreased with a decrease in spatial resolution and then dropped significantly for images coarser than 40 mm. The results infer that spatial resolution of 40 mm would be a threshold for our case study. RF showed better accuracy than SVM in our case study. The RF classification using ortho-rectified mosaic of 40-mm resolution showed overall classification accuracy of 89% with kappa coefficient of 0.84. Although the high spatial resolution has advantages for higher classification accuracy, the higher spatial resolution requires more survey time and lower flight altitude resulting in changes of illumination conditions. If higher classification accuracy is required, 10-mm spatial resolution may be an option while it requires a 4.6-times longer flight time. Given the fact that this study is the first approach, which tested various UAS survey conditions for coastal sediment classification, we expect that this study can serve as a guideline for future UAS sediment mapping in a coastal environment. However, this study dealt with one study site and four types of coastal sediments. Thus, additional studies at various environment and sediment types are required for more generalized use.