Baltic Sea Ice Concentration Estimation From C-Band Dual-Polarized SAR Imagery by Image Segmentation and Convolutional Neural Networks

In this study application of convolutional neural networks (CNNs) preceded by synthetic aperture radar (SAR), image segmentation for sea ice concentration (SIC) estimation over the Baltic Sea from dual-polarized C-band SAR imagery is studied. Three algorithm variants were studied and trained using FMI ice chart SIC or a synthetic SIC dataset with different SIC values generated by combining pure open water and sea ice blocks by applying binary masks. The first two algorithm variants were trained using only open water and fully ice-covered patches, based on the FMI ice charts, and they had a similar CNN structure. These two algorithm variants differ only in deriving the segmentwise SIC from the CNN output. In the third algorithm, variant synthetic SIC data derived as a mixture of open water and fully ice-covered ice patches according to the ice charts were used in training. The estimation results were evaluated with respect to the FMI ice chart SIC for an independent test dataset. The results were very encouraging for operational purposes and significantly better than for our earlier SAR-based SIC estimation algorithms. The algorithm version trained with synthetic SIC data clearly outperformed the two other algorithm versions and our earlier SIC estimation results based on dual-polarized SAR, using independent FMI ice chart SIC as a reference.


I. INTRODUCTION
T HE sea ice concentration (SIC) in high resolution is an important sea ice parameter for sea ice navigation, offshore operations, data assimilation to and validation of numerical weather and sea ice models, and climate research in the form of time series. SIC (C i ) describes the fraction of the water surface covered by ice of the whole area A tot [1] where A ice is the area of ice within A tot and A ow is the area of open water within A tot . SIC can be given either as a pure ratio (range: 0.0-1.0), as a percentage (0%-100%), or in tenths (from 0/10 to 10/10), providing a coarser accuracy, as it is typically expressed in ice charts. Due to its definition ice concentration is dependent on the scale, in a fine enough scale SIC is always either zero (open water) or one (ice). This fact also complicates the direct comparison of different SIC products in different resolutions. SIC difference due to different resolutions, caused, e.g., by different image segmentation, selection of SIC computation polygons, or resolution grid cells as the basic units of the SIC estimation, must be taken into account when interpreting different SIC estimates and comparing them. Space-borne passive microwave radiometers (MWRs) are the major data source for operational SIC estimation [2]- [6]. The operational state-of-the-art MWR SIC estimation algorithm used by the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) Ocean and Sea Ice Satellite Application Facility (OSI SAF) is described in [7].
The major restriction of MWR's for SIC estimation is their poor resolution, typically from several kilometers to tens of kilometers. MWR-based SIC estimates are a common source of observational SIC data used in operational sea-ice data assimilation. For ice navigation, the typical operational data are the ice charts, which represents the visual interpretation of synthetic aperture radar (SAR) data and other possibly available sea ice observations, such as data from optical/near-infrared (NIR)/infrared (IR) instruments, made by ice analysts. Other data used in ice navigation are the direct visual interpretation of the SAR imagery onboard ships. Visual interpretation of SAR imagery, in addition to ice charts, is used for example onboard Finnish and Swedish ice breakers in the Baltic Sea.
SIC in a finer resolution can also be retrieved from instruments capable of measuring surface temperature, i.e., operating at IR or NIR frequencies. However, the waves at the frequencies measured by these instruments do not penetrate the potential cloud cover. Due to relatively long cloudy periods, which can last even several weeks over the Baltic Sea and many other ice-covered sea areas during the wintertime, there exist long temporal gaps in SIC estimates made using these frequencies. Examples of IR/NIR algorithms are MPA (MODIS potential open-water algorithm) [8], developed at the University of Bonn, using the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument data and the algorithm for SIC estimation from Visible IR Imaging Radiometer Suite (VIIRS) onboard the Suomi NPP satellite, as proposed in [9].
Use of single-band SAR for the estimation of ice concentration has been studied, e.g., in [10]- [15]. Automated This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ sea ice classification schemes, which implicitly include ice concentration or an open-water class, based on single-band and dual-band SAR texture and backscattering, have also been proposed, e.g., in [16]- [23]. These methods apply multiple different techniques such as the gray-level co-occurrence texture features [24], Markov random fields (MRFs) [25], and Gabor filters [16], [26] for classifying the sea ice SAR data. Dokken et al. [27] developed an SAR ice concentration algorithm that is a combination of mean ratio (relating average SAR backscatter to typical open-water and sea ice values), a local threshold, and a wavelet method for RADARSAT-1 and ERS C-band SAR imagery. It has also been shown that using dual-polarized C-band SAR data (HH / HV polarization combinations) improves the ice concentration estimation compared to single-channel SAR (HH polarization combination) [28]. A method for ice classification into four ice classes and open water by combining Special Sensor Microwave/Imager (SSM/I) radiometer ice concentration and SAR data was introduced in [29]. Another approach toward data fusion from radiometer and SAR data for SIC estimation has recently been presented in [30] and [31]. This approach is based on a process similar to data assimilation. Algorithms combining MWR data and SAR image segmentation for SIC estimation were also presented in [32] and [33].
Deep convolutional neural networks (CNNs), here shortly called CNNs, have been applied for SIC estimation from SAR in [34] and [35] using Canadian Ice Service (CIS) ice charts as training data. In [36], MWR data were used to train a CNN for SIC estimation. CNNs have also been applied to sea ice classification to ice types and open water in [37]. From such a classification, also, SIC estimates can be derived. Recently, a method combining CNN applied to SAR and upsampled MWR brightness temperatures at the final layer of the neural network with a sigmoid activation has been proposed in [38].
Of the CNN-based methods for SIC estimation, Wang et al. [34] report L1 difference (D L1 ) of 20.6 percentage points (pp) with respect to SIC from CIS ice analysis, which is reported to be competitive with passive microwave SIC, over the Beaufort Sea. For the Gulf of Saint Lawrence region, Canada, the D L1 was 26.5 pp outperforming the MWR SIC estimation over this area. In [35], the bias of −2.7 pp, D L1 of 13.0 pp, and the root-mean-square difference (D rms ) of 22.1 pp were reported for freeze-up data SIC estimation. In [33], the SIC estimates of MWR (ASI algorithm) and the algorithm, based on SAR texture and MWR brightness temperature ratios, over the Baltic Sea were compared to the FMI iced chart SIC. The difference statistics for the MWR were 1.3 pp (bias), 8 (D L1 ) and 19.4 pp (D rms ), and 2.1, 6.7, and 18.3 pp for the combined MWR/SAR algorithm, respectively. Using SAR and MLP for SIC estimation, the corresponding numbers in [33] were 9.3, 13.7, and 24.8 pp. These values of earlier studies can be used as references when evaluating the performance of the proposed methods. In [39], the U-net neural network primarily used for semantic image segmentation has been applied to SAR imagery to estimate SIC using the MWR ASI SIC for training the U-net.
In this study, three different variants of SIC estimation based on CNNs and SAR image segmentation are proposed. The idea behind this study was to study and compare approaches utilizing CNNs and SAR image segmentation in SIC estimation and also compare the results to the SIC estimation method based on SAR image segmentation and explicitly computed SAR texture features studied at FMI earlier [33]. SAR image segmentation systematically extracts and separates the homogeneous areas in the SAR imagery with high accuracy. It can then be assumed that the segments represent uniform sea ice regions or open water. The difference to a similar analysis based on ice chart polygons drawn by ice analysts is that SAR image segmentation is systematic, not dependent on an individual ice analyst, and the image segmentation also has a higher accuracy of segment boundaries and level of detail than in the typical operational ice charts with a limited amount of human resources and strict deadline times. The first and second variants use pure sea ice and open water patches in the CNN training phase. The classes used in the training are based on the digitized FMI ice charts. The first variant then classifies the pixels within each segment into sea ice or open water and, finally, counts the ratio of the number of sea ice pixels to the number of all the pixels in each segment to derive segmentwise SIC estimates. A similar idea applied to ice chart polygons was proposed in [38]. This variant SI/OW is here called SI/OW. The other variant uses the same CNN and training as SI/OW but uses the CNN output, which is trained to be in the range [0, 1], directly as an SIC estimate. The CNN output zero indicates open water and one pure sea ice. The second variant is called SIC-1. The third variant uses mixtures of patches derived from pure sea ice and open water patches in the training. The mixtures were generated using binary masks. In this approach, the SIC of the training data is known exactly. The third variant is called SIC-2. For all the methods, the FMI daily digitized ice chart SICs were used as reference data in the comparisons. In the tests performed, SIC-2 clearly outperformed SI/OW and SIC-1 and also our earlier SIC estimation algorithms based on dual-polarized SAR data.

II. STUDY AREA AND DATASETS USED IN THE STUDY
Our study area, the Baltic Sea (see Fig. 1), is a semienclosed brackish water basin with a seasonal ice cover located in northern Europe, approximately between latitudes 53 N and 65 N and longitudes 9 E and 30 E. The area of the Baltic Sea is 422 000 km 2 , and the average annual maximum ice cover extent is 170 000 km 2 . Many of the harbors in the Baltic Sea are ice-surrounded every winter and precise and timely ice information are necessary for navigation in the ice-covered areas. The winter ship traffic is maintained with the aid of ice breakers. A typical Baltic Sea ice season lasts from November to December until late May in the northern parts (Gulf of Bothnia). The thermodynamic ice growth in the fast ice zone is in maximum around one meter, on average, 72 cm [40], in the northern Gulf of Bothnia. In deformed ice areas, ice thickness can be several meters, in ice ridges even 25 m [41], [42]. The maximum ice extent in the Baltic Sea is typically reached in February and March.
The digitized FMI ice chart SIC fields have been used in this study to derive training data for the SIC estimation algorithms and as reference data for evaluation of the proposed algorithms. In ice charts, SIC is estimated by ice analysts for polygons they draw on a map base according to their interpretation of the ice conditions. Each polygon represents an ice type or multiple ice types, which can uniquely be described in terms of the ice charting guidelines provided by the World Meteorological Organization (WMO) [43]. SIC is also assigned to each of these polygons; in the ice charting based on the WMO guidelines, one polygon can involve more than one ice type and, thus, also multiple concentrations of these multiple ice types. In ice charts, these all are typically indicated by the egg code assigned to each polygon [1]. The FMI ice charts over the Baltic Sea are made by ice analysts daily during the winter period. The input data for making the FMI ice charts are satellite data from multiple instruments, including SAR (Sentinel-1, RADARSAT-2, and TerraSAR-X/TanDEM-X) and optical/IR data (MODIS and VIIRS), observation data from coastal observers, observations from the Finnish ice breakers, and the FMI operational sea ice forecast model results. The most important data source is the Sentinel-1 C-band SAR images. In the FMI ice charts, the ice analyst first locates the areas with homogeneous ice conditions that are presented by polygons in the ice charts. For each ice chart polygon, five attributes (ice concentration, ice minimum thickness, ice average thickness, ice maximum thickness, and degree of ice deformation) are assigned by the ice analyst. In this study, we have used the ice concentration attribute of the digitized and rasterized FMI ice charts in a grid of a 500-m resolution in the latitude-longitude coordinate system. In FMI Baltic Sea ice charts, the given ice concentration is the total ice concentration of the polygon given in percent, not in tenths as in the ice charts according to the standard WMO guidelines (in the included egg diagrams). In FMI Baltic Sea ice charts, neither partial ice concentrations nor the stage of ice development is given. Instead, the five attributes listed above are assigned to each polygon; for open water areas (polygons), the sea surface temperature (SST) attribute is assigned.
The SAR data used in this study were Sentinel-1 extra wide (EW) swath Ground Range Detected Medium (GRDM) resolution dual-polarization mode data [44]. The two channels of the images represent the HH and HV polarization combinations, where the first letter indicates the transmitted polarization and the second letter indicates the received polarization, H is horizontal polarization, and V is vertical polarization. The swath width of the used acquisition mode was about 400 km. The SAR data consisted of 650 Sentinel-1 images. The monthly distribution of the imagery is shown in Fig. 2. Of the April 2019 images, 30 were used for training, and the remaining 620 images were used as the test dataset. The training images were selected randomly, and they represent both cold conditions and ice melt conditions. The reference dataset for all the experiments was the daily digitized FMI ice charts of the same day with the SAR acquisitions. The FMI ice charts were used to generate the training datasets and as reference data in the comparisons evaluating the performance of the algorithm variants.
The SAR data were calibrated and georectified to Mercator projection with WGS84 datum and 61 • 40' reference (correct scale) latitude, and the logarithmic σ 0 values were quantized to eight bits per pixel (8 bpp), such that, for the HH channel, σ 0 of −30 dB or less corresponds to the pixel value of one and 0 dB to the pixel value of 255. For the HV band mostly with a lower σ 0 , the corresponding values were −40 and 0 dB. The pixel value zero was reserved for background (no data and land mask). The land masking was performed based on a land mask derived from the Global Self-consistent, Hierarchical, High-resolution Geography Database (GSHHG) coastline dataset [45] applied to the georectified SAR images. Based on earlier experience, this quantization preserves the SAR texture well and with sufficient accuracy for automated classification. With the channelwise quantization ranges defined as above, there will not appear large areas of pixels saturated to the upper (0 dB) or lower boundaries (−30 or −40 dB). A similar quantization scheme applied to dual-polarized (HH/HV) C-band SAR data has earlier been used for example in [46]  (for Radarsat-2 data) and [33] (Sentinel-1) in the context of SAR texture-based SIC estimation. The 8-bpp data were then downsampled to the resolution of 500 m. Also, the HH/HV channel cross correlation was computed. The HH and HV channels and the HH/HV cross correlation were used as three input channels to the CNNs.

III. METHODOLOGY
For image segmentation, the mean-shift (MS) algorithm [47] was first applied to locate the modes of the 2-D (HH and HV channels) 500-m resolution SAR data. The MS algorithm has been empirically adjusted such that about 10-15 modes will be produced after convergence. The initial 10-15 categories based on MS were then used as a starting point for iterated conditional modes (ICMs) segmentation [48]. After the ICM segmentation segments, smaller than 100 pixels were merged to the neighboring segment with the closest 2-D (vectors consisting of HH and HV pixel values) vector median [49], computed within each segment. This merging was performed to reduce the random variation due to speckle and to provide large enough units for reliable SIC estimation. This is the FMI standard segmentation procedure for the FMI operational Copernicus Marine Environment Monitoring Service (CMEMS) SAR data.
Two very similar variants of CNN were applied in the three algorithm variants included in this study. The structure of the second applied CNN variant is shown in Fig. 3; the first CNN variant is similar, except that, in it, the last layer has a sigmoid activation function instead of a linear one. The CNN applied had three convolution layers and one sigmoid layer and one linear (output) layer, as shown in Fig. 3. The sigmoid activation function in the fully connected layer was used because it gave good classification performance (SI/OW) in preliminary experiments with a relatively small number of training and validation samples. The use of a computationally simpler rectified linear unit (ReLU) activation function would have reduced the training and estimation time slightly. In the convolutional layers, the ReLU activation function and the maximum pooling (maxpool) were used. The maxpool sizes were 5 × 5, 5 × 5, and 3 × 3 from the input layer toward the output layer. The numbers of the convolution units at the three layers were 64, 64, and 128, respectively, and the number of processing units in the fully connected layer was 1024. The Adam (adaptive moment estimation) [50] optimizer was applied in the training. The inputs to the CNN are 32 × 32 pixel windows (a number divisible by four because of twice downsampling by two in the CNN) of three channels. The window size was selected such that it is large enough to contain information on SIC context and, on the other hand, small enough to enable high-enough resolution SIC estimation. This selection is always a compromise between the desired resolution and the contextual interpretation capability. The values in the three input channels were the calibrated HH and HV quantized 8-bpp SAR backscattering coefficients and the HH/HV channel cross correlation. The channel cross correlation, denoted by C c here, at the location (row, column) = (k, l) between the SAR HH and HV channels, denoted by I HH and I HV , respectively, is where N (N = 13) is the number of pixels within the computation window W , i.e., including pixels with a distance D ≤ R from the center pixel, and R = 2. μ HH and μ HV are the means of I HH and I HV , and σ HH and σ HH are the standard deviations of I HH and I HV , within the computation window. The cross correlation is quantized to eight bits such that the value C c = 0.0 corresponds to the pixel value of zero and C c = 1.0 corresponds to 255. An example of an input RGB image combining the quantized 8-bpp HH, HV, and C c images is shown in Fig. 4.
The two variants of CNN applied in this study differed only in the final layer: in the first one, the final layer had a sigmoid activation function, and the binary cross-entropy was used as the loss function. The two classes of open water (output = 0) and sea ice (output = 1) were trained for this CNN variant. In the second CNN variant, the output layer was linear, and the mean-squared error (mse) was used as the loss function. The SIC value varying from zero to one was trained for this CNN variant.
Based on the first CNN variant, two algorithms to estimate the segmentwise SIC were studied. The first one was to count the number of sea ice pixels (CNN output ≥ 0.5) within each segment and compute their ratio to the total number of pixels within the segment. The segmentwise SIC estimate of the first algorithm variant is also the ratio of the sea ice pixels to the total number of pixels within a segment.
The other approach to estimate SIC utilizing the first CNN variant was to use the CNN output, varying between zero (open water) and one (pure sea ice) directly as a pixel SIC estimate, and then use the segmentwise median of these pixelwise SIC estimates as the segmentwise SIC estimate. The first CNN variant was trained by selecting the training (and The second CNN variant training and validation datasets were also generated based on the FMI ice chart SIC by first selecting pure open water and sea ice windows as above. The final training and validation datasets were generated by combining open water and sea ice areas and applying binary masks to produce mixtures of open water and sea ice. With this approach, the SIC of each training data block is exactly known. The training data were balanced such that there was the same number of training samples representing each SIC tenth (0/10-10/10). Binary images to generate binary masks applied to make mixtures of open water and sea ice windows are shown in Fig. 5. These binary images were generated by randomly drawing ellipses and rectangles of different sizes and at different orientations. To extract the random binary mixture masks, a 32 × 32 pixel window was placed randomly over a randomly selected binary image. Based on a binary data mask, a training data sample is generated by setting the pixels of a 32 × 32 window to the values of a randomly selected open water window from the earlier generated dataset consisting of 32 × 32 pixel open water and pure ice data windows if the mask pixel has a value zero (black) and to the values of pure sea ice window otherwise (i.e., zero-valued or white mask pixels). In addition to the generated training datasets, a smaller validation dataset was generated for both the CNN variants. The sizes of the validation datasets were approximately the size of the training dataset divided by ten.
The validation datasets were used to find an optimal number of training epochs.
In the CNN training phase, the best weights were kept after each epoch. Data augmentation was applied with rotation in 22.5 • steps and horizontal/vertical flip. The dropout rate of 0.25 and the learning rate of 0.0001 were applied in the training of both the CNN variants applied in this study.
The CNN parameters were selected experimentally. The number of neurons at the layers was selected by starting with a large number of neurons and reducing them until the performance in training, based on the validation data, started to decrease and then selected the numbers of neurons such that they were well above this threshold of decreasing performance. Adding neurons above the threshold did not actually increase the performance anymore. The numbers of neurons could be fine-tuned at different layers of the CNNs to the minimum number with a desired estimation/classification performance to minimize the execution time. However, the aim of this study was not the optimal execution time. The execution times for SIC estimation with the proposed setups were reasonable. The execution time optimization may become necessary if higher SAR and product resolutions will be used on a desktop or laptop computer.
The three different algorithm variations are also referred to here as SI/OW, SIC1, and SIC-2. SI/OW is the algorithm version counting sea ice pixels within each segment, SIC-1 is the algorithm version using the SI/OW classification CNN output value in the range [0, 1] directly as an SIC estimate, and SIC-2 is the version estimating SIC trained using synthetic SIC windows based on known open water and pure ice mixtures generated by applying the binary masks.
The numbers of training and validation data (before data augmentation) for SI/OW and SIC-1 were 8828 open water class and 8692 sea ice class 32 × 32 pixel sample windows for training and 802 and 790 windows for validation, respectively. The minimum loss value in training of OW/SI and SIC-1 was approached after 20 epochs. The effect of the number of training samples was evaluated for SI/OW and SIC-1 by testing training with a smaller number of training samples. The validation dataset size was about one-tenth of the training dataset. After exceeding the number of about 2000 training samples, the loss function did not show any significant decrease when increasing the number of training samples anymore. Based on these tests, it can be concluded that the number of training samples used in the experiments is large enough. For SIC-2 with a much larger variety of input data structures, significantly larger training and validation dataset sizes were used. For SIC-2, there were 435 942 training samples and 48 437 validation samples, and the numbers of training (and validation) samples were balanced between each tenth (1/10-10/10) of SIC values. The minimum of the loss function for the SIC-2 validation dataset was found after seven training epochs. For the first CNN variant, the used loss function, i.e., cross-entropy, for the validation dataset after convergence (selected set of CNN weights) was 0.04, and for the second CNN variant, RMSE (used as a loss function) for the validation dataset after convergence was 3.47 pp.

IV. RESULTS
The SIC values produced by the algorithm CNN variations were compared to the SIC based on the FMI ice charts to evaluate the performance of the SIT estimation. In the comparisons, the following measures of difference between the SIC estimates and the reference SIC of FMI ice charts were used: are the values of the reference SIC data (from gridded FMI ice charts) at the same location as X est i . D L1 is the L1 difference, and D sgn is the signed L1 difference giving the estimation bias, positive bias indicating overestimation, and negative bias indicating underestimation. D rms is the root-mean-square difference. In practice to evaluate the performance, we randomly selected a large  Tables I-III: D L1 in Table I, D sgn  in Table II, and D rms in Table III. A sample giving a general daily view of the SIC estimation by the proposed methods and the FMI ice chart SIC over the Baltic Sea (February 1, 2019) is shown in Fig. 6. The figure is a mosaic over the Baltic Sea consisting of several separate SAR SIC estimation products. The SIC mosaics were generated by overlaying the most recent available SIC estimation results, corresponding to the date  the grid cells may also represent the previous day or even the day before it, but the situation is the same in ice charts where the most recent data are also used. The results indicate good accuracy in SIC estimation compared to SIC of the FMI ice charts. The best results were achieved by the SIC-2 method. The SI/OW method produced some better estimation results than SIC-1. One reason for the result that SI/OW gives a better comparison metric is that most of the segments represent either open water or pure sea ice, and SI/OW gives accurate SIC estimates for most of these segments. Instead, SIC-1 estimates of open water or pure sea ice segments often tend to slightly overestimate open water SIC and underestimate pure sea ice SIC. SI/OW is also good for locating segments of pure open water or sea ice, as expected based on the training. It also seems that SIC-1 overestimates SIC, mainly due to the overestimation of low SIC and open water areas. Over the sea ice with higher SIC, the estimates better correspond to the FMI ice chart SIC. SIC-2 gives the most realistic SIC estimates, also for areas with a higher SIC value with respect to the FMI ice chart SIC. When comparing the best method, i.e., SIC-2, to the values from the reference literature, it seems to outperform the methods based on solely SAR data and even being very close to the method using both SAR and MWR data [33]: D L1 = 6.7 pp for SAR/MWR, D L1 = 8.8 pp for SIC-2, D rms = 18.3 pp for SAR/MWR, and D rms = 21.9 pp for SIC-2.
We additionally provide two sample cases with a wide SIC range to better visually characterize properties of the studied methods. One of the sample cases is over the Gulf of Finland and the other over the Gulf of Bothnia. These cases with the corresponding SAR channel images are presented in Figs. 7 and 8. These figures contain the SAR channel images, the FMI ice chart SIC image, and the SIC estimation results of all the three studied methods. In these figures, the slight overestimation of SIC of open water and underestimation of SIC of pure sea ice is visible. There also exist dark patches in the SAR images, e.g., in the northern part of the Gulf of Finland in Fig. 7, which are interpreted as open water or very low SIC sea ice by ice analysts and high SIC sea ice by the algorithms. These are actually very difficult cases either for a human or an automated classifier because both open water and smooth thin sea ice appear very similar in SAR imagery, especially in low wind conditions. Similar differences can be seen in the western part of Fig. 8. Also, the larger number of details and more detailed segment boundaries of the image segmentation with respect to the ice chart SIC (c) can be seen in the figures.
According to this study, there was no significant variation in the monthly difference statistics of Tables I-III. The differences were slightly higher during the midwinter, in February and March. On the other hand, the differences were rather low also during the melt period (April and May); this is probably due to the training dataset containing also melt period data of late April.
The estimation of different SIC categories for SIC-2 was also studied, expressed as tenths in the ice charts, with the ice chart FMI SIC converted to tenths from the percentages by assigning each percentage to the closest tenth. The same dataset of N s = 2 250 000 samples was used in this experiment also. It should be noted that most of the ice chart SIC values were either 0% (water) or 100% (ice), and only a relatively small amount of data was assigned to the other nine (intermediate) tenths according to the ice charts. The relative amount of SIC values according to the FMI ice charts of the intermediate tenths varied from 0.3% to 14%, the tenths 5/10 and 6/10 having the smallest proportions. The 9/10 category had the 14% proportion, but water and ice categories together represented almost 70% of the data. The result of the comparison can be seen in Fig. 9. In the figure, the cyan line represents the average estimated SIC for each category, the blue lines represent the average with the standard deviation added and subtracted. It can be seen that the lower SIC categories clearly tend to be overestimated, and the higher SIC categories were slightly underestimated. This was also confirmed by the bias computed for the categories; it was 6 pp for water, around 20 pp for 1/10-4/10, and then −9 pp for 5/10, 0 for 6/10 and negative for the higher SIC categories, and up to −12 pp for the ice class. The pattern was similar for SIC-1, except that the standard deviations were  significantly higher (about 1.5 times compared to SIC-2 shown in Fig. 9).
A confusion matrix with a reduced number of five SIC categories is shown in Table IV. The SIC categories used   TABLE IV   CONFUSION MATRIX OF COARSE SIC CATEGORIES. THE ESTIMATED SIC  RANGES ARE IN THE HORIZONTAL DIRECTION, AND THE FMI ICE  CHART SIC RANGES ARE IN THE VERTICAL DIRECTION to compute this confusion matrix were 0%-20%, 20%-40%, 40%-60%, 60%-80%, and 80%-100%. The confusion matrix was computed using the FMI ice chart SIC as reference data as in all the other comparisons. The SIC-2 confusion matrix and Fig. 9 both indicate that low and high SIC areas are estimated well by the SIC-2 algorithm. In the intermediate SIC values, there is more deviation and also rather large portion of low and high SIC values deviating from the reference SIC value. One reason for this behavior is that the automated algorithms using image segmentation, in general, provide more detailed blocks (segments) than the ice chart polygons and, thus, reveal more details of the SIC distribution within the ice chart polygons still giving reasonable on average SIC correspondences. Part of these deviations is naturally due to SIC estimation inaccuracies both by the algorithms and the ice analysts, but these are very difficult to distinguish from each other.

V. CONCLUSION AND DISCUSSION
In this study, the estimation of SIC from dual-polarized Sentinel-1 SAR data over Baltic Sea by applying CNNs was explored. The results were promising, and the methodology will be a potential candidate for future operational SIC estimation to replace the algorithm currently in use [33]. The estimation results, especially for SIC-2, were comparable or better to the other CNN-based SIC estimation algorithms and better than our previous SAR-based algorithms. In fact, the estimation accuracy was even close to our earlier algorithms utilizing both SAR and MWR. The SI/OW method is suitable for locating segments of pure ice or open water, and the SIC-2 is the best alternative for segmentwise SIC estimation, clearly outperforming SI/OW and SIC-1. Also, comparisons to FMI ice chart SIC in the intermediate SIC range (0% < SIC < 100%) were promising. Even though there exist quite many values of 0% and 100% SIC over these segments, their proportions are reasonable. It should be taken into account that the segments provided by an automated image segmentation usually have a higher level of detail, and the ice chart SIC of larger polygons can be interpreted as an integral of the smaller segment SICs within each polygon. It should also be noted that the ice chart SIC is based on the interpretation of an ice analyst on duty. The experiment made in [51] clearly indicates that there exist differences between the SIC estimates provided by different professional ice analysts.
There exist several sources of error affecting the estimation results with respect to the reference data, i.e., FMI ice chart SIC. The spatial accuracy of the ice charts and level of detail at the polygon boundaries are not as high as provided by the automated image segmentation algorithm. This results in increased difference measures at the differing segment/polygon boundaries. Often, there also exist several segments with different SIC estimates within an ice chart polygon, thus increasing the difference measures. One factor that should also be taken into account and affecting the difference measures is that the FMI ice chart SIT is daily and still used as reference data for all the SAR images acquired during the same day. This will present a time difference between the SAR and the reference data, and this time difference is typically several hours. Even though in the Baltic Sea ice is typically not drifting very fast, these time differences also have their increasing contribution to the difference measures.
The computer used for the study was a common desktop PC with the following hardware; AMD Ryzen 5 with a 3.6-GHz clock-rate CPU having four CPU cores, 16-GB RAM, and Radeon RX 580 GPU running Ubuntu 18.04 Linux OS. The software used was keras under python-3, and GPU processing was implemented by using PlaidML [52]. The execution times of a few minutes in maximum for the whole processing chain of a typical SAR scene were reasonable for operational use.
In the future, new image segmentation algorithms to replace our current operational ICM SAR image segmentation algorithm will be studied. The potential alternative is image segmentation methods based on deep neural networks. Next, the use of U-net [39], [53] for SAR segmentation will be studied. Also, the integration of MWR data in the SIC estimation with CNNs will be studied, and best practices will be adopted.
The major conclusion is that the studied methods show great potential for operational sea ice monitoring, and after more thorough training, using data covering a longer time span (preferably over multiply winters to get a representative training dataset for all possible ice conditions), it then can be included as part of the FMI operational SAR production chain.
The resolution of 500 m is a bit coarse for all the ice navigation. Ice navigation is one of the major target applications of the Baltic Sea SIC estimates. A preferable resolution for ice navigation would be closer to a typical ship scale, i.e., around 50-100 m, which is already available in SAR imagery. However, there exists a paradox related to the high resolution: as the resolution gets higher, also smaller local changes (deformations) in sea ice cover become visible, and typically, their time scale is shorter than that of larger scale deformations. This leads to the requirement of shorter delivery times to get timely information on the local ice conditions. On the other hand, the amount of data and also transmission times increase as a function of the improved resolution. In the future, also, the use of higher resolution data in SIC estimation with CNNs will be studied. However, to be able to run CNN algorithms in, e.g., a 100-m resolution will require more parallel processing, either utilizing computers with multiple CPUs or efficient graphics adapters (GPUs).