Development, assessment and implemenation of an automated gap filling method for tide stations with dual water level sensors

High quality water level time series are important for providing tidal datums for littoral boundaries, long term relative sea level trends, information for navigation as well as emergency management information during tropical storms and hurricanes. Gaps occasionally appear as a result of gauge malfunction, transmission problem or damage to the station. Gaps can be minimized by collocating redundant sensors. Dual water level measurements are typically collected by sensors based on different physical principles, such as, acoustic and pressure based measurements for this study. The resulting differences in measured values must be accounted for prior to filling gaps. Presently, gap filling of water level time series is primarily performed by an expert using a hierarchy of techniques including linear fits, back-up data, least squares fit or nearby station records selected based on the length of the gap and ocean conditions. This paper presents an automated gap filling method and the assessment of its performance for gaps of six minutes to 120 hours for a set of 15 tide stations in different coastal settings, such as, open coast, embayment, lagoons, and ship channels. In this method, a fourth order polynomial is calibrated over a period of 30 days around the gap to be filled. The 30 day calibration period was selected to include a tidal cycle and several weather events such as frontal passages. Longer calibration periods did not improve the performance of the method. The fourth degree polynomial was found to be more flexible than a simple linear regression allowing for the modeling of shifts in the relationship between the two sensors particularly during events, such as, larger waves or higher winds. In particular a fourth order polynomial was found to improve upon the accuracy of lower order polynomials for the modeling of the highest and lowest 1% of water levels. The statistical significance of the improved accuracy gained from using a fourth order polynomial was established based on a t-test. Other steps of the method include verifying that the selected data is at least 90% gap free and detecting and removing potential outliers by comparing the primary measurements with the corresponding values modeled with a simple linear model based on the back-up measurements. The comparison between modeled and measured values further allows detecting potential drifts in the instruments identified by an usually large number of outliers or an overall poor average performance of a linear model. A final improvement to the gap fill is produced by adding a linear correction based on the difference between the fourth order model and the primary water level on each side of the gap. The method's accuracy is compared with four other gap filling methods, (1) linear interpolation across the gap, (2) linear interpolation of the surge component, (3) back-up data with a simple offset, and (4) fourth order polynomial fit without further adjustments. The accuracy of the methods are compared by randomly generating 1,000 gaps of increasing lengths from six minutes to 120 hours at random locations in one year data sets and filling the gaps using the methods described above. The new method has the best gap filling performance for all stations and all gap lengths. Mean Absolute Errors (MAE) are lower than 0.010 m for 13 of the stations for gap lengths up to five days. For two of the stations, Texas Point and Galveston Bay Entrance/North Jetty, the MAE is below 0.010 m only up to six hours and nine hours, respectively. MAEs for the longest tested gap, 120 hours, are 0.017 m and 0.011 m, respectively. These two stations are sentinel stations designed to sustain the storm surge and conditions of hurricanes up to category four. The lower performance of the gap filling method is attributed to the stations' longer stilling wells leading to larger and varying temperature gradients presently not accounted for in the acoustic measurements. Identified oceanic and atmospheric forcing influencing the relationship between the primary and backup measurements include wave climate, water and air temperatures, and wind. The algorithm is now implemented as part of a mostly automated data processing system. The gap filled data, using proven scientific methods, is considered to be of the same quality as the observed data for datum computation purposes.