Feasibility Study for the Fine Crack Width Estimation of Concrete Structures Based on Fiducial Markers

Crack width estimation through remote imaging and digital image processing are the leading diagnostic technologies used for infrastructure. However, owing to limitations in the accuracy of these techniques, alternative methods to detect target crack widths larger than the diagnostic standard are being developed. In this study, the coordinates of fiducial markers were extracted using the Harris corner detector, and fine crack widths of approximately 0.3 mm or less were estimated using the ground sample distance, varied numbers of adjusted pixels, and varied focal lengths. The results of these examinations were compared with the measured values. As a result, it was confirmed that the number of pixels rather than the ground sample distance was the dominant factor affecting the accuracy of the estimated value, and the accuracy could be improved by adjusting the number of pixels, increasing the focal length, and ensuring an effective object distance. The results showed the feasibility of estimating fine crack widths with a high accuracy through the fiducial-marker-based method under the aforementioned conditions. In the future, if the number of crack samples is increased and more detailed focal length and object distance data are supported, this method may provide estimations with a higher accuracy.


I. INTRODUCTION
Digital technology has recently been introduced into the overall field of construction, and the efficiency and productivity of infrastructure management have been promoted as a result [1]. In the field of maintenance in particular, various digital technologies such as artificial intelligence, robots, unmanned aerial vehicles (UAVs), and the Internet of Things have been to utilized to overcome problems caused by aging facilities [2], [3]. Because UAV can freely navigate three-dimensional space, it is possible to acquire images or videos of structures that are restricted from human access using on-board imaging equipment. Therefore, systems for The associate editor coordinating the review of this manuscript and approving it for publication was Tomasz Trzcinski . the efficient maintenance, quantitative performance evaluation, condition evaluation, reinforcement, and safety management of infrastructure facilities using UAVs are being actively developed [4]- [6].
Prior to the development of such technologies, studies on cracks in concrete using remote imaging equipment were mainly focused on crack detection or 3D modeling owing to the various advantages provided by these techniques [7]- [10]. Recently, there has been an increasing trend in research using additional equipment such as laser scanners, ultrasonic distance measurements, light detection and ranging (LiDAR) systems, global positioning systems (GPS), and fiber Bragg grating (FGB) sensors [11]- [13]. However, when additional equipment is mounted on a UAV, there is a limitation associated with acquiring images of a quality above a certain level unless sufficiently a stable flight performance is achieved [14]. Because each method has pros and cons and the weight of the equipment differs, various operating conditions must be considered [15]. When constructing a 3D model through scanning, GPS is used for image registration. However, GPS signals are limited when scanning under a bridge. To overcome the limitations of GPS, various studies have been conducted to supplement modeling based on machine learning using flight information such as the flight path and speed [16].
Research on crack detection and the visualization and modeling of cracks has also been actively studied [17]- [20]. Various techniques, including machine learning, have been introduced for the analysis of crack images [21]- [24]. However, the verification of fine cracks (0.3 mm or less) has only been considered in a very limited capacity [25]. In particular, methods that rely on digital image correction should be carefully verified when targeting fine cracks that are influenced by unit pixels [26], [27]. Image processing techniques can improve image quality, but they may produce distorted results compared to the original image [28].
A more conservative approach is required if the target is cracks in facilities such as bridges and high-rise buildings, which can lead to serious casualties in the event of collapse. Countries world-wide stipulate the allowable crack width of concrete in design standards or specifications considering the load resistance, durability, and serviceability of structures. In the Republic of Korea, this metric is stipulated as 0.3 mm in a wet environment [29]. Therefore, to utilize remote imaging and image processing technology for the precise safety diagnosis and inspection of facilities, it is necessary to collect and process information concerning fine cracks of approximately 0.3 mm or less.
To acquire crack information, it is necessary to know the pixel size of an image according to the shooting distance, which is called the ground sample distance (GSD) [30]. However, because the resolution and GPS accuracy of the imaging equipment mounted on a UAV are not intended to collect such precise target information, it is difficult to determine the distance to the target and calculate the accurate GSD. However, if the acquired crack image contains an indicator with known geometric information, the distance information of the crack can be estimated based on the GSD of this indicator. In addition, if there is a periodic mark for which the size or distance are known that was installed during construction or for maintenance purposes after completion, it will be easier to obtain crack information through remote imaging.
In this study, crack widths are estimated from fine crack images that include fiducial markers. The GSD is calculated by extracting the coordinates of the marker, and the estimated crack width is compared with the actual crack width. In addition, the estimated results based on crack size, object distance, pixel count correction, and focal length adjustment are considered, and the valid conditions for and feasibility of this technique are discussed.

II. METHODOLOGY FOR CRACK WIDTH ESTIMATION
To obtain the geometry of an object from an image, the GSD, which indicates the size of the pixels within the image, is essential. The GSD is usually obtained from the imaging distance or a reference object included in the image. When using a reference object, the coordinates, length, or size of the object must be known. This section considers crack width field measurements, crack image acquisition using a fiducial marker as a reference object, and the process of extracting coordinates from an object through a detector for GSD calculations.

A. MEASUREMENT AND DIGITAL IMAGE ACQUISITION
Cracks that can be identified and subsequently accessible for visual measurement and image acquisition for substructures of small road bridges or pedestrian bridges in Daejeon were explored. Field measurements were performed on various types of cracks in the piers and abutments in this region. A crack gauge was used to measure the crack width, as shown in Fig. 1(a). The crack gauge was equipped with a 10x magnifying glass and 12 measurement scale models were used including the standard scale. For field measurements, a 0.0635-mm scale was selected, as shown in Fig. 1(b). An arrow indicator in a direction perpendicular to the crack direction was used to identify the location of the measured crack and acquire an image. This is because cracks have different directions and widths depending on the detailed location, so it is necessary to determine the detailed location to be subject to visual measurement and image processing. The measurement of each crack was performed based on the naked eye, but auxiliary imaging equipment was used if necessary. Fig. 1(c) shows a close-up image obtained with a magnifying glass. Cracks of various geometries and widths were measured, and the crack widths of the samples used for analysis were 0.127, 0.227, and 0.308 mm.
Images of the measured cracks were acquired using a camera and laser ranger, and the photographed foreground is shown in Fig. 2(a). A fiducial marker for coordinate extraction and GSD calculations was attached near the crack surface. The marker used was a chessboard shape in which squares with a side length of 20 mm were arranged in a 3 × 3 array, as shown in Figure 2(b). To minimize noise and errors depending on the condition and illuminance of the crack surface, as well as to obtain images of a constant quality, imaging was carried out at noon on a sunny day when the crack surface was shaded.   The camera was a Sony A7R IV model, which is an SLR-style mirrorless camera equipped with a full-frame of 35.7 mm × 23.8 mm in size and a 61 million (9504 × 6336) pixel image sensor. The detailed specifications of the camera and built-in sensor are listed in Table 1 [15].
The Planner T 50-mm standard single-focal lens from Sony was used. To measure the imaging distance, an aluminumlevel staff and a three-axis laser leveler used for leveling, and the object distance, which is the distance between the crack surface and camera sensor, was measured. Imaging was performed repeatedly by adjusting the distance in 200 mm increments within the maximum distance limit of 2 m.

B. EXTRACTION BY A HARRIS CORNER DETECTOR
To easily extract feature points from an image, changes in the shape, size, or position of the target object should not affect its identifiability. Even if the viewpoint and lighting of the camera is changed, the corresponding point should be easily determined. The best feature point that satisfies these conditions is a corner point, and most feature point extraction algorithms are based on corner-point detection [31].
Harris Corner and Shi & Tomasi detectors are representative methods for tracking feature points [32]- [35]. The Shi & Tomasi detector can obtain similar results to the Harris corner detector when considering the minimum pixel brightness and has the advantage of a fast processing speed. However, it is necessary to define the corner points and edges according to the sizes of the minimum and threshold values. Another method for extracting feature points is a scaleinvariant feature transform (SIFT) detector that blurs the image by applying a Gaussian filter based on the scale-space theory and expands the scale of the image [36]. SIFT is evaluated to have excellent feature point extraction results, but the image stitching efficiency according to the extraction parameters is different, and it takes a relatively long time to calculate because the extraction target is wide and requires vast calculations. Therefore, speed-up robust feature (SURF) has been proposed to solve this problem. In addition, features from accelerated segment test (FAST) detectors optimized for feature extraction speed and adaptive and generic accelerated segment test (AGAST) detectors with improved FAST performances may be used, and detection operators and algorithms optimized for arbitrary purposes are continuously being developed and supplemented [37].
The Harris corner detector is computationally time consuming because it is based on the Moravec corner detector and autocorrelation method [38], [39]. When many isolated points are present, such as in salt-and-pepper noise, accurate corner detection is difficult [40]. In addition, because a Prewitt window is used, this method is vulnerable to diagonal components [41]. However, because the fiducial marker included in the crack images in this study is a shape that is easy to identify according to the imaging distance, camera point-of-view, and lighting, it is not sensitive to noise and the diagonal components are minimized. Therefore, in this study, the coordinates of the markers were extracted using the Harris corner detector, and the GSD for estimating the crack width was subsequently calculated.
The Harris corner detector is a modified and supplemented version of Moravec's method, and it determines the presence of a corner by moving a small square box window up, down, left, and right on the object and analyzing the changes in pixel values in the image [42]. For Harris corner detection, three cases should be considered, as shown in Fig. 3: (1) when the window is located on a flat area inside of the object, the pixel value in the mask is always kept constant.
(2) When the mask is located on the boundary of the object, the pixel value changes according to the left and right movements of the window; therefore, it can be confirmed that a boundary is present. (3) When the mask is located in a corner, the change in pixel values can be determined by moving it in any direction (up, down, left, or right), and the part with the highest rate of change can be extracted as a corner.
The amount of change E of the sum of squared differences (SSD) of an image due to the shift (u, v) in the above cases can be expressed in a mathematical form composed of a window function and image intensities, as shown in Equation (1).
Assuming that the shift value is very small and may be approximated by the Taylor expansion, Equation (2) is derived. Here, ω (x, y) is the Gaussian window, I (x, y) is the initial intensity, and I (x + u, y + v) is the shifted intensity. To maximize the E function, the intensity square term must be maximized. A symmetric matrix M , which is a structure tensor, is given as in Equation (3), and two eigenvalues and corresponding eigenvectors can be obtained via eigenvalue decomposition: where I x and I y are the derivations for the x and y directions of the image, respectively. λ 1 and λ 2 are the eigenvalues of the matrix M . These eigenvalues indicate the amount by which the image varies E in the direction of the corresponding eigenvector. If both are large, the location can be determined as a corner point; when both values are small, a flat area is indicated; only when one value is large and the other value is small is an edge area present. The corner response R obtained by eigenvalues is defined by Equation (4): where k is an empirical constant and ranges from 0.04 to 0.06. If R> 0, a corner point is present, while if R < 0, an edge is indicated. If |R| is a very small value, flat area can be determined, regardless of the sign of R. The result of plotting the corner response value R as a region for each eigenvalue is shown in Fig. 4. The dotted line on the left of Fig. 5 shows a flow chart of this study indicating the process of acquiring an image and subsequently calculating the crack width using the Harris corner detector. First, the acquired image is converted into a gray image, and a region of interest, including a marker, is determined from the entire image to shorten the computation time. Four corner points are then extracted by applying the Harris corner operator to the marker of the set region of interest, and the GSD is then calculated. Finally, the crack width is calculated and compared with the image processing result obtained using the vision sensor presented in a previous study [15].
The right side of the flowchart shows an image processing process that is not based on markers. First of all, the image  was improved and smoothed through a preprocessing image process in which the target was clarified and a median filter was applied. After specifying the threshold, the image was converted to black and white using the Otsu method, and the foreground was separated from the background using an area filter that considered the proximity and connectivity of neighboring pixels. Finally, cracks were determined from the maximum area values. Fig. 6 shows the process and step-by-step results of the marker extraction method achieved by applying the Harris corner detector to the acquired image. First, an input image including the target crack and marker was acquired as shown in Fig. 6(a). The acquired image was then converted into a gray image, and the state of setting the region of interest (ROI) is shown in Fig. 6(b). Fig. 6(c) shows the state in which the reference coordinates are set, and the area is set for feature point extraction from the ROI. Finally, using the Harris corner detector, the coordinates around the edge and boundary were detected, and the results from extracting the VOLUME 9, 2021 TABLE 2. GSD and crack width (C r = 0.308 mm). coordinates closest to the corner are shown in Fig. 6(d). Errors caused by the tilt angle with respect to the optical axis and the focal length of the lens were up to around 1% or less, so additional image processing or camera calibration was not considered [43].

III. RESULTS AND DISCUSSION
The crack width was estimated based on the GSD of the marker, and the estimated results according to the adjusted number of pixels and focal length change were compared with the actual crack width. In addition, the main factors affecting the results when estimating the fine crack width using fiducial markers, methods to increase the estimation accuracy, and valid conditions for this technique were discussed.

A. GSD ESTIMATION AND PIXEL COUNT APPROXIMATION
The coordinates of the markers were extracted from the images of cracks of various sizes, and the side length and GSD of the cracks were calculated. The number of pixels according to the object length and the calculated and estimated crack widths are shown in Tables 2 to 4.
In each table, C r is the actual crack width and D o is the object distance, which is the distance between the image sensor and marker surface. L r is the side length of the marker calculated using Euclidean geometry, and GSD m is the GSD of the marker. N m , C m , and E m are the number of pixels of the  marker, calculated crack width, and error, respectively. N a , C a , and E a are the approximated number of pixels according to the object distance, calculated crack width, and error, respectively. E Dc is the error of the object distance calculated inversely using the GSD of the marker.
It was confirmed that as the target crack width decreased, the number of pixels decreased, and the error in the estimated crack width increased. In addition, when the crack size was very small, the object distance was located too far from the camera, and the pixel size was not detected. In particular, when the measured crack width was 0.127 mm and the object distance was 1,600 mm, the error reached 70%; therefore, it is desirable to assume a valid object range. It can be seen that the effective distance gradually decreased as the crack width decreased when the effective distance was assumed to be the effective distance at which an error occurred within 10%. In particular, when the crack width was 0.127 mm, the extracted pixel values were all less than five, and the effective distance was very limited. As the object distance increased, the variation in the error size did not show a clear trend, but the absolute value gradually increased, which can be confirmed by the error of each inversely calculated object distance. When the crack width was 0.308 mm, the error was approximately 4%, but when the crack width was 0.127 mm, it increased to a maximum of 10%. The pixel values extracted from each crack image appear close to integer values in the case of vertical or horizontal cracks, but they contain decimal units in the case of oblique cracks; therefore, the direction of the cracks can be estimated. The size of the crack can also be estimated from the number of pixels. However, when the number of pixels is very small (i.e., ten or fewer), the change in the pixel value according to the object distance can be a major factor influencing the estimation result along with the GSD calculated from the marker because a fine object is being considered. This is confirmed by the error variation according to the object distance of each crack, as shown in Tables 2 to 4. This occurred because cases were observed in which an error within 1% to 2% was observed even at a location outside of the effective distance, and the estimated results were very close to the measured crack width. Therefore, the adjusted pixel values were derived using the approximation curve of the pixel values according to the object distance of each crack, and the results are shown in Tables 2 to 4 and Figure 7.
The crack width was calculated using the GSD and pixel values. The GSD theoretically increases linearly with the object distance. Therefore, the curves approximated using the power law distribution of the pixel values according to the object distance were predicted, and the coefficient of determination of each curve approached 1 to ensure maximum reliability. The approximated pixel values can be seen in Tables 2 to 4, and it was confirmed that the effective distance was maintained or increased for cracks of all sizes.
The GSD of the marker tended to increase linearly with the object distance, as shown in Figure 8. The smaller the crack width, the smaller the GSD of the marker at each object distance, and the larger the distance from the measureddistance-based GSD. When the crack image was acquired, the focal length was 50 mm. When the crack width was 0.127 mm, the GSD was found to be up to 8% smaller than the GSD calculated based on the measured distance with the same focal length.
The increase or decrease in the estimated crack width according to the object distance was alleviated by approximation, and a tendency to increase or decrease over a certain  distance was observed. Figure 9 shows the change according to the object distance of each crack width calculated using the marker and the approximate number of pixels.
For a measured crack width of 0.308 mm, the estimated value showed an error within 9% for all object distances. The error increased slightly at distance of 1,000 mm or greater, and the largest error occurred at the maximum distance. When the measured crack width was 0.227 mm, the approximate crack width gradually decreased as the object distance increased, and when the object distance was 1,800 mm, the error was greater than 10%. However, the number of pixels extracted near this distance was approximately 2, which is difficult to use for estimation purposes. Even with approximation, the estimated crack width achieved when the measured crack width was 0.127 mm showed a relatively large error. The object distance required to estimate the crack width within an appropriate error range was within 450 to 600 mm, and a minimum of 3 pixels was required for the adjusted number of pixels.
As a result of the above analysis, it was confirmed that the number of pixels affected the effective distance and result VOLUME 9, 2021 accuracy more than the GSD according to the object distance when estimating the fine crack width using markers. It was also confirmed that the pixel value approximation method was able to partially extend the effective distance, but this technique was limited when the crack width was very small. The minimum pixel value required within a 10% error was not constant. Therefore, it was confirmed that alternative methods such as adjusting the marker GSD and ensuring an appropriate number of pixels by changing the focal length should be considered. It should be noted that if the focal length is increased even for an object of the same size, the pixel value increases even though the GSD decreases.

B. CRACK WIDTH ACCORDING TO THE FOCAL LENGTH
By increasing the focal length, the pixel value of the object at the same object distance can be increased. The measureddistance-based GSD and crack width estimation results according to the focal length were compared. The crack width was estimated based on the pixel values extracted from previous studies using the GSD at focal lengths of 50, 100, and 135 mm. The accuracy and error of the estimation results were examined. The change in the GSD for each focal length is shown in Figure 10.
All GSDs based on the measured distance increased linearly as the object distance increased, and the slope decreased as the focal length increased. At the same object distance, the GSD decreases as the focal length increases, but the pixel value recognized by the sensor increases. Therefore, in terms of ensuring an appropriate number of pixels, increasing the focal length is advantageous for estimating the fine crack width. However, when the focal length is increased, a restriction occurs considering on the proximity distance for focusing. Therefore, this is a major consideration along with ensuring appropriate image quality when obtaining fine crack images using drones or remote imaging equipment.
The pixel values, estimated crack widths, and errors according to the focal length of each crack are listed in Tables 5 to 7. F 50 , F 100 , and F 135 refer to the conditions in which the focal lengths are 50, 100, and 135 mm, respectively. The N, C, R, and subscripts represent the pixel value, estimated crack width, and error obtained at each focal length, respectively. Because the focal length is the same when extracting the crack pixel values from the marker image, N 50 is equivalent to N m .
As a result of estimating the crack width through the measurement-distance-based GSD and focal length change, the effective distance was found to be narrower than the estimated result based on the GSD of the marker at the same focal length. However, considering that the GSD based on the measured distance shows a linear variation, it was confirmed that the crack width can be sufficiently estimated even from the markers with a slight error in the GSD. As the focal length increased, the pixel values and effective distance gradually increased, but some limitations were encountered in terms of the proximity distance for imaging. As the crack width   decreased, the pixel value at the effective distance decreased to 2.
When the measured crack width was 0.308 mm, the effective distance of the crack width calculated based on the GSD of the marker was able to be determined up to 1,600 mm. Conversely, as shown in Table 5, when the focal length was 50 mm, the effective distance was significantly shortened to 500 mm. Therefore, it can be seen that the distance-based GSD does not guarantee an estimation error reduction. However, as the focal length was increased, the effective distance increased remarkably, and the minimum required pixel value was 6 when the maximum focal length was used.
Such a sudden change in the effective distance did not appear when the crack width was reduced to 0.227 mm, as shown in Table 6. As a result of estimating using the distance-based GSD under the same conditions with a focal length of 50 mm, the effective distance and maximum error decreased slightly, and the minimum pixel value at the effective distance decreased by 2. When the focal lengths were 100 and 135 mm, errors within 10% were obtained for all object distances.
It can be seen from Table 7 that both the upper and lower limits of the effective distance are limited when the measured crack width was 0.127 mm. However, the range of the effective distance increased compared to the estimation based on the markers. This occurred due to the increase in the pixel value according to the increase in the focal length, and the minimum pixel value at all focal lengths was 4.
The results of the estimated crack width for each focal length of the cracks according to the object distance are shown in Fig. 11. When the focal length was 50 mm, irregular increases and decreases were evident as the object distance increased, and the error size gradually increased.
This trend was especially noticeable when the measured crack width was small. However, it was confirmed that the error decreased noticeably as the focal length was increased to 100 mm or more. When the focal length was 135 mm, no significant error reduction was observed.
The result of comparing the root mean square (RMS) error to examine the accuracy according to the estimation method of each crack is shown in Figure 12. The comparison of RMS errors was conducted using results with at least three pixels that were extracted or approximated when estimating the crack width.
As a result of comparing the RMS errors of the calculated crack width according to each estimation method, when the focal length was 100 or 135 mm, the estimation accuracy tended to increase as the crack width decreased. However, as the crack width increased or decreased in all estimation methods, the estimation accuracy did not show a tendency to increase or decrease.
The RMS error obtained when approximating the pixel value according to the object distance was smaller than the estimation error obtained based on the marker. However, this error differed when the crack width was varied. For the smallest crack width of 0.127 mm, the decrease was the smallest. As confirmed in Figure 7, the relatively small coefficient of determination obtained when the measured crack width was 0.127 mm was reflected by the reduced width.
As a result of comparing the RMS errors of the markerbased estimation methods using the measurement-distancebased GSD at the same focal length, it was confirmed that the accuracy of these methods was increased for all crack widths. As was previously confirmed in Figure 8, the GSD of the marker was up to 8% smaller than the GSD of the distance at the same focal length. However, it was confirmed that applying the GSD of the distance does not guarantee adequate estimation accuracy.
By comparing the RMS errors obtained with varied focal lengths, it was confirmed that the estimation accuracy of the crack widths of all sizes increased as the focal length increased. However, a relative difference was observed in the extent of the RMS error reduction. When the crack width was 0.227 mm, the RMS error was the largest at a focal length of 50 mm, and the RMS error decrease that occurred as the focal length was increased was the smallest under these conditions.

IV. CONCLUSION
Fine crack widths were estimated using the GSD calculated from the fiducial marker coordinates of a crack image and the extracted pixel values. As a result, it was confirmed that the error increased as the crack width became smaller or the object distance increased, and the effective distance at which an error less than 10% decreased under these conditions.
The GSD of the marker estimated by the Harris corner detector increased linearly, similar to the GSD calculated based on the measured distance. As the crack width decreased, the GSD of the marker was up to 8% smaller than the GSD of the measured distance. However, the estimation obtained by applying the GSD of the marker at the same focal length was more accurate than the result of applying the GSD of the measured distance.
When estimating the fine crack width, it was confirmed that the variation of the pixel value according to the object distance was the dominant factor influencing the estimation results compared to the GSD. Therefore, to alleviate the variation of the pixel value and improve the estimation results, an approximation curve of the power expression was developed and applied. It was confirmed that the RMS error was reduced using this technique, and the estimation accuracy was improved via the approximation method.
In the estimations, the minimum pixel value decreased to 1 or was not extracted at all as the object distance increased, and the pixel value at the effective distance was not constant. To improve the estimation result achieved by increasing the pixel value, the RMS errors obtained with varied focal lengths were compared, and it was confirmed that the estimation accuracy increased as the focal length increased. However, because close-up photography is limited when the focal length is increased, this must be considered when setting the effective distance for remote photography.
In conclusion, the estimation results obtained in this study confirmed the feasibility of estimating crack widths of approximately 0.3 mm and less with a high accuracy by applying the GSD of a fiducial marker, the approximate pixel value, and ensuring an appropriate focal length.
The estimation accuracy and reliability of this technique may be further improved if crack images at a subdivided object distance for fine cracks of 0.1 mm or less are supported. Because close-up photography is limited when the focal length is increased, it is necessary to consider this restriction when setting the effective distance for remote photography applications.