Localization, Mapping, Navigation, and Inspection Methods in In-pipe Robots: A Review

In the oil, gas, and water industries, in-pipe robots have shown promising solutions to reach the inaccessible parts of pipeline networks for condition assessment, leak detection, and fluid quality monitoring. However, the pipeline networks have different sizes and configurations, and there are long lines of pipes that make the operation of in-pipe robots challenging. In this paper, we first categorize the mechanical systems of in-pipe robots and classify them. We then define four missions performed by the in-pipe robots: localization, mapping, navigation, and inspection. Next, we present the core methods used for each mission and list the studies that use the presented methods and the way they are implemented. Since image processing approach is a common way to accomplish task(s) for an in-pipe mission, the team decided to provide a comprehensive categorization of image processing techniques and the required tools. We also list the sensors used in in-pipe robots which would help the researchers select the appropriate sensor or sensor module for a specific technique.


I. INTRODUCTION
Pipelines are prone to damages. So, it is essential to perform regular condition assessments in pipe networks to prevent or cure those damages. However, pipelines are long and have complicated configurations buried underground, making most parts inaccessible. In-pipe robots have shown promising results in condition assessment when the sensing unit(s) are mounted on a body with controlled motion inside the pipe. Various missions can be performed by in-pipe robots, which help with saving time and energy for pipeline maintenance.
There are different mechanisms and configurations for in-pipe robots. Considering the mechanism, Shao et al. [1] classified the in-pipe robots with active locomotion into wheeled type, Caterpillar type, and non-wheeled type mechanisms. Robots with passive motion (moving by the flow in the pipe) use pressure inspection gauge (PIG) mechanisms [2], [3]. Table I shows different classifications for in-pipe robot mechanisms.
In wheeled type robots, the simple structures refer to the robots that have wheels connected under their body and move inside the pipe [4], [5]. In the wall-press types, the wheels press the pipe while they are connected to the endpoint of some adaptable arms anchored on a central body making a 120° angle with each other [6]. In the screw drive types, the wheels are mounted on a rotational and a fixed units that transmit the spiral motion of the wheels to linear motion along pipe axis [7].
In the Caterpillar type robots, belts and wheels move the robot inside the pipe with high friction between the robot and the pipe wall that ensures a reliable motion compared to wheeled robots. The caterpillar type robots are divided into the simple structure [8] and wall-press type [9].
In the non-wheeled types, snake types have some connected modular units that enable the motion for the robot inspired by the motion type of a snake [10]. Inchworm type robots move inside the pipes by contraction and extension of a flexible body along the pipe axis [11]. Another type of nonwheeled robots that are less common are legged robots that provide better maneuverability in complicated configurations by the expense of sophisticated motion control algorithms [12], [13]. Finally, free-swimming robots suggested in the literature are designed for small diameter pipes and move in the pipe with propellers that provide propulsion force for the robot during motion [14] (see Fig.  1).
As for the actuation of the in-pipe robots, different kinds of actuators have been used in the literature. Pneumatic actuators [11], [15], [16] and electrical actuators that have DC motors [17]- [28] are the most common in in-pipe robots. Some new actuators like shape memory alloy (SMA) actuators have also been used to generate traction and press forces in in-pipe robots [29]. The power supply for in-pipe robots can be either provided by cable [18], [19], [26], [28], [30]- [34] or battery [9], [17]. Fig. 2 shows the classification of the robots in terms of locomotion, actuator, and power supply. Also, Table II compares different mechanism types of in-pipe robots in different performance indicators. Due to many uncertainties, disturbances, and complicated configurations in the pipelines, the in-pipe robots need to have smart motion in pipelines during operation.
In some cases, among tethered robots, communication is facilitated by wire, and among the battery-powered robots, communication is facilitated by wireless communication. However, the short length of wires limits the inspection range of the robots. Moreover, due to the high signal attenuation of radio signals, wireless communication is a challenging task [35]. In addition, pipeline networks include long pipes with complicated configurations [36].
Due to advancements in numerical methods, it is possible to facilitate smart motion for in-pipe robots and enable long pipe inspections with smart navigation. In this paper, we define four specific tasks that are desirable in inpipe missions: Mapping, Localization, Navigation, and Inspection. In this paper, we describe these concepts and the different associated methods used for each concept.
The remainder of the paper is as follows: In section II, the localization is explained, and the methods for localization are described. In Section III, mapping methods are summarized; in Section IV, the navigation methods are presented; and in section V, the inspection methods are explained. We explain and categorize the image processing techniques and sensors used in in-pipe robots in sections VI and VII, respectively. Finally, the paper is concluded in Section VIII.

II. LOCALIZATION
The term localization for in-pipe robots refers to determining the location of the robot in the pipeline network. Localization is desired once the robot moves in the long and complicated configurations of pipelines. There are different methods that the researchers use to localize the robot in the pipe. Most localization techniques are a combination of different methods. In the following subsections, we describe the core methods used for localization and present the use of these core methods with other methods to facilitate localization for a specific application.  [8]; (e) Wall-press Caterpillar Type [9]; (f) Snake Type [10]; (g) Inchworm Type [11]; (h) Legged Robot [12]; (i) Free swimming Type [14].

(h) (g)
Kalman filter is an optimal estimation algorithm used in two applications; first in the application where it is not possible to measure the desired parameter, and second to estimate a parameter from various measurements subject to noise. Kalman filter is a recursive approach and works based on the state-space representation of a system. A system can be represented as: where is the current time-step, is the state, is the state transition model of the system, is the input model of the system, is the system input, is the system measurements, and is system's output model. Also, ~(0, ) and ~(0, ) are process and measurement noises,  Simple Structure respectively, that have zero mean and normal probability distribution (Gaussian distribution). Our goal is to estimate , that we call it ̂, using system measurements . The estimation in the Kalman filter comprises two steps: 1) prediction and 2) update. The prediction step can be represented as follows: (4) In Eq. (3), ̂− is a prior state estimate before the current measurement is done and in Eq. (4), − is the prior estimate error which represents a measure of uncertainty in the state estimation. − comes from process's noise and the propagation of the uncertain ̂− 1 . The initial values for and ̂ are their initial estimates. The second part of the estimation (the update step) uses the prior estimate, ̂−, that is calculated in the prediction step. The update step is represented as follows: In Eq. (6), ̂ is the posterior estimate of the state at step . The process of prediction and update is repeated until the results converge (see Fig. 3) [37]- [39].
Wu et al. [40] proposed a method to use onboard sensors, including an Inertial Measurement Unit (IMU), gyro, and leak sensors to localize an in-pipe leak detection robot. To do so, two methods are presented which utilize IMU data, average and instantaneous speed, and relative distance considering candidate in-pipe joints. The Rauch-Tung-Striebel (RTS) smoother algorithm and Extended Kalman filter are applied to extrapolate and correct the robot's trajectory. Siqueira et al. [41] suggested a strategy to localize an in-pipe robot to determine the defect position. They proposed using the steering encoder data to pose the robot and to use the IMU to acquire reliable robot orientations. Also, the extended Kalman filter technique is implemented to fuse the encoder and IMU data. Liu and Krys [42] proposed a method to detect the pipe surface anomaly using a laser range finder. A Kalman filter is implemented to the laser range finder readings to reduce the noises and errors. Anjum et al. [43] presented a method that utilized a low-cost accelerometer, gyroscope, and encoders for robot localization and slips error compensation. An unscented Kalman Filter is applied as an efficient sensor fusion technique. Maneewarn and Thung-od [44] have suggested a method with the combination of the Extended Kalman filter (EKE) and Iterative Closest Point (ICP) matching algorithm to localize, inspect, and map the pipe. To help increase the localization process's accuracy, artificial landmarks have been introduced in the environment. They used Reduced Inertial Sensor System (RISS) and data fusion by extended Kalman filter to improve PIG localization. Al-Masri et al. [45] have proposed two mechanization approaches, Inertial Navigation System (INS) and 3-D Reduced Inertial Sensor System (RISS), to improve the Pipeline Inspection Gauge (PIG) to navigate and localize the robot and map the pipe profile. An Extended Kalman filter is used to fuse the IMU data (used in the navigation problem) with PIG measurements and to reduce the accumulated errors.

B. PARTICLE FILTER
The particle filtering method uses the parameters in the system that can be measured directly from sensors to estimate other parameters in the system that cannot be measured directly (e.g., the location of an in-pipe robot) using the mathematical relation between those parameters (similar to Kalman filter). For that purpose, a distribution of points named particles with random weights (usually uniform) is assigned to the unknown parameters updated iteratively by their mathematical relationship with the measured parameters. In these iterations, the updated values for the particles will converge to the true estimate of the unknown parameters. The advantage of particle filtering over the Kalman filter is that neither the input nor the output distributions need to be Gaussian [46]. Algorithm I shows the particle filtering algorithm. In this algorithm, is the set of particles that is empty at the beginning and updated in each iteration. is the unknown parameter (e.g., location of the robot in the pipe) at step . is the measured parameter    at time . is associated with the weight of particles that would be updated based on the likelihood of the measured parameter given the unknown parameter.
is a random noisy movement that would be added to the particles at step . Also, η is the normalization factor updated by in each time step.
In [47] and [48], the researchers used pipe vibration amplitude as the input to the Kalman filter to estimate the parameters of the map and the particle filter to estimate the robot's location in the pipe. The wireless radio frequency (RF) signals have periodic fading in metal pipes that can be used as a sense of the map of the pipe. The authors in [49] presented the particle filter method using the periodic nature of RF signal fading for the initialization of the particles. Also, it is possible to locate wireless transceivers at different locations of the pipelines as "beacons." In [50], the authors deployed several wireless transceivers at different pipeline locations, which give the pipe map to the particle filter algorithm for localization (see Fig. 4). Kazeminasab et al. proposed a localization method based on particle filtering which uses the non-straight configurations of the pipeline that the robot needs to pass through during operation as the map of the pipeline [51].

C. SIMULTANEOUS LOCALIZATION AND MAPPING (SLAM)
Simultaneous localization and mapping (SLAM) is a cyclic nature problem for in-pipe robots in the pipeline environment. In this problem, the autonomous robot needs the information about its location in the network to build a map of the environment while it needs the map of the pipeline to localize itself simultaneously. As shown in Fig.  5, in the SLAM problem, the output of the robot's localization is the input of its mapping and vice versa [52]. Several studies have been done to solve this problem in the pipeline environment, and some algorithms are proposed to address the issue. The SLAM algorithms used in robot navigation, robotic mapping, and odometry are based on concepts in computational geometry and computer vision. In [47] the authors proposed a method so-called PipeSLAM based on the RaoBlackwellised particle filter (RBPF). Their technique is based on exciting and measuring pipe vibration using a pair of hydrophones to create a map of mean vibration amplitude along the length of the pipe. Their algorithm utilizes a radial basis function network which is linear in the parameters to decompose the map. Then, they used Kalman filter to estimate the map parameters. Also, the particles have been used for estimating the location of the robot in the pipeline. In [53], the authors utilized a multisensor fusion system that can construct a global image of the internal surface of the pipeline and also localize nondestructive testing (NDT) sensors. For localization purpose, they utilized rangefinders to provide the distance between the cameras and the front objects. The sequential images of a digital camera are stitched together using an inertial navigation system (INS). Sequential images acquired from the camera are used to measure travel distance and to construct the global image of the internal surface. In [54], the researchers proposed a method for fault detection in a sewer pipe with data fusion of depth camera, infrared, and g-sensor and built the 3D map of the pipeline with SLAM. The authors in [48] considered the SLAM problem for metal water pipes. In this paper, mapping is based on hydrophone-induced vibration signals of metal pipes, and signals from hydrophone excitation of the metal pipe lead to a map of pipe vibration amplitude over space. The mapping algorithm uses spatial warping and averaging of dead reckoning signals to calibrate the map based on dynamic time warping. The localization is based on nonlinear state estimation, and they performed both terrain-based extended Kalman filtering and particle filtering. In the proposed method in [55], the orientation sensor and position sensor provide the information about the direction change and the moving distance, respectively. Using these sensors' information and images taken by the CCD camera, they can construct the pipeline map. Also, they utilized a gyro sensor and encoder for both the localization of the robot and the mapping of the pipeline. In [56], for mapping, the authors utilized: 1) The encoder information to locate the robot inside the pipeline, 2) the gyro sensor and accelerometer to measure the roll motion, and 3) the information of the camera image to detect the pitch and yaw angles. Finally, they used the information given from the camera image and sensor data to construct a complete map of pipelines. Also, using the map, they can localize the robot by measuring the moving distance of the robot. In SLAM, landmark detection is a challenge, and designing a temporal visual landmark on the

Mobile Object
Wireless Base Station inner surface of the water pipe is a promising solution to this aim. The authors in [57] designed an artificial landmark generation module using paintball guns placed on the robot to make artificial landmarks and used a mono camera located at the robot's tail for SLAM purposes in the future. In [58], the authors proposed a pipe inspection robot called the boiler header inspection robot (BHIR). For the localization of the robot, they utilize a moving average filter method to ensure the accuracy of the localization system. Also, they used an accelerometer sensor and an embedded encoder sensor to measure the robot's position.

D. LANDMARK DETECTION
A pipeline consists of two general configurations: 1) straight configuration, and 2) non-straight configurations such as elbows and branches. The changes in robot direction are fixed to a certain value when it passes non-straight configurations (see Fig. 6). Therefore, we can use the nonstraight configurations for localization purposes in which these standardized pipe elements of certain geometries will be referred to as landmarks. With the provided explanation, landmarks can be used for robot tracking if the mobile robot is constrained to move on a horizontal pipeline. In addition, since the non-straight configurations have certain angles, only the number of changes in the direction of the robot is sufficient to find the its position in the network. Several studies have been done to recognize the landmarks in the pipeline environment such as vision systems [59], sonar sensors [60], and IR sensors [61]. We can categorize these methods into two main categories: 1) vision-based systems 2) laser-beam projection based. In vision-based systems, a special illuminator has been used in an in-pipe robot for obtaining shadow images. The robot can use these shadow images to identify landmarks, traverse through the pipeline, and simultaneously create a map of the environment. In methods based on laser beam projection, the robots use the fact that the projected beam on landmarks has different line patterns that can be useful for localization and navigation inside the pipelines [62].
In the method provided in [62], the robot uses three features of pipelines by laser scanners for landmark detection: pipe inlets, manholes, and pipe joints in sewer pipes. In [61], the robot uses an Infrared (IR) sensor to get the landmarks of the pipe and localize itself in the pipe using the pipe inlets, pipe joints, and T-junctions in pipes. In [63], the robot uses a 2D laser scanner and fuse data with a fisheye camera for landmark detection. In [64], the authors calculated the position of the landmarks such as manhole and pipe joint using the image of the KANTARO camera system. The study converted the taken images into a map which can be a descriptor for landmarks and distances between them. The authors in [65] developed a competitive sensor called a bespoke sensor that can detect the corner direction and radius. Also, they have a custom-designed test rig for reliable testing of the developed sensor package (e.g., the testing of the radius prediction on different radius bends). In [66], the authors used an illuminator that has been designed on the robot for detecting landmarks. This shadow will be generated by the shapes of the elbow or branch. When the robot moves inside the pipe, they captured the pipe images and check them for the existence of a shadow region. For this purpose, the authors implemented a pattern matching image process to recognize the shadows. Then, they use the result of this process to extracts the landmark type and pipeline directional change.
The difference in viewing direction between the camera and line laser projector will generate shadow regions. In landmarks and straight pipelines, the shadow region's distribution is different. In [67], the authors project a line laser beam on the internal surface of the pipeline. This generates some unique line patterns on the surface of the landmarks. The robot can utilize these line patterns to detect and distinguish landmarks and follow the direction of the pipeline path. In [68], the authors used a line laser beam projected on the internal surface of the pipeline to deploy the generated unique line pattern. Then, they applied an image processing procedure to detect generated line patterns. For making this procedure easier, they utilized a line laser beam with a distinct color. They rotated the laser beam around the camera to obtain a set of line segments. In [69], the authors proposed an algorithm for navigation and self-localization to reduce the unreliability of landmark classification for navigation using the Markov decision, KURT platform. In [70], the authors used the line-laser beam landmark detection method. They used a red laser that rotates around the center of the image. Because of the special shape of landmarks, the laser beam can't reach certain regions. They detected these regions and extract geometrical information of the landmark. The method will return three outputs when it encounters a landmark, including: 1) the type of landmark, 2) distance, and 3) the relative rolling angle which is useful for the yawing estimation. They utilized three kinds of sensors to build the map: 1) posture sensor: to recognize its posture, 2) line laser beam to detect landmark, and 3) distance sensor to measure the distance of robot movement.
In [71], the authors studied the application of time-offlight (TOF) cameras in pipelines. They used this camera to detect and track the features or landmarks in the pipeline. In the feature extraction process, they fitted a cylinder to the images taken by the TOF camera. In [72], the authors used the gaps between the pipe wall and soil-voids as a new type of landmark feature which can be sensed by ultrasonic  scanning. The landmark detection method in this paper is based on the outside of the pipe and voids that existed on the soil. The authors in [73] took advantage of the 3D data provided by the TOF camera to find significant deviations from the cylindrical shape of the pipeline. To facilitate the obstacle traversal and find the obstacle position relative to the pipeline, a simplified model is used to map the robot's environment into an along-axis view. In [62], the authors proposed two methods for landmark detection. 1) Landmark Detection Using Shadow using the landmark's unique patterns of shadow produced by the robot's illuminator. 2) Landmark detection using the laser projection method which uses a line-laser beam projector to extract the geometrical feature of pipeline elements. They utilized two imageprocessing methods to extract landmark information. In [74], the authors used conventional gradient measures for stereo matching [75] to calculate the distance data of each feature pixel at manholes or joints [76]. Then, using these results, they measured the distance between the robot and landmarks. When this distance becomes zero, the technician can detect it. Also, to reduce the error of the stereo image, the authors applied the laser scanner data to detect the exact location of the landmark. They only checked the laser scanner data if the distance to a landmark is less than a threshold. In [77], they proposed a landmarks detection method to navigate an autonomous sewer inspection robot. They used stereo camera images to estimate the location of the robot in the sewer pipe. But since this method lonely is not enough for a good estimation, they used the laser scanner data to ensure the accuracy of the landmark localization and reduce the error in the estimation of distance. In [78], the authors presented two methods to provide position and orientation information for mobile robot navigation with a panoramic camera and the results of a synthetic and real experiment are reported. The location detection methods track the projections of environment landmarks predicted according to the robot camera's position. The first used a modified corner tracking procedure in a quasi-cylindrical view, and the second tracked line segments and determined corners from the junction of two-line segments. In [79], they proposed a method for localization. They used a pre-generated map that aid the robot. They used three types of sensors for localization IMU sensor, an IR sensor, and an ultrasonic sensor. They utilized the IMU sensor to determine the robot's orientation, the IR sensor for the motor encoder, and the ultrasonic sensor to determine the type of landmark in the way.

E. TOPOLOGICAL ROBOT LOCALIZATION
Topological maps include a network of distinctive places which are connected by distinctive travel edges [80]. Fig. 7 shows a schematic of a typical pipeline network that consists of several distinctive places and the path between them. In a pipeline environment, landmarks such as elbows and branches are distinctive places [81]. The in-pipe robot can utilize sensors and cameras to detect landmarks from nearby regions called "neighborhoods." It can locate itself in a pipeline environment related to the position of landmarks. In the motion control level, a distinctive place is a local maximum of a distinctiveness measure related to its next neighborhood which can be found by hill-climbing control strategy. Hill-climbing is a mathematical optimization technique that can be used for local search. This algorithm starts with an arbitrary solution and tries to iteratively find a better solution by making an incremental change to the current solution [80], [82]. Also, a distinctive travel edge is measured by a path-following control strategy that the robot can follow. Using the path following approach results in many advantages on the control performance and design of the robot since it removes time dependence of the problem [80]. The identification of distinctive places and distinctive travel edges can be done using the interaction between the robot's control strategies, its sensorimotor system, and the surrounding environment. The robot updates its states at landmarks, and it does not represent the orientation and position between states. In this way, the robot only navigates between local distinctive places, and hence, motion errors do not accumulate globally in topological maps. Also, since topological maps only represent landmarks, they are more compact in representing the pipeline environment [81].

Distinctive Travel Edge
Distinctive Place In probability theory, a Markov model is a stochastic model used to model changing systems [83]. It is assumed that current states ( 0 ) depend only on the immediate previous state ( ( 0 −1) ), not on the events that occurred before the previous state ( <( 0 −1) ) (Markov property). Hidden Markov Model (HMM) is a statistical Markov model in which the modeled system is a Markov process with hidden states (X). The goal of HMM is to learn about X by some observations, Y. It is assumed that current observations ( 0 ) depend only on current (i.e., hidden) states ( 0 , e.g., current location) and does not depend on the previous states ( < 0 ); in other words, the transition between the hidden states does not affect the current observation [81]. Fig. 8 shows an example of an HMM [84].
In [69] and [81], the authors proposed an HMM to localize a robot in the pipeline. In [73], the hidden state has three components, including the robot's current position ( ), previous position ( −1 ) and current orientation ( ). The robot state is only updated at junctions or the ends of pipes. But the robot's position and orientation are not considered in transitions between these states. In this method, the distance traveled since its last state is measured by odometry (or dead reckoning).

F. Radio Frequency (RF) Signals Method
A Radiofrequency odometry method is used to localize an in-pipe robot. Most localization methods are failed to determine the robot's position due to the limitation of the pipe size. As a solution, it is possible to use radio frequency methods to estimate the robot's location as wireless communication. Generally, three main approaches using wireless communication can be implemented to localize the robot which are described as follows:

1) OUTDOOR LOCALIZATION BASED ON THE RELAY NODE PLACEMENT IN A STRAIGHT PATH
Relay node (RN) placement is a method in which some radio transceivers are placed aboveground along with the water pipe. RNs pass the information from the sensor network (SN) to the base station (BS) (see Fig. 9). The proposed system is a so-called wireless sensor network (WSN). In [85], the researchers investigated the wireless communication channel and developed a path loss model for

Relay Node (RN)
the communication channel considering signal attenuation (due to absorption of water, plastic pipe, and soil), signal reflection, and refraction at the interface of each medium, and multi-path characterization. In [86], this team investigated the optimum number of RNs required to have a reliable WSN. In [87], the authors proposed a localization method to develop WSN based on the RSS from the SN and the Kalman filter.

2) RELY NODE LOCALIZATION ON NON-STRAIGHT CONFIGURATIONS OF PIPELINES
Kazeminasab et al. [88] proposed a method using RNs that extends the RN-based method's application to nonstraight paths. In this method, the RNs are located at nonstraight configurations of the network that is the operation path of the robot (see Fig. 10). The underground robot switches its communication with each RN based on its location in the network. A wireless sensor module is also developed for the robot that facilitates a bi-directional communication link between the robot and RN. Also, a multi-phase motion control algorithm is developed for the robot in which:  Stabilizes the robot with zero velocity at the junctions.  Steers the robot to the desired path at junctions like bends, Y-shapes, and Tees.  Stabilizes the robot and enables it to track the desired velocity at straight paths. The wireless sensor network is synchronized with the multiphase motion control algorithm. The robot switches between different phases of the motion control algorithm based on the motion commands from the RN. There may be an overlap between the commands from the RNs that are delivered to the robot. To address this effect, the map of the operation which is an array and includes the non-straight configurations (in which the robot needs to path through) is given to the robot. Hence, the robot can distinguish the right command from the associated RN.

3) ABOVEGROUND PARALLEL MOVING VEHICLE
Qi et al. proposed a localization method in metal pipes using extreme-low frequency electromagnetic pulse to overcome the shielding feature of metal pipes. In this method, the robot moves inside a pipe underground. A vehicle moves on the ground approximately in parallel with the robot which contains a sensor array. The robot sends pulses, and the sensor array on the vehicle analyzes the strength of the received signal. The location of the sensor with stronger RSS is the location of the robot underground [89], [90] (see Fig. 11). Magnetic induction is a promising solution for underground wireless communication which mitigates the effect of multi-media propagation in the underground environment. The authors in [91] proposed a localization method based on magnetic induction in which two coils with 10-cm diameter are located on two robots: one robot moves inside the underground pipe (A-robot), and another one moves above the ground in parallel with the underground robot (U-robot). A-robot and U-robot have constant wireless communication during operation (see Fig.  12). However, in this method, the coils need to be in exact parallel orientation with respect to each other. Otherwise, communication link efficiency drops dramatically.

4) RADIOFREQUENCY SIGNALS INSIDE THE METAL PIPES
The metal pipes behave as waveguides for RF signals if the frequency of the carrier signal is beyond a value called the cutoff frequency. To localize the robot inside the pipeline, each in-pipe robot is equipped with a microwave module for communication. The system consists of a signal transmitter, including the continuous wave signals to create the radio signals and a mounted spectrum analyzer as an RF receiver. The transmitter is placed at the beginning of the pipe, and the RSS recording is done when the receiver is moving along the pipe with constant speed. In comparison to other localization methods, this method requires no previous known map to localize the robot [92]. It's possible to use the periodic fading feature of RF signal as waveguide inside pipes using multiple transmitter-receiver setups and internal configurations. In other words, using the periodicity nature of radio signals as odometry measurement, one can localize the in-pipe robot by RSSI module inside the pipe. The effect of the antenna position and orientation in the robot over the periodic fading is determined in [93].

G. MISCELLANEOUS METHODS
The authors in [94] proposed a localization method in sewer pipes based on sensor fusion of the IMU and the cable encoder based on optimizing a set of windowed stages. Echo-localization is another method for localization. The  researchers in [95] proposed a mechanism by placing a loudspeaker and microphone on the in-pipe robot to do acoustic echo-based localization. In this method, they measured the position of the robot and detected distant features in the pipe using echoes of the acoustic mechanism. Also, for sensitivity analysis and removing undesired measurements, they classified the acoustic echoes by estimating the length of the pipe. In addition, they used Kalman filter to incorporate the robot's motion into the localization to remove uncertain and limited measurements. In [96], the authors used pipe junction detection to correct continuous azimuth and pitch errors to enhance position estimation done by MEMS-based Inertial navigation systems.

III. MAPPING
As its name suggests, mapping is the process of constructing the map of the pipeline using an in-pipe robot. The pipelines were usually implemented a long time ago. Hence, their map is not accurate. The in-pipe robots are promising solutions that can be used to this aim. There are several methods introduced so far to extract the pipeline map. In the SLAM-based methods, the operation map is constructed at the time localization is performed and we covered it in section II. In this section, we introduce other main methods that are popular in mapping as follow.

A. 3D POINT CLOUD
3D point cloud contains coordinates of a set of points in space that form the shape of an object in 3D space. 3D point clouds can be used to achieve the dynamic 2D global map of the pipes that show the real-time position of different objects in the map. Using a 3D LiDAR or stereo camera, a real-time 3D point cloud can be generated (see Fig. 13a). However, since it would take a long time to process the 3D map for robot navigation, the real-time 3D point cloud should be converted into a dynamic 2D point cloud. For that purpose, the ground, slopes, and planes are removed from the 3D map using image processing techniques (see Fig. 13b); a 2D static global map would be obtained by projection of the remained 3D map. Then 2D dynamic global maps can be achieved by using the projected 2D static global map (see Fig. 13c) [97]. In [98], they used an omnidirectional camera and laser to acquire an image sequence of the robot during motion and calculated the 3D coordinates of the measurement points, and hence proposed a reconstruction method for the pipe. Next, they estimated the camera motion and produced a triangular mesh from the measurement results. Finally, they generated a 3D model of the pipe by mapping the texture to the triangular mesh. In [99], a method called Structure-from-Motion (SfM) system is used for mapping. SfM is a visionbased 3D reconstruction that reconstructs the 3D structure of pipe from sequential images for a pipe trajectory with an endoscope camera. SfM extracts the local features of the image to get 2D correspondence between images for ordering a set of images. It uses the 2D series of images to construct the 3D structure of the object. Also, in this method, bundle adjustment is applied for minimizing the 3D points from corresponding 2D observations.

B. MISCELLANEOUS METHODS
The authors in [100] proposed a method to reconstruct the trajectories of a pipeline inspection gauge. In this method, the data from an IMU and odometers are fused by an extended Kalman filter. In [101], the authors used a rangefinder constructed with an omnidirectional camera and an omnidirectional laser to construct the 3-D shape of the pipe. They calculated 3-D coordinates using the light section method and integrated it with the camera motion information which is estimated by the Structure from Motion (SFM) technique to reconstruct the shape of the pipe. In [102], the robot location and pipeline map have been estimated using a sound-based localization method. The authors put a microphone on the robot and a loudspeaker at the entrance of the pipeline and measured the time of flight (ToF) of a reference sound emitted from the loudspeaker to estimate the distance of the robot. They combined the calculated distance with the robot orientation estimated from IMU observations to simultaneously estimate the robot location and pipeline map. For this purpose, they present a nonlinear state-space model to show the relationships among the observations, robot location, and pipeline map. They estimate the robot location and the pipeline map using an extended Kalman filter and the past locus of the robot location, respectively. The researchers in [103] presented a visual odometry and mapping system using calibrated fisheye imagery and a sparse structured lighting system. They also proposed an image processing method implemented offline after dataset collection and provided the necessary data for the pose estimation and mapping. The authors in [104] used the Kalman filter and reference stations (Landmarks) for localization and mapping purposes. The robot uses a laser rangefinder to scan the inner surface of the pipeline and utilizes decentralized exploration and centralized mapping methods to construct the pipeline environment map [105].

IV. NAVIGATION
Navigation is the process of movement of the robot in the pipe and steering to a desired direction in non-straight configurations. Navigation is the nature of an in-pipe robot and once we talk about a mobile in-pipe robot, it means it moves in the pipe. Motion controller algorithms are the heart of navigation that control the motion of the in-pipe robot in different configurations of pipelines. In the following, we explain the motion control algorithms used in in-pipe robots.

A. LINEAR QUADRATIC REGULATOR (LQR) CONTROLLER
LQR controller is an optimal state feedback controller used for dynamical systems and minimizes the cost function [106]. To design an LQR controller for a system, the statespace representation of the system is required in the following form: In Eq. (8), , , and are the system states, input, and output, respectively. Also, , , , and are system matrices, input matrices, output matrices, and feedforward matrices, respectively. Based on the system matrices and the desired functionality for the states, a cost function in the quadratic form is defined as: In (9), weights the stabilizing states, and weights the input vector. The cost function, ( ) is minimized with the value of as: = −1 (10) is computed with the algebraic Riccati equation: − − − + B −1 = 0 (11) The control input, , is computed in (12) as: = − (12) In [107], an LQR-assisted controller is designed for an under-actuated in-pipe robot that stabilizes the robot inside the pipeline against uncertainties and disturbances. The LQR controller in this robot is combined with a velocity-tracker controller. The authors in [108] proposed a control system based on the LQR controller for a screw-type in-pipe robot.

B. PID CONTROLLER
Most of the in-pipe robots use direct current DC motors as actuators and proportional-integral-derivative (PID) controllers are the best choice for controlling the position velocity and torque of DC motors. For each parameter in the system that needs to be controlled, an error of ( ) is defined as the difference between the reference value of the parameter ( ) and the measured parameter ( ). The PID controller assigns the control input ( ) to the system based on the following equation: , , and are proportional constants, integral constants, and derivative constant, respectively [109].
Kakogawa et al. developed an omnidirectional bent-pipe self-adaptation mechanism to pass through bends by proposing a PID-based controller that controls the joint torque in which a motor is attached to that joint [110]. The authors in [111] proposed a multi-phase motion controller for their developed under-actuated in-pipe robotic system that enables their robot to have a reliable motion against disturbances and uncertainties in in-service distribution systems for water quality monitoring. The multi-phase motion controller comprises three phases; in phase 1, the robot stabilizes motion with zero velocity, in phase two, the robot moves in a straight path with the desired velocity, and in phase 3, the robot steers to the desired direction in nonstraight configurations based on the PID controller.

C. FUZZY LOGIC CONTROLLER
Fuzzy logic control (FLC) works based on fuzzy values that can take continuous variables between 0 and 1 rather than crisp values of 0 or 1. The input to the fuzzy controller is usually two values 1) error and 2) the rate of error [112]. functions is shown in Fig. 15. The second stage is the processing stage in which the membership labels with their degree are fed into several logic rules (IF-THEN statements). The common general form of logic rules is: Where stands for membership function. The final stage is the defuzzification stage in which the output values of all the rules are solved together to generate a crisp composite output [112].
Type-1 FLC, triangular and trapezoidal membership functions, is used to avoid the robot from turning over [113]. Also, [114] used a charge-coupled device (CCD) camera and image processing for locating elbow or junction and navigation using FLC. In [115], a two-mode controller consisting of a proportional-integral-derivative (PID) controller for controlling the actuators and an FLC for setting the output of sensors such as speed, angle of climbing, and the rate of this angle was used.
Other navigation methods for in-pipe robots are also suggested for in-pipe robots. In [116], the authors proposed a sliding mode controller [117] for a screw-type robot for navigation in the presence of flow in pipelines.

D. MISCELLANEOUS METHODS
In [118], the authors presented a new laser vision system to navigate mobile robots. To obtain the selective 3-D frontpipe maps, which are needed to recognize T-junctions and elbows, they mount weaving 2-D laser scanners on robots. Based on the pipe scenarios for straight pipes and T-junction, a partial weaving plan and full weaving plan are needed, respectively. Also, for the recognition purpose, they developed algorithms using the mathematical approach to describe the pipeline structures. In another research, the authors used multiple robots that each of them acts as a relay communication node and used wireless relay communication to assist the robot to extend the inspection range [119]. The authors in [120] developed a crawler robot that can move in pipes with vertical directions using a magnetic adsorption mechanism with pipe walls. In [121], an in-pipe robot used position-sensitive device (PSD) sensors for navigation and distinguishing between t-branch and miter. In [122], a controller for negotiating elbows in pipe based on differential motion is proposed. This paper [123] used a laser spot array (LSA) and a camera to find the pathway direction. To do so, the Levenberg-Marquardt method (LM method [124]- [126]) is applied as a solution to the nonlinear least square problem used to estimate the pipe central vector.

V. INSPECTION
The inspection of the pipelines is categorized as assessing the conditions of the pipes, leak detection, and monitoring the quality of the fluid inside the pipe. In the following, we describe these methods.

A. LEAK DETECTION BY PRESSURE GRADIENT
To avoid using several pressure sensors on the pipe wall, a leak detection method via pressure gradient technique is proposed [129]. This method is based on finding a localized pressure gradient ( ) where denotes pressure and denotes the radial coordinate of the pipe. In the vicinity of the leaks, there is always a pressure gradient between the inside ( High ) and outside ( Low ) of the pipe. This pressure gradient causes the fluid to come out of the pipe. Referring to Fig. 16, we have the following formulas: is the friction force applied on the membrane, is the cross-sectional area of the leak, is the radial distance from the point of exertion of on the membrane to the center of rotation of the drum, point . In fact, in the pressure gradient method, instead of identifying a radial pressure gradient directly, we can measure the torque on the detector (i.e., ) by force sensors installed on it and calculate the pressure gradient indirectly. This method is independent of the pipe material and radius [127].

B. PIPE PENETRATION RADAR
The pipe penetration radar (PPR) survey system )as a non-destructive method( is applied for in-pipe condition assessment and inspection, including accurate pipe wall thickness estimation and pipe corrosion and void detection in non-ferrous (such as concrete, PVC, HOPE, vitrified clay, etc.) underground pipes. The PPR system includes two highfrequency ground-penetrating radar (GPR) antennas which consist of a high-resolution electromagnetic technique mounted on a tracked robot to move along a survey line into an underground pipe to build the pipe profile. The GPR data is analyzed on the pipe penetrating radar data interpretation application (PP-RADIAN). For data correlation and having more accurate results, CCTV is applied to record the pipe information simultaneously [128].

C. ARTIFICIAL NEURAL NETWORKS
The fundamental concept of the artificial neural network was given from the function of the biological nervous systems. Like the natural brain, a neural network has the ability to learn based on the available data to perform tasks. An artificial neural network includes an input layer of neurons that receives the input values, an output layer of neurons, and hidden layer(s) between input and output layers. The neurons are connected by links (Fig. 17) such that each link is labeled by a numeric number named connection strength or weight which conveys a signal from one neuron to another one. This value determines whether the signal can be amplified (positive weight) or diminished (negative weight) using the corresponding link; when there is no connection, the weight would be zero [129]- [131].
The output can be calculated using the activation function using the following formula [131]: Where is the activation function, are the weights, are the inputs, ℎ are the threshold of hidden layers, and is the number of inputs.
Alejo et al. [132] have designed a mobile platform (SIAR) to measure the gas concentration and detect detection faults such as cracks and holes in gas pipelines. As for localization, Monte Carlo Algorithm fuses the information from the wheels and the RGB camera in the odometry prediction stage. This platform uses prior information such as topology, elements' positions like manholes, and crosssections traveled to update the localization. As for manhole detection, a 3D image and also an artificial neural network are used. The features of the 3D images are extracted with features from the Accelerated Segment Test (FAST) algorithm. Waleed et al. [5] have designed an in-pipe robot and equipped it with several pressure sensors to detect the leak using pressure-distance diagrams. To do so, an operator detects a leak by analyzing the pressure changes across the total length of the pipe. To verify the accuracy and robustness of the detection approach, an artificial neural network is proposed to process the pressure sensor data with 501 hidden layers and two output layers (no leak vs. leak). Results showed that the in-pipe inspection robot could detect the leak with an 83.3% success rate. Duran et al. [133], [134] have suggested a method to use laser profilers to improve CCTV illumination used in inspection in-pipe robots. Their proposed method has two steps of defect detection and classification using an artificial neural network and fusing positional and intensity data. Duran et al. [135] have proposed a method that used image processing (based on the light intensity variations) and neural networks to identify the defective and non-defective pipe sections. They employed MLP, which is a commonly used neural network architecture. Also, they apply a preprocessing step on the input of the neural network to highlight the characteristics of defective versus non-defective pipe sections.

D. PULSE EDDY CURRENT
Pulsed Eddy Current (PEC) is one of the non-destructive (NDT) methods applied for electrically conductive materials' inspection at very high speeds [136]. A PEC system includes an exciter coil, a detector coil, an amplifier, and a voltage generator for excitation [137]. The coil, which is excited by a voltage pulse, can detect a greater depth of the material thickness in a more significant frequency [136]. This voltage creates a magnetic field around the sensor, and the resulting the magnetic field induces an eddy current in the conducting piece. As Fig.18 shows, the induced eddy currents affect the more significant area than the area physically covered by the sensor [137]. Ulapane  produced by the sensor is used to train the Gaussian process model [137]. Later on, Miro et al. used the NDT PEC-based method for metallic pipe wall thickness evaluation and corrosion inspection. In this method, a signal is excited and sent to the pipe wall, and then reflected. By analyzing the reflected signal, it is possible to analyze the pipe wall thickness [138]. One of the most significant advantages of this method is that it doesn't require direct physical contact between the test location and the sensor [139].

E. ION-SELECTION SENSORS FOR WATER QUALITY MONITORING
In some applications, it is needed to monitor the concentration of target analytes in the flow. However, since there is space limitation in the pipe, miniaturized sensors are needed. Researchers have recently developed small-sized sensors operated by the ion selection technique. In this technique, two electrodes of working and reference electrodes generate a voltage based on the concentration of the ion (target analyte) called potentiometric voltage. For example, PH is a health-related parameter in water and hydrogen ion, + represents the value for this parameter. Fig. 19 shows the operation principle of ion-selective sensors in which the potentiometric voltage ( ) is generated between the working electrode (WE) and the reference electrode (RE). The value of the parameter (e.g., PH) can be computed by using the Nernst equation [140]. In [141], the concept of multi-analyte biochip (MAB) is presented in which multi-parameters can be measured with one chip. In [111], a wireless sensor module is developed based on an active radio frequency identification (RFID) synchronizing the miniaturized ion-selective sensors and motion controller. The robot operates in an in-service water distribution network for quality monitoring.

F. MISCELLANEOUS METHODS
Researchers in [144] and [145] developed a practical wireless radio communication (using coaxial leakage cable) that can be used for localization of in-pipe robots and also developed a rotating probe using a piezo element that will be used for inspection using a touch sensor system in pipes. In [144], a free-swimming acoustic-based leak detection device is designed to operate in long pipelines and locate small leaks and bends in large pipes. To overcome the problems with regular ultrasonic inspection for automatic inspection in steel pipelines, the authors in [145] developed an ultrasonic inspection robot with an electromagnetic acoustic transducer (EMAT) that does not need a coupling medium in pipelines. The authors in [146] proposed a non-destructive testing method by using an In-Line Inspection (ILI) robot. They utilized several sensors such as ultrasonic, light, and Light Detection And Ranging (LiDAR) sensors to collect typical data features. They used some sensors in the defect detection system and the remaining sensors support navigation and improve data visualization. Also, for localization purposes, the IMU sensor is used to keep track of the traveling distances. In [128], the authors used pipe penetrating radar (PPR) to inspect two diameter pipes (21" and 36"). Their suggested method can determine the amount of H2S corrosion in these interceptors. When a hydrophone produces a sound wave in the kilohertz range, the metal pipes show vibrations. This vibration is different over space and can be used as a map in loop closing [147]. The use of the acoustic signal with pose-graph optimization for robot localization in a pipe has been demonstrated in [147]. They incorporated information from the measurement of an acoustic spatial field, such as the combination of a correlation-based matching method and a quadratic fit-based prediction method.

VI. IMAGE PROCESSING IN IN-PIPE ROBOTS
This section summarizes the algorithms and methods associated with image processing methods that have been implemented for in-pipe missions (the summary shown in Table III). Indeed, this categorization provides a comprehensive overview for future studies to explore the effective image processing methods that have been frequently implemented for those four specific tasks, including in-pipe localization, mapping, navigation, and inspection.
Generally, image processing can be categorized into image analysis and manipulation, image enhancement (including quality and resolution), image rectification, and image compression. Image processing has been widely utilized in inspection, fault detection, and localization inside the pipe environments.
The sensors that have been frequently implemented for image processing studies can be categorized into two groups: 1) Range sensors conclude range-type lasers or ultrasonic rangefinders, and 2) Different kinds of cameras include eye fish, monocular endoscopic, CCD, CCTV, time-of-flight (TOF) and RGBD cameras.   Image processing is applied to detect irregularity in pipes.
The distribution of Image Processing methods in the literature review associated with described main groups, including Localization, Mapping, Navigation, and Inspection, has been graphically presented in Fig. 20. It is revealed that Image processing contributes to 35% of utilization in in-pipe inspection and defect detection. Localization with 27%, navigation with 25%, and mapping with 13% are placed respectively. In another categorization, 77% of image utilization has been used for image analysis and detection (See Fig. 21

VII. SENSORS IN IN-PIPE ROBOTS
In this section, we list the common sensors used in the inpipe robots for different tasks in Table 4. This categorization helps the researchers to find the appropriate sensor for each task or method. Section VI lists the sensors or tools for image processing methods, so we do not list them here.

VIII. CONCLUSION
In this paper, we categorize in-pipe missions into four parts: localization, mapping. navigation, and inspection. In each part, we listed the core methods that were used to accomplish that mission. Image processing is a common approach in these techniques; we did a comprehensive categorization of the image processing methods and their usage in each mission. We also listed the sensors used in inpipe robots for each mission that helps the researchers pick an appropriate sensor for each method. This paper provides a comprehensive perspective for the researchers that work on in-pipe robots and need to have deep understanding of the methods, sensors, and their implementation technique in the in-pipe missions.