Robot position control in pipes using Q Learning

In the most critical hydro crisis in Brazil, 37 percent of the whole amount of treated water is wasted before reaching consumers. A robot with a position control to travel inside a pipe is an important step in the pursuit of an autonomous solution to detect and correct pipes failures. This paper shows a Q Learning controller algorithm implemented using a microcontroller in a mechanical body of a commercial pipe inspection robot. Using only the measurements of a gyroscope, and controlling the wheels' motors on the left and right sides, the controller learned the best set of movements to ride inside a 300mm sewer pipe, in the tested conditions. Real tests in a 300mm pipe were performed using the developed algorithm and it was compared to a random movement and to a straight forward movement.