Reconstructing Detailed Human Body Using a Viewpoint Auto-Changing RGB-D Sensor for Rescue Operation

In the aftermath of a disaster, such as an earthquake or the collapse of dilapidated buildings, it is necessary to enter the ruins to rescue the wounded, which is often accompanied by the casualties of rescuers. We propose a wounded body reconstruction system for rescue operations in disaster zones to replace humans with robots for rescue in hazardous environments. However, many limitations such as self-scanning constraints, reliable human semantic parsing, accurate surface reconstruction, and plausible motion tracking make this work even more challenging. We employ a parametric model to break the self-scanning constraints and obtain human semantic parsing fundamentally. To enable the parametric model to present human detail, we improved the human parametric model by defining a detail layer applying to the template as an additional offset. We propose automatically changing the viewpoint according to the human pose with the help of a robotic arm to obtain a dense and complete observation and achieve a high precision surface reconstruction. To meet the requirements of the rescue operation, we propose a new pipeline to track the wounded motion accurately. The experiments demonstrate that our method can accurately recover the surface of the human body and achieve accurate motion tracking free from the constraints of self-scanning and preserving reliable human semantic parsing.