VR Digital Twin-based Dynamic Dataset Synthesis for Learning Channel Disaster Detection Network

Channels are a major component of water conservancy infrastructure and play a crucial role in agricultural irrigation, industrial production, flood retention and discharge. Cracks, fractures, and blockages caused by large floating debris can destroy the structural integrity of concrete riverbanks. Therefore, real-time detection of potential disaster is of great significance for ensuring the safety and stability of channels. However, recent deep learning-based methods for detecting disasters heavily rely on the high-quality of datasets. Yet, in the real world, it is challenging to collect datasets covering channel cracks, fractures, and large floating debris. To address this problem, we propose a Virtual Reality (VR) digital twinbased dynamic dataset generation method for synthesizing videos of channel damages and floating debris that are difficult to be captured in real-world scenarios. First, we employ a multiview image-based approach to recover camera poses and point cloud structures of the channels. Then, an epipolar line-based interactive editing method is used to construct a 3D channel model, while concrete cracks are reconstructed and augmented in a similar method. To generate high-quality dynamic scenes of fractures and damages, we adopt a 3D Voronoi diagrambased simulation method combined with a physics-based rigid body model. Next, a wave spectra-based approach is used to simulate turbulent flood in channel, while spherical quaternion interpolation is applied to model the motion of large floating objects such as cars and trunks. Finally, the synthesized data is annotated and used to train a YOLO deep learning model. Experimental results demonstrate that the synthetic dataset achieves a detection accuracy of over $96 \%$ on our test dataset with 30 fps, meeting the requirements for efficient and precise channel disaster detection.