A comprehensive solution for inspecting cable brackets in aircraft based on augmented reality and deep learning

The reliability of the avionics system is crucial to the safety of aircraft. In the process of aircraft final assembly, the avionics system is fixed by tens of thousands of cable brackets. In order to guarantee the reliability of the avionics system, cable brackets need to be inspected before the assembly of the avionics system. Such an inspection task is usually performed by manual verification with CAD models, hence is time-consuming, labor-intensive, and error-prone. In order to improve inspection efficiency, many researchers attempt to develop automatic inspection systems for aeronautical mechanical assemblies. However, cable brackets are numerous and ubiquitous all over the aircraft, so cable bracket inspection is quite different from other aeronautical mechanical assemblies. In this paper, a comprehensive solution is proposed for inspecting cable brackets in aircraft. Firstly, the general architecture of the cable bracket inspection system is proposed based on augmented reality and deep learning. Especially, adaptive inspection solutions based on 2D/3D vision are proposed for aircraft structures with different geometrical features. Then, the prototype system is deployed on two kinds of platforms. One is an AR glasses-based 2D visual inspection system for 2D planar structures, and the other one is a handheld tablet-based 3D visual inspection system for 3D curved surface structures. Finally, the verification experiments are carried out on the C919 aircraft final-assembly workshop, and the experimental results show that the approach is promising for large-scale industrial applications.