Error Correction in Robotic Assembly Planning From Graphical Instruction Manuals

To enhance the accuracy of robotic assembly planning by understanding the graphical instruction manual, this paper proposes a novel two-step error correction method. While constructing the Assembly Task Sequence Graph (ATSG) from the instruction manual, we performed an error correction focusing on the component, symbol, speech bubble, and model number included in the manual. The component and symbol information were used to check the correctness of the manipulated components, the needed motion, and tool used in a single step of the assembly task. The speech bubble information was used to remove the repeatedly drawn components on the instruction images. The model number was recognized to distinguish components that have different models but were classified into the same class in the object detection procedure. After constructing the ATSG, we additionally performed the error correction by checking the total number of components used for the assembly task. The effectiveness of the proposed method was verified by comparing the ATSG generated from four different error correction methods of five different chairs. The results show that the proposed method decreased the influence of wrong detection results to generate an ATSG with higher accuracy and generalized the previous ATSG for different types of chairs. Finally, we built an extended ATSG from the graphical instruction manual of a kid chair, and the chair assembly task was performed by a dual-arm robot.