Multi-Task Incremental Learning with Refined Pseudo-Label Generation for Robotic Semantic Grasp Detection

Semantic grasp detection in robotics exemplifies the application of multi-task networks in this field. In this paper, we propose a novel framework to address the challenges of lacking multi-task datasets and poor scalability of multi-task networks. The proposed approach encompasses incremental learning of multiple tasks while exclusively leveraging data from the current task during the training of new task branches. To efficiently utilize ground truth labels for the current task, we additionally introduce a knowledge distillation based method that incorporates task-relation functions into the generation of pseudo-labels. These functions effectively address the issue of domain mismatch in multi-task incremental learning, thereby enhancing the overall performance of joint tasks. The effectiveness of the proposed approach has been evaluated through comparative experiments in the context of robotic semantic grasp detection. Furthermore, we investigated the impacts of different task-relation functions within the MTIL framework. By integrating these functions, we achieve experimental performance comparable, or even surpassing, that of joint training while utilizing only half of the labeled data.