Cross-Supervised Learning for Instance Level Multi-Task Training

We expect to accomplish multiple tasks through one model, especially the omni-directional analysis of an instance, such as detection, segmentation, keypoint, parsing and so on. But according to the existing region-based approach pipeline, it requires a dataset to provide all kinds of annotations for each task. In order to train an instance-based model using several datasets with different annotations, we analyze two feasible schemes and propose a better method: Cross-Supervised Learning. Cross-Supervised Learning is conceptually simple and flexible. It is suitable for region-based approaches where different tasks can share backbones and task-specific heads can be processed in parallel. Cross-Supervised Learning is end-to-end training, and greatly reduces the training time compared with other methods. In this paper, we use Cross-Supervised Learning to train a model on three different datasets, which can concurrently perform human detection, keypoint detection, human part parsing and densepose estimation. We also study the transfer learning relations across three datasets, and exploit them to improve performance in the most efficient way. It is noteworthy that we don’t need to optimize the model structure, and we can achieve state-of-the-art results on several benchmarks.