Multi-label Image Recognition with Asymmetric Co- Dependency Graphs

Multi-label image recognition is a practical and challenging task. Modeling co-occurrence dependencies between categories is the key to improve performance. Existing methods use conditional probability to measure co-occurrence dependencies, and represent co-occurrence dependencies among all categories in the form of directed graph. Then the Graph Convolution Network (GCN) is applied on the directed graph to transfer dependent category features along the edge direction. However, the occurrence frequencies of different categories are different. Accordingly, the conditional probabilities between a pair of common and rare categories are highly asymmetric so that most rare categories have no in-edges to receive transferred knowledges from other categories. Therefore, this paper investigates the effects of edge direction between two co-occurred categories on the recognition performance, then proposes a model to work on a pair of directed graphs to learn a comprehensive representation of co-occurrence dependency. Extensive experiments on public benchmarks show that our method can achieve better performance than baseline models. On some multi-label image recognition datasets with strong co-occurrence dependencies, our method can improve the mAP by 4%.