COMET: Contrastive Mean Teacher for Online Source-Free Universal Domain Adaptation

In real-world applications, there is often a domain shift (distribution change) from training to test data. This observation recently resulted in the development of test-time adaptation (TTA). It aims to adapt a pre-trained source model to the test data without requiring access to the source data. Thereby, most existing works are so far limited to the closed-set assumption, i.e. there is no category shift (class change) between source and target domain. We argue that in a realistic open-world setting a category shift can appear in addition to a domain shift. This means, individual source classes may not appear in the target domain anymore, samples of new unknown classes may be part of the target domain or even both at the same time. Moreover, in many real-world scenarios the test data is not accessible all at once in form of a dataset but arrives sequentially as a stream of batches which require an immediate prediction. Hence, TTA must be applied in an online manner. To the best of our knowledge, the combination of these aspects, i.e. online source-free universal domain adaptation (online SF-UniDA), has not been studied yet despite its practical relevance. In this paper, we are the first ones to tackle this challenging task. We introduce a Contrastive Mean Teacher (COMET) tailored to this novel scenario. It applies a contrastive loss to rebuild a feature space where the samples of known classes build distinct clusters and the samples of new classes separate well from them. It is complemented by an entropy loss which ensures that the classifier output has a small entropy for samples of known classes and a large entropy for samples of new classes to be easily detected and rejected as unknown. To provide the losses with reliable pseudo labels, they are embedded into a mean teacher (MT) framework. We evaluate our method across two datasets and all category shifts to set an initial benchmark for online SF-UniDA. Thereby, COMET yields state-of-the-art performance and proves to be consistent and robust across a variety of different scenarios. Our code is available at https://github.com/pascalschlachter/COMET.