Category-Aware Test-Time Training Domain Adaptation

Machine learning models often struggle to generalise to out-of-distribution (OOD) data. One promising solution for solving this problem is test-time training domain adaptation, which adjusts a trained model to the new test data without revisiting the source dataset, thus preserving the privacy of source data. However, existing test-time training methods have not considered the mining of category information of the test data in the training for domain adaptation, thus suffering in inaccurate domain alignment. In this paper, we propose a novel method called category-aware test-time training (CAT
³
) to adapt the pre-trained model to test data on the fly. CAT
³
first trains a model with multiple diverse classifiers using the source datasets and conducts a source data summarisation. Then, it assigns the pseudo-labels for the test data that have consistent classification results from the trained classifiers and adjusts the pre-trained model by aligning these reliable test data to their corresponding source data categories iteratively. Unlike existing test-time training methods, CAT
³
reveals the category information of the test data and aligns these reliable test data to the source data at the category level instead of the dataset level. Empirical results demonstrate that CAT
³
can outperform the current state-of-the-art methods on several benchmarks, indicating our proposed method’s effectiveness.