Rainbow UDA: Combining Domain Adaptive Models for Semantic Segmentation Tasks

In this paper, we propose Rainbow UDA, a framework designed to address the drawbacks of the previous ensemble-distillation frameworks when combining multiple unsupervised domain adaptation (UDA) models for semantic segmentation tasks. Such drawbacks are mainly caused by overlooking the magnitudes of the output certainties of different members in an ensemble as well as their individual performance in the target domain, causing the distillation process to suffer from certainty inconsistency and performance variation issues. These issues may hinder the effectiveness of an ensemble that includes members with either biased certainty distributions or have poor performance in the target domain. To mitigate such a deficiency, Rainbow UDA introduces two operations: the unification and the channel-wise fusion operations, to address the above two issues. In order to validate the designs of Rainbow UDA, we leverage the GTA5 $\to$ Cityscapes and SYNTHIA $\to$ Cityscapes benchmarks to examine the effectiveness of the two operations, and compare Rainbow UDA against a wide variety of baseline approaches. We also provide a set of analyses to show that Rainbow UDA is effective, robust, and can evolve with time as the ensemble grows.