NEM: Nested Ensemble Model for scene recognition

Scene recognition predicts the category of the scene in which an image is taken. With ensemble learning as its basis, we propose a novel scene recognition method called the Nested Ensemble Model (NEM). As a first step, we employ ensemble learning to construct HybridNets that use object and scene information simultaneously. To take advantage of different neural network architecture paradigms, we developed HybridNet-Transformer, HybridNet-ConvNeXt, and HybridNet-CNN. Our NEM is then constructed by fusing these models together based on ensemble learning. By doing this, NEM not only benefits from object and scene knowledge together, but also from CNNs’ locality and global modeling of Transformers and ConvNeXts. On MIT-67 and Scene-15, two of the most popular scene recognition datasets, NEM achieved to the accuracy of 89.30% and 96.80%, respectively. These results indicate that this approach is superior to existing methods.