Multimodal Deep Learning Framework for Enhanced Semantic Scene Classification Using RGB-D Images

Typically, an abstract scene is made up of multiple less abstract elements, such themes or objects. The semantic gap between low-level visual characteristics and abstract scenes makes it very challenging to infer scenes from visual features. To get around these issues, researchers have naturally turned to inserting depth information into the typical RGB image as input. By using depth data, this technique—also referred to as RGB-D object detection or depth-aware SOD—aims to enhance saliency recognition in these kinds of scenarios. Our proposed technology has proven to be significantly better than the most advanced techniques available in this industry. The high effectiveness of our suggested approach is demonstrated by extensive kind of autonomous vision applications, such as robotic vision, sports analysis, and security systems. Our system’s primary feature is its original segmentation framework, which makes use of multi-object segmentation. This framework allows the system to identify objects and recognize a scene with resilience and precision. Subsequently, Convolutional Neural Networks (CNNs) are utilized to enhance the system’s overall recognition performance by further processing these extracted properties for scene recognition. The experimental outcomes using the NYUDv1 dataset and RGB-D scene dataset show $90.10 \%$, and $93.5 \%$ total accuracy, respectively. These results validate the applicability of the method and outperform current state-of-the-art approaches.