A Comprehensive Approach for Enhancing Deep Learning Datasets Quality Using Combined SSIM Algorithm and FSRCNN

The quality of the datasets used in deep learning problems is crucial in almost all application domains. There are a variety of common issues that can degrade datasets quality, including problems like data duplication, insufficient image resolution, and low data quality. Well-established techniques exist for mitigating these problems individually, such as using the SSIM algorithm to efficiently find similarities and duplicates in data or leveraging neural networks like FSRCNN to increase image resolution and quality. However, while each of these approaches offers its own unique benefits, they also possess some inherent limitations and disadvantages when applied in isolation. To mitigate the disadvantages of individual methods and provide a more comprehensive solution, a data preprocessing technique is proposed which involves combining the preceding algorithms and applying them as a single pipeline. The key insight is that SSIM's ability to identify duplicates combined with FSRCNN's power to super-resolve images can offer complementary strengths. Experiments conducted demonstrate that this hybrid approach achieves an improvement of approximately 3% in classification accuracy on the CIFAR-100 image dataset benchmark, with only 1.5 times increase in computational training duration. The joint technique outperforms using SSIM or FSRCNN independently. This highlights the potential for hybrid solutions to enhance datasets quality. Further research can explore combinations of other complementary data enhancement methods and study their performance across diverse datasets and machine learning models. The early findings clearly show that using multiple techniques together can improve model accuracy by addressing various issues with low-quality datasets.