Efficient Feature Extraction for Image Analysis through Adaptive Caching in Vector Databases

For popular machine learning (ML) models like MobileNetV3 and ResNet, the extraction and storage of image features serve as a critical step for both the training and inference stages. Unfortunately, the state-of-the-art ML systems exhibit a performance bottleneck on the extraction and storage of image features. To that end, this paper proposes a new caching subsystem that employs an in-memory vector database. The key idea is to preprocess images using MobileNetV3 and ResNet, extract features, and store them in an in-memory vector database instance, which is realized by a MySQL deployment on the RAM disk that is extended with similarity search. The implemented enhancements, including batch insertion and parallel processing using ThreadPoolExecutor, have yielded a notable 36.6% improvement in the execution time of MobileNetV3 image processing compared to the sequential approach. For ResNet, the execution time without caching was 15.46 seconds, and was reduced to 11.61 seconds with parallel processing, resulting in a notable improvement of 24.9%.