Presenting a Reliability Evaluation Framework for Cloud-Based Machine Learning in Microservices

Machine learning has seen wide adoptions, although its deployment is resource-intensive and time-consuming with interoperability and performance concerns. Cloud deployment and microservice architecture have been chosen by researchers and practitioners as solutions to these issues. However, the reliability aspects of such systems have yet to be explored. The reliability of any system is important, especially for the end users. To this end, we proposed an evaluation framework to equip practitioners with guidelines that consist of metrics and threshold selection, which can be integrated with a software's testing life cycle. We conducted an analysis of ISO/IEC 25023:2016 standard to study the software reliability requirements. Afterwards, we demonstrated the utility of our framework on a healthcare application that runs two CNN models and multiple services. The evaluation within this work was done for 84 hours and the proposed framework successfully guided the reliability evaluation of the selected case study. This work concluded that the proposed evaluation framework successfully gauged the reliability of cloud-based machine learning in microservices.