Scalable Hindsight Experience Replay based Q-learning Framework with Explainability for Big Data Applications in Fog Computing

Nowadays Internet of Things (IoT) applications are proliferating with explosive growth and their computational load is very high. Fog computing is an important structure used for processing the IoT devices data at cloud proximity. Big data applications demand scalable computing with stringent performance requirement. The currently available machine learning models do not match the growing scale of big data applications and lack explainability. In this paper an explainable Q-learning framework with hindsight experience replay (Q-HER) is developed to provide holistic scalability solution for big data applications using minimum number of fog nodes. The reward engineering process is streamlined and each episode with original goal and subset of other goals is repeated to yield high quality scalability policies. The mathematical modeling reveals that the generated scalability decisions satisfy the quality assurance parameters like correctness, robustness, model relevance, and ∊ -Differential Data privacy. The performance of the proposed Q-HER is found to be good towards the performance metrics like accuracy, latency, cost, and average resource wastage under two different scenarios of limited and unlimited processor fog nodes.