App-centric and environment-aware monitoring and diagnosis in the cloud

Infrastructure-as-a-Service environments are becoming increasingly popular. When there is a failure, many applications require service restoration within a few seconds. Reaction to failures in Cloud is still slow for many applications. Monitoring is limited to instance metrics that are not conducive to precise diagnosis due to complexity of virtualization in physical hosts. Interferences among different VMs complicates the diagnosis. We propose a new dynamic monitoring module as a part of multi-agent based cloud management framework named LAMA. Applications are at the center of our framework. Agents distributed throughout the cloud infrastructure are responsible for aggregating metrics and performing customized diagnostics for each application. This reduces overhead, removes centralized bottlenecks and allows customized configuration with finer granularities. Our approach is also environment aware as each app agent has access to the application, virtual instances and hosting infrastructure metrics. This feature enables the creation of more efficient diagnostics algorithms customized to application's needs. We develop and deploy LAMA in our datacenter to demonstrate (1) how an integrated approach with access to the state of the app's improves efficiency of failure detection, (2) how our monitoring and diagnosing architecture can improve load distribution in the network and (3) the impact of finer granularity on failure detection time.