Monitoring and Autoscaling IaaS Clouds: A Case for Complex Event Processing on Data Streams

Cloud computing is the notion for delivering access to scalable on-demand computing resources and IT services. The resource management system in IaaS Clouds dynamically allocates the resources based on predefined customers' needs using Service Level Agreements (SLAs) between the Cloud provider and customers. One of the challenges of resource management is to continuously monitor resource utilization, manage, and adjust these resources in real-time fashion to meet the SLAs while not over provisioning resources. Though, there exist numerous Cloud monitoring solutions, they are often highly specialized and restrict the administrator in defining automation rules. Our work aims to develop a framework based on concepts from Complex Event Processing (CEP) and data stream processing where data from various primitive metrics streams are collected and treated as event streams (e.g., from CPU, memory, and disk sensors). By automatically detecting complex patterns and relationships among these primitive events, we can detect and understand more high-level situations from the information provided by sensor steams. In this paper, we describe our CEP-based resource monitoring framework and discuss a use case for implementing auto scaling facilities to the Proxmox platform.