Hypercall-Oriented Abnormal VM Status Detection System: A Non-Intrusive Solution for Both Hypervisor and Guests

Hypervisor is a VMM (Virtual Machine Monitor) that creates and runs multiple VMs (Virtual Machines) through abstracting resources from a physical machine. Hypercall is a special and crucial call used in virtualized systems as it serves as a main communication channel between VMs and the hypervisor. However, hypercall attacks occur when an attacker manipulates the communication channel, and it could cause abnormal VM status, potentially leading to the abnormal resource allocation of the host OS (Operating System) and crash of VMs. Therefore, the virtualized system should execute abnormal VM status detection to identify potential abnormal behaviors to protect VMs and the host OS; however, existing works are either for reconstructing the hypervisor or hardware isolation, not for the VM status detection for abnormal hypercall. This study develops a hypercall-oriented abnormal VM status detection system called HypercallDetector based on the following three innovations: 1) we implement a hypercall tracing based on eBPF to obtain the hypercall-related running status (including CPU usage, memory usage, network traffic, etc.) of each VM; 2) we implement a window division technology to divide the VM status into multiple status windows of the same size, and appropriate window size with balanced detection precision (95.0%) and latency (within 8.8 ms) obtained by proposing the window regulator; and 3) we implement a CS-H algorithm (Compressing Sensing for Hypercall) to distinguish whether the VM status is abnormal. HypercallDetector shows higher precision and lower latency than its opponent and consumes only 8.6% CPU of single core and 0.3% memory usage when starting 240 VMs.