Detection of Fileless Malware

Fileless malware resides solely in memory, thereby leveraging advanced evasion techniques to bypass conventional antivirus software, which relies on disk-based scanning. Existing approaches such as static analysis, dynamic analysis, and memory forensics often fail due to their dependency on large databases, high resource requirements, and limited capability to detect highly obfuscated threats. This paper proposes a detection framework using a Windows kernel driver designed to operate as an Endpoint Detection and Response (EDR) solution capable of real-time identification of fileless malware. Our approach utilizes a Windows kernel-mode driver to intercept and monitor low-level system activities, including API calls, system calls, and memory allocations. The kernel driver continuously hooks critical functions to detect suspicious behavior patterns typical of fileless malware. The driver maintains a dynamic whitelist to differentiate from malicious activities. With the integration of an ATmega32U4 microcontroller configured as a human interface device (HID), we can deploy custom firmware as part of the malware detection framework. This enables the system to monitor the status of specific services on the target system, identifying the status of the services. In addition, the microcontroller also facilitates the restarting and redeploying of the service, ensuring robust and efficient service management while maintaining system integrity.