Design of an Autoencoder-based Anomaly Detection for the DoH traffic System

DNS has encountered complex and diversified attacks over the years due to its special status on the Internet. The concept of DNS-over-HTTPS (DoH) has been proposed to protect user privacy by encapsulating DNS into HTTPS, which increases the difficulty of DNS tunnel detection but also faces some new attacks. In recent years, many researchers have discussed the detection methods of DoH tunnel. However, most of them need large-scale labeled datasets and extract statistical features, which is time-consuming and costs immense manpower, so it is impractical to be used in the real-world. In this paper, we developed a system called AADDS: an Autoencoder-based Anomaly Detection for the DoH traffic System consists of Traffic Capture module and Anomaly Detection module. The Traffic Capture module is developed based on nff-go, which can collect features stably in a high-speed Ethernet environment and greatly reduce the workload. For the Anomaly Detection module, we used bidirectional Long and Short-Term Memory (Bi-LSTM) to build an autoencoder network. Several essential experiments proved that our method has fewer parameters while ensuring higher accuracy, and it outperforms the state-of-the-art methods.