Brutector: A Probabilistic Detection Model for Bruteforce Attacks in SSH Server

The network traffic is growing at unprecedented rates owing to high speed network connectivity and numerous devices adding to it every day. The recent change in work pattern has shifted reliance on access to remote servers and cloud based platforms. The security of the user information and hosts is based on login credentials. Thus, they are vulnerable to bruteforce attacks. The paper looks at bruteforce attacks in SSH server. SSH is an application layer protocol which provides a secure encrypted communication channel between the host and the client. The paper proposes a probabilistic approach to classify network traffic emerging from an IP into 3 categories namely, Benign, Suspicious and Malicious. Events of good and bad activity are extracted from the traffic which is used to update the belief(probability) of IP in the three states. This helps in filtering out malicious traffic to the server and thus protects it from attacks. The approach stands out as it does not look for bruteforce in isolation. Here, other bad events like portscan helps in better classification of IPs. Brutector resulted in an accuracy of 96.8% on the test dataset. The dataset is collected by setting up an SSH server open to the network. It contains the real traffic and captures the behaviour of attackers actively seeking for hosts.