Detection and classification of TCP/IP network services

Computer intruders are employing more sophisticated techniques to compromise computer systems. Once compromised, in most cases, intruders install remote terminal software to ensure continued, undetectable access to the victim site bypassing standard system audit and security features. Detection of this type of intruder activity was a problem for law enforcement during a computer intrusion investigation that went to prosecution in Australia. The increasing availability of remote terminal software to intruders poses a significant problem to both the detection and monitoring of an intruder's activities. This paper discusses an approach to the analysis of network traffic to detect the presence of unauthorised and anomalous network services. The aim of the project is the development of a network connection signature for common network services, therefore allowing connection type recognition independent of the port information. The specific service signatures can then be used to correlate port information with observed connection types facilitating the detection of anomalous and unauthorised network connections. The detection of anomalous connections may indicate the presence of unauthorised modifications to systems on the network being monitored or the installation of illicit remote terminal software on those systems. A modified neural network was used to analyse the network traffic captured for the experiment. Apart from its learning and generalisation properties, the neural network engine lends the application the ability to adapt to the different network environments on which the software may be employed.