Intrusion detection testing and benchmarking methodologies

The ad-hoc methodology that is prevalent in today's testing and evaluation of network intrusion detection algorithms and systems makes it difficult to compare different algorithms and approaches. After conducting a survey of the literature on the methods and techniques being used, it can be seen that a new approach that incorporates an open source testing methodology and environment would benefit the information assurance community. After summarizing the literature and presenting several example test and evaluation environments that have been used in the past, we propose a new open source evaluation environment and methodology for use by researchers and developers of new intrusion detection and denial of service detection and prevention algorithms and methodologies.