A benchmarking methodology for NPU-based stateful firewall

It is currently a challenge to evaluate and compare network processor-based (a.k.a. NPU-based) applications due to the heterogeneity of the application designs and hardware architectures. Many research works have been done in the area of packet classifications, which were directed towards proposing high-speed firewall implementation algorithms. However, few researches focus on providing standard benchmarking methodology for a specific class of firewall. This paper presents a benchmarking methodology for NPU-based stateful firewall. The aim is to allow 'apple-to-apple' comparison among similar firewalls available in the market, and to provide practical benchmarks that exhibit realistic performance numbers. The key aspects of this work are benchmark specifications, separation of benchmark granularity in a layered manner, and the means of measurement. Finally, a system-level benchmark is applied on a stateful firewall implemented on Intel's IXP1200 network processor as a proof-of-concept of this work.