Cryptographic Contests: Toward Fair and Comprehensive Benchmarking of Cryptographic Algorithms in Hardware (Abstract)

A fair comparison of functionally equivalent digital systems is a challenging and non-trivial task. Objective difficulties include lack of standard interfaces, influence of tools and their options, the dependence of the obtained results on the time spent for optimization, etc. In cryptography, there is a strong need for such a fair evaluation, associated with the way new cryptographic standards are being developed, namely through open competitions of algorithms submitted by research groups from all over the world. Such competitions included for example the AES contest in the U.S., the NESSIE and eSTREAM competitions in Europe, and the CRYPTREC project in Japan. At this point, the focus of attention of the entire cryptographic community is on the SHA-3 contest for a new cryptographic hash function standard, organized by NIST. In this talk I will analyze typical evaluation pitfalls and objective challenges facing the evaluators of cryptographic algorithms from the point of view of performance in hardware. I will present practical benchmarking methodologies and tools that facilitate overcoming these difficulties, and can be used to fairly compare competing algorithms, hardware architectures, development platforms, languages, and tools. I will also discuss the remaining challenges and difficulties worth exploring in the future, and the ways of generalizing experiences gained from benchmarking cryptographic hardware, and applying them to other domains, such as communications and digital signal processing.