Biometric Evaluation on the Cloud: A Case Study with HumanID Gait Challenge

Biometric datasets are growing in size with time. Procurement of such datasets and resources for development and evaluation of biometric algorithms is expensive, time consuming and often requires expertise in systems software. Our goal in this project is to build a cloud-based evaluation system, which can host a common dataset and allow the submission of algorithms either as source code or Linux x-86 executable, to enforce a standard experimental protocol, and to provide results in a standard format. This facilitates comparing algorithms with each other and benchmarking progress. In order to efficiently service these algorithms, we need expensive computers with lot of storage and processing power. Having such systems eliminates the need for procurement of datasets and resources for experimentation, thus lowering the barrier for engaging in biometrics research. This style of cloud-based online evaluation system will encourage other biometric and research communities to build similar systems. The preferred solution to deploy this web-app is using Amazon Web Services, which provides computing power as well as storage capacity. In this paper, we share our experience with transitioning the HumanID Gait Challenge from traditional data+code type structure to a cloud based solution. It is available at http://marathon.csee.usf.edu/GaitBaseline/gaitcloud.