Achieving High-Performance Video Analytics with Lightweight Cores and a Sea of Hardware Accelerators

High performance and energy efficient video analytics systems that can extract rich metadata from voluminous visual content, will enable a variety of high-value surveillance, driver assistance, video tagging, and first person analytics systems. These big-data applications are pervasive across retail, automotive, medical, agriculture and security domains. However, current trends in general purpose and multicore architectures will not keep pace with the growing computational demands of cutting edge visual perception algorithms. Hardware acceleration is crucial to surpassing what is realizable on modern multicore and GPGPU architectures. In this paper we detail a Sea-of-Accelerators, SoA, platform that combines a mix of macro-accelerators, microaccelerators, and lightweight processors to achieve high performance and energy efficiency in video analytics applications. In this paper, we describe a framework for video and image analytics and highlight its benefits with a case study of a customized visual saliency accelerator. We describe the architecture of a full custom macro-accelerator that is suitable when raw performance is of critical importance. As an alternative, we illustrate the composition of an accelerator from a constituent of loosely coupled microaccelerators and evaluate the performance achievable when performance and flexibility are competing objectives.