A Neural Network-Based Optimal Tile Size Selection Model for Embedded Vision Applications

Graph-based programming models for accelerating computer vision applications such as OpenVX have shown the potential to facilitate the expression of high-performance computer vision algorithms on embedded platforms effectively. The motivation behind this graph-based paradigm is to decouple algorithms (vision pipelines and data flow among them) from schedules (runtime) so that exploiting parallelism and locality through optimizations such as tiling becomes more feasible. While OpenVX allows benefits from tiling easily through a provided tiling extension, crafting an optimal tile size selection (TSS) model is still challenging due to the inherent nonlinearity between tile size and performance. In this paper, we propose a neural network model that utilizes unique access patterns in OpenVX applications to learn and predict optimal tile sizes effectively. Our mechanism extracts optimal features from raw profiling data such that it maximizes the clustering quality of vision operators, thereby generating highly accurate TSS models. The experimental evaluations using both individual kernels and full application benchmarks show that our proposed model predicts optimal tile sizes accurately; the prediction error is 10% in the worst case, and mostly around 3%. More importantly, compared with previous TSS methods based on analytical models and the state-of-the-art neural network approach, our proposed method can predict optimal tile size for vision applications with better accuracy and less inference time.