Adaptive Inference Using Hierarchical Convolutional Bag-of-Features for Low-Power Embedded Platforms

Using early exits provide a straightforward way to implement models that can adapt on-the-fly to the available computational resources. However, early exits in many cases suffer from significant limitations, which often prohibit their practical application, especially when placed on convolutional layers with narrow receptive fields. In this work, we propose a method capable of overcoming these limitations by a) using a Bag-of-Features (BoF)-based pooling approach, that allows for keeping more information regarding the distribution of the extracted feature vectors, while also maintaining more spatial information and b) employing a simple, yet effective, hierarchical approach for designing the exits, allowing for efficiently re-using the information that was already extracted by the previous layers. It is experimentally demonstrated that the proposed approach leads to significant performance improvements, allowing early exits to be a more practical tool that can be used in many real-world embedded applications.