Beyond Grids: Scaling Up Continuous Kernels via Adaptive Point Representations for Sensor-Based Human Activity Recognition

Deep convolutional neural networks (CNNs) have recently achieved outstanding success in human activity recognition (HAR) due to their ability to automatically extract hierarchical features. However, prior most works have always relied on a discrete grid-based formulation, which lacks flexibility and adaptiveness to handle multimodal activity signals captured from on-body sensors, since they are often distributed unevenly over input signal domain. To improve the shortcoming, we propose to model the convolutional kernels as a continuous function via leveraging adaptive point representations (APRs) and interpolation schemes, where each sampling location can contain its own weight parameters and be independently updated. Through connecting these nearby points, it only requires fewer parameters to cover the interesting area in low-frequency areas, while more points may be aggregated to capture high-frequency components, yielding richer expressiveness. In contrast to prior grid-based arts, this article is the first attempt to show the effectiveness and efficiency of continuous convolution in large-kernel setting, presenting significant performance gain over multiple HAR benchmarks. An actual evaluation is implemented on an ARM-based mobile device.