A Modular Approach to Programming Multi-Modal Sensing Applications

The growing ubiquity of personal connected devices has created the opportunity for a wide range of applications which tap into their sensors. The sensing requirements of applications often dynamically evolve over time depending on contextual factors, evolving interest in different types of data, or simply to economize resource consumption. The code implementing this evolution is typically mixed with that of the application's functionality. This paper presents ModeSens, an approach to modeling and programming multi-modal sensing requirements of applications. ModeSens enhances modularity and reuse by separating the application's functional concerns from those of the evolution of its sensing requirements, which are modeled separately as transitions between modes. A graphical interface allows programmers to program reusable multi-modal sensing requirements for applications. New modes can be specified explicitly or learned. Particularly, we present a way for new modes to be learned by recognizing mode signatures from examples of sensor data sampled in the mode. Both ModSens and the mode learning system have been prototyped, the latter specifically for Activity Recognition. Our experimental evaluation demonstrates that ModeSens incurs low processing overhead and consequently has a small energy footprint. Although the learning system, by design, is for occasional use, we also documented its processing and energy overhead.