LLM-CoSen: Revisiting Collaborative Sensing With Large Language Models (LLMs)

Collaborative sensing has emerged as a novel sensing paradigm, entailing multi-sensor data sharing and multimodal modeling to collaboratively understand sensing behaviors. However, current solutions, i.e., data-level and decision-level fusion methods, fall short of generality, expert knowledge, and holistic/chronic perspective. In this paper, we propose LLM-CoSen to revisit collaborative sensing with Large Language Models (LLMs). Specifically, LLM-CoSen designs a semantic-level fusion approach for inference results for collaborative sensing. Such an approach is characterized by its generality, making it applicable to any heterogeneous devices, and its expert knowledge incorporation, which provides chronic, holistic, and insightful perspectives on the inference results. Regarding inference absence challenges, we propose a personalized model design method to constrain inference time, and a voting-based two-pass prompt engineering strategy for token completion. Regarding inference error challenges, we propose an accuracy restoration strategy for personalized models, and a two-level error estimator coupled with self-correction. Experimental results of human digital system use case on four corresponding benchmark datasets show LLM-CoSen can decrease inference absence by 72.83% and inference errors by 7.65% on average.