Driver State Modeling Through Latent Variable State Space Framework in the Wild

Analyzing the impact of the environment on drivers’ stress level and workload is of high importance for designing human-centered driver-vehicle interaction systems and to ultimately help build a safer driving experience. However, driver’s state, including stress level and workload, are latent variables that cannot be measured on their own and should be estimated through sensor measurements such as psychophysiological measures. We propose using a latent-variable state-space modeling framework for driver state analysis. By using latent-variable state-space models, we model drivers’ workload and stress levels as latent variables estimated through multimodal human sensing data, under the perturbations of the environment in a state-space format and in a holistic manner. Through using a case study of multimodal driving data collected from 11 participants, we first estimate the latent stress level and workload of drivers from their heart rate, gaze measures, and intensity of facial action units. We then show that external contextual elements such as the number of vehicles as a proxy for traffic density and secondary task demands may be associated with changes in driver’s stress levels and workload. We also show that different drivers may be impacted differently by the aforementioned perturbations. We found out that drivers’ latent states at previous timesteps are highly associated with their current states. Additionally, we discuss the utility of state-space models in analyzing the possible lag between the two latent variables of stress level and workload, which might be indicative of information transmission between the different parts of the driver’s psychophysiology in the wild.