In this work, we introduce methods for studying psychological arousal in naturalistic daily living. We present an activity-aware arousal phase modeling approach that incorporates the additional heart rate (AHR) algorithm to estimate arousal onsets (activations) in the presence of physical activity (PA). In particular, our method filters spurious PA-induced activations from AHR activations, e.g., caused by changes in body posture, using activity primitive patterns and their distributions. Furthermore, our approach includes algorithms for estimating arousal duration and intensity, which are key to arousal assessment. We analyzed the modeling procedure in a participant study with 180 h of unconstrained daily life recordings using a multimodal wearable system comprising two acceleration sensors, a heart rate monitor, and a belt computer. We show how participants' sensor-based arousal phase estimations can be evaluated in relation to daily activity and self-report information. For example, participant-specific arousal was frequently estimated during conversations and yielded highest intensities during office work. We believe that our activity-aware arousal modeling can be used to investigate personal arousal characteristics and introduce novel options for studying human behavior in daily living.