Recent advances in low-power wireless networking have enabled remote and nonintrusive access to households' electric meter readings, allowing direct real-time feedback on electricity consumption to home owners and energy providers. Fine-grained electricity billing based on appliance power load monitoring has been investigated for more than two decades, but has not yet witnessed wide commercial acceptance. In this paper, we argue that the required human supervision for profiling and calibrating appliance load monitoring systems is a key reason preventing large-scale commercial roll-outs. We propose ANNOT, a system to automate electricity data annotation leveraging cheap wireless sensor nodes. Characteristic sensory stimuli captured by sensor nodes placed next to appliances are translated into appliance operating state and correlated to the electricity data, autonomously generating the annotation of electricity data with appliance activity. The system is able to facilitate the acquisition of appliance signatures, training data and validate the monitoring output. We validate the concept by integrating the automated annotation system to the RECAP appliance load monitoring system.