A comparison of performance metrics for event classification in Non-Intrusive Load Monitoring

In this work, we analyse experimentally the behaviour of 18 different performance metrics when applied to classification algorithms in event-based Non-Intrusive Load Monitoring, identifying relationships and clusters between the measures. Our results indicate that performance metrics have more in common than what was initially expected. Our results also suggest that in this multi-class classification problem, researchers should avoid micro-average and unweighted macro-average metrics in favor of their weighted macro-average counterparts. Finally, the results also suggest that probabilistic measures can provide important information that is not available when using more traditional performance metrics.