Continuous-Valued Annotations Aggregation for Heart Rate Detection

In the medical field, experts usually annotate the bio-signals manually, and this is regarded as the gold standard. The manual annotating mode is time-consuming so widely replaced by an automated annotating algorithm. To address the low precision and low robustness of algorithm, we used a probabilistic model to synthesize the heart rate (HR) annotations from multiple annotators for electrocardiograph (ECG) signals and inferred the underlying true annotations and the precision of each annotator when the ground truth was not available. We further introduced signal quality indices in the model to improve our estimation. The 100 noisy ECG recordings in 2014 PhysioNet/computing in cardiology challenge database were divided into two parts, and various annotations for HR were generated by six available annotators. By employing the expectation maximization algorithm, we obtained the estimated true annotations for 80 recordings, and this result had an improvement not only over the best single annotator (17.46%) used in this paper but also to the mean and median strategies (the highest of 23.12% and 42.23%). Furthermore, the estimated precision of the single annotator from the proposed model served as the weight of the test data. In independent test, the weighted average of multiple annotations was superior to the single annotator and the mean strategy on 20 recordings, and its root mean square error (14.22 bpm) was close to that (13.96 bpm) of the proposed model on 80 recordings, demonstrating the robustness of the proposed continuous-valued annotation aggregation model.