Reliable signal classification using quality assurance and clustering-an application to sleep stage scoring

This paper examines how quality assurance techniques can be applied to the challenging, real-world problem of sleep stage scoring. Quality assurance not only promotes the user's confidence in computer-generated scoring, but more importantly acts as a safeguard against poorly scored sleep studies. Ninety all-night studies were examined. Without quality assurance, the average agreement with a human expert was 86%. However, only 28% of the studies met the desired goal of 90% agreement. Eleven of the studies were below 80% agreement, with a low of 66%. By verifying the correctness of random samples, and performing additional work on poorly scored studies, the agreement rose to 91%, with a low of 84%. 76% of the studies met the target of 80% agreement. A second approach, using clustering to help pick the samples to be verified, further increased the average agreement to 92%, with the low rising to 86%, while the number of events needed to be examined by the somnologist decreased. Additionally, 88% of the studies met or exceeded the 90% agreement. As a result of using clustering, greater and more efficient protection is provided.<>