Grading of rheumatoid arthritis on ultrasound images with deep convolutional neural network

Greyscale ultrasound (US)imaging method is usually used to assess synovitis in rheumatoid arthritis (RA)in clinical practice. This four-grade scoring system depends highly on the sonographer's experience and has relatively lower validity compared with quantitative indexes. In this paper, we proposed an automatic RA grading method using deep convolutional neural networks (DCNN)to assist clinical assessment. Gray-scale ultrasound images of finger joints are taken as inputs while the output is the corresponding RA grading results. In order to make up for the lack of large annotated training dataset, we performed data augmentation to increase the number of training samples. Motivated by the approach of transfer learning, we pre-trained the GoogLeNet on ImageNet as a feature extractor and then fine-tuned it on our own dataset. The final classification accuracy on the testing samples exceeded 95% which demonstrates that our proposed method achieves performance comparable to RA experts in multi-class classification. In future, we hope our proposed DCNN-based RA grading method can have the abilities to provide an objective and accurate reference for clinicians to assist RA diagnosis and treatment.