Sketch Face Expression Recognition under Simulated Prosthetic Vision

For patients with severe retinitis pigmentosa (RP) or age-related macular degeneration (AMD), retinal prostheses could be the most effective means of visual rehabilitation at present. Facial perception is an ability that potential subjects of prostheses are eager to regain, and expression recognition is one of the important components of facial perception. However, limited by the number of available retinal prosthesis stimulation electrodes currently, only low resolution perception can be generated. Thus, it becomes an important task to investigating and applying different image processing strategies to optimize the perceptual effects presented to prosthesis implant recipients. To achieve accurate recognition of expressions, a lightweight image style conversion method was proposed to realize the conversion of original images to sketch images, and the expression recognition effect after pixelating the sketch images and original images at three resolutions: 24×24, 32×32 and 48×48 was investigated. The results of psychophysical experiments showed that the sketch images at low resolution, i.e., 24×24, were more conducive to expression recognition compared to directly pixelating the original image. Furthermore, the expression recognition effect of original image and sketch image under normal human vision were also studied, the results showed an accuracy of 70.41 % and 59.18%, respectively and could provide a reference for subsequent related research.