Irregular Artificial Vision Optimization Strategies Based on Transformer Saliency Detection

To improve the performance of object recognition under artificial prosthetic vision, this study proposes a two-stage method. The first stage is to extract the saliency and edge Mask of the object (SMP, EMP). Then, the irregular visual information of the object is processed using Irregularity Correction (IC). We design eye-hand coordination tasks and simulate artificial vision with retinal prostheses to validate strategy effectiveness, and select direct pixelation (DP) as a control group. Each subject retained a phosphene map in the same stochastic pattern in all his/her trails. The real-time experimental results showed that the deep saliency-based optimization strategies improved the performance of the subjects when completing tasks, in terms of head movement, recognition accuracy, and response time, and counts for successful small-objects recognition. The subjects have the smallest-scale average head movement (76.53 deg ± 20.75 deg), higher average objects recognition accuracy (91.18% ± 2.52%), and less time for finishing the task (35.71 s ± 8.66 s) and better successful search times of the small target objects (1.35 ± 0.33) under the SMP strategy. When integrating with IC, subjects’ average performances have further improved to 63.39 ± 15.38 deg, 94.22% ± 3.94%, 25.76 s ± 6.24 s and 1.05 ± 0.30 respectively, which also significantly outperformed the DP condition. These results indicated that when utilizing the deep-learning-based saliency detection and IC processing, subjects could shorten the searching process and were able to discern the target objects more reliably. This work could be informative to future prosthetic devices considering implementation with the technique of artificial intelligence.