“SPEAK-OUT”: A Reinforcement Learning Based Social Media Application for Visually Impaired People

Mobile applications are widely used around the world for social media and communication platforms. Despite many efforts that have been made in these applications to suit everyone, blind and deaf people are still not able to fully use these platforms. Facebook, Instagram, and X (i.e., formerly Twitter), have added new features such as verbally described images, but these are still not enough to fully describe the content. This research proposes a novel social media mobile application, “Speak-Out”, which is particularly designed for blind and deaf people. The application helps them enjoy most of the existing features and stay updated with the latest trends on other social media platforms. It offers free accounts to provide the user with full control through voice over and verbal commands, which helps them get the best possible experience. A Reinforcement Learning (RL) based recommendation system was also developed to suggest personalized newsfeeds to the user according to their historical interests. By combining the ε -greedy policy with SARSA prediction method, the best of both was obtained by giving the user better chances to explore new trends that match their interests. Users can like, share, comment to others’ posts, and can also create their own posts on “Speak-Out”.