Product Barcode and Expiry Date Detection for the Visually Impaired Using a Smartphone

The use of an expired product may cause harm to its designated target. If the product is for human consumption, e.g. medicine, the result can be fatal. While most people can check the expiration date easily before using the product, it is very difficult for a visually impaired or a totally blind person to do so independently. This paper therefore proposes a solution that helps the visually impaired to identify a product and subsequently `read' the expiration date on a product using a handheld Smartphone. While there are a few commercial barcode decoder and text recognition applications for the mobile phone, they require the user to point the phone to the correct location - which is extremely hard for the visually impaired. We thus focus our research on helping the blind user to locate the barcode and the expiration date on a product package. After that, existing barcode decoding and OCR algorithms can be utilized to obtain the required information. A field trial with several bind- folded/totally-blind participants is conducted and shows that the proposed solution is effective in guiding a visually impaired user towards the barcode and expiry information, although some issues remain with the reliability of the off-the-shelf decoding algorithms on low-resolution videos.