Classification of Digital Chess Pieces and Board Position using SIFT

Assistive technology has been given more attention in recent years to help people with disabilities to perform common tasks. Rather than designing a specialised tool for the task, it is more cost-effective and less inhibitory to make use of existing hardware integrated with a smart interface. Towards this end goal, we present our work on assisting a visually impaired person playing an online chess game. We evaluated an invariant feature descriptor, i.e., SIFT, for the task of classifying individual chess pieces across multiple visual themes. We compared two strategies for building the visual codebook, i.e., k-means clustering vs. image blending. The proposed pipeline receives live screen feeds from the browser at a fixed interval and produces an output in the form of chess pieces’ label and board position. Our proposed pipeline, paired with a visual codebook built using k-means clustering, managed an average accuracy rate of 6/10.