A Hand-Gesture Recognition Based Interpretable Type-2 Fuzzy Rule-based System for Extended Reality

In recent years, technologies such as Augmented Reality (AR) and Virtual Reality (VR) have become more popular and available to broader audiences, leading to the research and development of a myriad of extended applications. A modality for end-users to interact with these applications is through hand gestures, hence the importance of detecting the different gestures in real time. This paper presents an interval type-2 Fuzzy Rule-based System (FRBS) optimised by the Big Bang-Big Crunch (BB-BC) algorithm that uses the fingers’ position from the hand-tracking technology in extended reality (XR) headsets (namely HoloLens 2 and Oculus Quest 2) to classify the user’s hand gestures. This approach achieved an accuracy of 96.4%, and it is an interpretable model that can be understood and adjusted by end-users. The interval type-2 FRBS was tested against a type-1 FRBS and a k-nearest neighbours (KNN) model. It outperformed the type-1 FRBS and was close to the 98.9% accuracy performance of the KNN model, making our suggested approach a competitive alternative to opaque models.