Enhancing Facial Expression Recognition With AI Agents: A Semisupervised Guided Adaptive $\beta$-VAE Coupled With Interval Type-2 Fuzzy Classifier

Facial expression recognition (FER) is a complex task, hindered by subtle distinctions between expression classes, significant variability within each class, and external influences such as identity, pose, age, and ethnicity. As a result, achieving pure expression encodings that are resilient to exogenous factors proves elusive, thereby compromising the downstream classification tasks. This study presents a novel intelligent FER scheme that mitigates the impact of external confounders by integrating disentangled representation learning with fuzzy logic. Building on Adaptive $\beta$-variational autoencoder (VAE) [1] as a backbone, we develop a semisupervised guided adaptive $\beta$ variational autoencoder (GA-$\beta$-VAE) capable of isolating expression features from exogenous factors. Specifically, the adaptive $\beta$-VAE is augmented with two additional branches: a deformable PCA-based secondary decoder that disentangles expression-irrelevant transformations from the core expression content, and an adversarial excitation–inhibition branch that forces the “target” (expression) latent variables to be informative only of expressions. This yields well separated, expression-centric embeddings that are subsequently processed by an interval type-2 (IT2) fuzzy classification unit to predict the corresponding expression classes. By avoiding reliance on paired data or explicit annotations, this approach offers a scalable and flexible solution for FER. Experimental evaluations on benchmark datasets [extended Cohn–Kanade (CK+), facial expression recognition plus (FER+), and real-world affective faces database (RAF-DB)] demonstrate the framework’s effectiveness in addressing the challenges posed by exogenous factors, achieving superior accuracy and interpretability compared to state-of-the-art methods.