Understanding Sarcasm Detection Through Mechanistic Interpretability

Detecting sarcasm remains a significant challenge in natural language processing due to the nuanced interplay of context, sentiment, and contradiction. This study investigates sarcasm detection using the BERT-Large model, emphasizing mechanistic interpretability to enhance performance and transparency. A novel interpretability framework is applied, combining SHAP explanations, attention visualizations, and ablation studies, to identify critical components within the BERT architecture. The research reveals that deeper layers and specific attention heads in BERT are instrumental in isolating sarcasm-specific patterns. Through fine-tuning on a sarcasm-labeled Twitter dataset and leveraging clustering techniques, the study demonstrates the hierarchical progression of sarcasm detection across BERT layers. This work contributes to building reliable and interpretable NLP systems for sentiment analysis, with implications for improving sarcasm detection in real-world applications.