Leveraging Large Language Models for Subjectivity Detection on Ride-Hailing App Review

Understanding subjective content in user-generated content is crucial, as it differentiate between opinion-based statements from fact-based ones, which is an important step in information filtering, review summarization, and natural language understanding. This study investigates the capabilities of large language models (LLMs) in detecting subjectivity within ride-hailing app reviews by experimenting with different LLMs, prompting methods, and evaluation metrics. This study aims to identify the most effective model-prompt combination. Through performance metrics and statistical significance testing (McNemar's test), the findings shows that Claude consistently demonstrates the highest overall performance, particularly when paired with Chain-of-Thought prompting. This suggests that structured reasoning significantly enhances Claude's ability to identify subjective language which highlight the importance of prompt structure in optimizing LLM-based subjectivity detection systems.