ScrapeAI: A Multi-Modal Approach to Detect Dark Patterns

This study proposes a multi-modal architecture to detect dark patterns, leveraging the usage of textual, visual, and temporal contexts. Dark patterns are deceptive elements used in websites and apps that manipulate users into making decisions that benefit the company, often at the expense of the user. In today’s world, with a significant rise in the adoption of technology across the globe, detecting dark patterns becomes important. This study is more focused on the dark patterns in the e-commerce space. We propose using three modules to detect this task, each of which processes different types of data. For textual data, we leverage Large Language Models to detect potential dark patterns. For visual data, we use object detection models to detect any visual cues that might reveal potential dark patterns. For the temporal context, we track and analyze the states of the website. Using these three modules makes our detection process robust and versatile. Furthermore, we created an image dataset DarkPattern-Capture (DPC), which has images of potentially deceptive elements; our object detection models are trained and tested on this dataset.