Capturing Preferences of Non-verbal Autistic Children While Watching Cartoons on YouTube

Non-verbal autistic children (NVAC) often learn their first language by watching videos on digital devices. One of the peculiarities of autistic language is the phenomenon of “unexpected bilingualism”, where the first words are spoken in a language other than the language of their parents. How they learn language by watching and which content appeals to them is unclear, but autistic children focus on cartoons, especially those with letters, numbers, shapes, animals, and vehicles, which form most of their first words they produce. This paper presents an ongoing study that examines NVAC interactions with the screen while watching YouTube on a tablet and correlates these indirect feedback signals with video content. We outline possible NVAC interaction events— touches, pauses, or skips—and suggest how they can be interpreted as indicators of engagement or disengagement. By matching these children’s interaction events to each cartoon’s visual and textual components, our approach can detect coarse-grained (overall video preference) and fine-grained (specific moments or objects of interest) NVAC preferences. Future work will test these assumptions in longitudinal NVAC studies and create a personalized system that supports NVAC language acquisition and helps psychologists understand how they learn language through watching cartoons.