Using Natural Language Processing to Accelerate Deep Analysis of Open-Ended Survey Data

In this poster session, we share our work exploring the use of natural language processing (NLP) to assist in the thematic analysis of qualitative data. With the affordances currently provided by artificial intelligence and machine learning to process textual data for meaning, our interdisciplinary team of computer and social science researchers is using a data set of open-ended text responses to test procedures for creating thematic categories to organize participant responses by content. By comparing parallel analyses of our text corpus using traditional and NLP thematic-analysis techniques, we identify key areas of congruity and incongruity between human analysts and NLP algorithms. While the areas of congruity point to the potential usefulness of NLP systems in qualitative research, the areas of incongruity suggest the need for a “human in the loop” NLP platform which iteratively refines the content analysis outputs based on feedback from an expert researcher until a satisfactorily coherent thematic description is achieved.