Evaluating ChatGPT's Capabilities in Sentence-Level Text Font Selection: A Comparative Study

Selecting appropriate fonts for sentence-level text, such as slide titles and poster headings, requires pleasing glyphs and semantic congruence with the text. This task is especially challenging for users without design backgrounds. Previous studies established associations between tags and fonts but overlooked the different design intents between sentence-level text and word-level text influencing font selections. Recently, large language models (LLMs) like ChatGPT with extensive knowledge bases have emerged as promising solutions for various tasks. This study evaluated the capabilities of GPT-3.5 in recommending fonts compared to professional designers to explore the application of LLMs in the design field. We conducted comprehensive evaluations, including both quantitative metrics to assess font diversity, accuracy, and consistency, and a user study involving designers and non-designers. The results indicate that while GPT-3.5 performs well in selecting common serif and sans-serif fonts suitable for general use that align with the preferences of users without design backgrounds, it falls short in stylized font selection. Our findings highlight the potential and limitations of using GPT-3.5 for font selection in collaborative design environments and provide insights into integrating such models into design tools to support multidisciplinary teams.