Leveraging User Interaction Signals and Task State Information in Adaptively Optimizing Usefulness-Oriented Search Sessions

Current information retrieval (IR) systems still face plenty of challenges when applied in addressing complex search tasks (CSTs) that trigger multi-round search iterations. Existing relevance-oriented optimization algorithms and metrics are limited in helping users find documents that are useful for completing CSTs, rather than merely topically relevant. To address this gap, our work aimed to characterize CSTs from a process-oriented perspective and develop a state-based adaptive approach to simulating and evaluating search path recommendations. Based on the data collected from 80 journalism search sessions, we first extracted intention-based task states from participants’ annotations to characterize temporal their temporal cognitive changes in searching and validated the state labels with expert assessments. Built upon the state labels and state distribution patterns, we then developed a simulated adaptive search path recommendation approach, aiming to help users find needed useful documents quicker. The results demonstrate that 1) different types of CSTs can be differentiated based on their distinct state distribution and transition patterns; 2) After a small number of iterative training, our adaptive recommendation algorithm can consistently outperform the best possible performance from individual participants in terms of the useful-based search efficiency across all CSTs. Going beyond traditional static viewpoint of task facets and relevance-focused evaluation approach, our work characterizes CSTs with a dynamic perspective and develops a domain-specific adaptive search algorithm that can help users find useful documents quicker and learn from online search logs. Our findings can facilitate future exploration of adaptive search path adjustments for similar types of CSTs in other domains and work task scenarios. CCS CONCEPTS • Information systems $\rightarrow$ Users and interactive retrieval.