Process Knowledge-Infused AI: Toward User-Level Explainability, Interpretability, and Safety

AI has seen wide adoption for automating tasks in several domains. However, AI's use in high-value, sensitive, or safety-critical applications such as self-management for personalized health or personalized nutrition has been challenging. These require that the AI system follows guidelines or well-defined processes set by experts, community, or standards. We characterize these as process knowledge (PK). For example, to diagnose the severity of depression, the AI system should incorporate PK that is part of the clinical decision-making process, such as the Patient Health Questionnaire (PHQ-9). Likewise, a nutritionist's knowledge and dietary guidelines are needed to create food plans for diabetic patients. Furthermore, the BlackBox nature of purely data-reliant statistical AI systems falls short in providing user-understandable explanations, such as what a clinician would need to ensure and document compliance with medical guidelines before relying on a recommendation. Using the examples of mental health and cooking recipes for diabetic patients, we show why, what, and how to incorporate PK along with domain knowledge in machine learning. We discuss methods for infusing PK and present performance evaluation metrics. Support for safety and user-level explainability of the PK-infused learning improves confidence and trust in the AI system.

Benchmarking datasets that assess the natural language understanding capabilities of large language models fall short in accelerating models to achieve user-level explainability, safety, uncertainty, and risk handling.^,a

[Online]. Available: https://tinyurl.com/KiL-MentalHealth-NLU.

These challenges are associated with the limitations of AI in restricting its learning tasks to classification and generation, which are single shots. In comparison, real-world applications demand an orchestrated response going through a multistep process of learning the high-level needs of the user, then drilling down to specific needs, and subsequently yielding a structured response having a conceptual flow. For example, triaging patients in mental health requires clinical process knowledge manifested in a clinical questionnaire. Figure 1 illustrates a scenario where the agent maps user input to a sequence of yes or no questions to compile suicide risk severity. The agent can keep track of user-provided cues and ask appropriate follow-up questions through these ordered sets of questions. Upon receiving the required information to derive appropriate severity labels, the agent’ outcome can be explained to MHPs for appropriate intervention. Similar but more complex applications include using Autism Diagnostic Observation Schedule to evaluate children with autism or using Montreal Cognitive Assessment score to measure the cognitive decline in poststroke Aphasia patients. To train conversational agents for such functionality requires specialized datasets grounded in the knowledge that enables AI systems to exploit the duality of data and knowledge for human-like decision making.^,b

[Online]. Available: https://tinyurl.com/duality-data-knowledge.

Furthermore, to develop agents that learn from such process knowledge-integrated datasets, we require interpretable and explainable learning mechanisms.c

[Online]. Available: https://tinyurl.com/petrinet-workflow.

These learning mechanisms have been characterized under the umbrella of Knowledge-infused Learning (KiL).