Understanding Physician’s Experience with Conversational Interfaces during Occupational Health Consultation

This paper presents an exploratory study on using conversational interfaces (CIs) to support physicians in conducting occupational health consultation. The CI was achieved through a web-based information dashboard with a chatbot assistant for providing real-time suggestions through text messages. Two system designs were developed: the first using a proactive chatbot, the second using an on-demand type of interaction. The effectiveness of the proposed CI and the two types of chatbot designs were investigated in a field study consisted of eight healthcare consultations. Quantitative results showed that the CI was positively evaluated as a reliable tool to be used during medical consultations and that occupational health physicians were eager to use this technology in their work. The qualitative data analysis suggested that our design concept might improve the workflow during the consultation, in particular with respect to the access to relevant information and structured decision-making processes using valuable references. The on-demand, lightweight type of chatbot interaction was better perceived than the proactive one. Based on these findings, we discuss implications for the future development of occupational health consultation based on CI and their potential contribution to computer-assisted, data-driven healthcare.


I. INTRODUCTION
The focus of human-computer interaction (HCI) research in healthcare expanded significantly from its beginning, spanning from persuasive designs for vitality promotion [1] to body interactions in motor rehabilitation [2] and medical informatics for the provision of healthcare services [3].In the healthcare field, it became increasingly prevalent to use technologies to support caregivers in everyday tasks.For instance, the adoption of electronic health records and artificial intelligence (AI) has facilitated clinical decision support tools (CDSTs).These systems have been designed and developed to assist physicians and other healthcare professionals in clinical decision-making tasks such as patient diagnosis, prognosis, and treatment options [4].
Despite technical improvements such as highly performant algorithms [5] and ubiquitous data acquisition [6], to date, very few CDST implementations have delivered on the expected promises.As suggested by many studies [7]- [11], one of the causes is the lack of attention in designing the human-computer interaction of CDSTs to be correctly deployed into the healthcare routine.In order to tackle this challenge, many research projects have focused on improving the effectiveness of CDSTs using theory-based design [12], data visualization [13], and distributed interfaces [14].During the last few years, it has been seen an increased interest in using conversational user interfaces to promote collaborative decision-making practices between clinicians and AI [11], [15], [16], emphasizing supporting medical reasoning [16], collecting patient intake [17], and facilitating clinical training [18].These practical applications of conversational agents can offer several benefits in the healthcare context, including increased transparency of machine learning models [16], workflow optimization [19], and improved collaboration [11].
In this project, we aimed at bringing the aforementioned conversational agent-based research to the underexplored scenario of occupational health (OH) consultations.As shown in Figure 1(a), OH consultation consists of a series of information-heavy clinical tasks in which the occupational physician needs to understand the patient, analyze the data, process the data, make medical decisions, and finally create a diagnostic report and a treatment plan [20].Previous research has shown that using chatbot-based interactions in intelligent decision support systems may assist the consultation workflow [11].However, there is no empirical evidence on how conversational agents should be designed to enhance doctor's work during medical consultations.This paper offers insights into this topic based on an exploratory field study.
In this paper, we investigate using conversational interfaces (CIs) to assist occupational physicians' consultation workflow (Figure 1(b)).A digital system named ConsultAI was used as a research probe [21] to facilitate the working mechanism of the proposed CIs.Specifically, the ConsultAI system contains a web-based information dashboard with a chatbot-like intelligent assistant that provides real-time suggestions for clinical decisions through text messages during the OH consultation.For this study, we developed two types of chatbot-based interactions: 1) a 'proactive chatbot' that offers proactive assistance depending on the progress of the consultation (Figure 1(c1)); 2) an on-demand chatbot that gives answers to doctors' questions in order to help clinical decision making (Figure 1(c2)).Based on our design concepts, we carried out eight OH consultations, using a Wizard-of-Oz approach [22], to investigate how the proposed CI designs could improve doctors' workflow and how different chatbot interactions could influence the doctors' subjective CI experience.
The contributions of this paper are twofold.Firstly, we leverage conversational intelligence to facilitate a new form of technology-assisted healthcare practices; Secondly, we provide empirical evidence that a CI consisting in a chatbot with on-demand and lightweight interactions can be effective in improving the consultation workflow.

A. CLINICAL DECISION SUPPORT SYSTEMS
Over recent decades, information technologies have been studied and implemented extensively as clinical decision support tools (CDSTs) to enhance medical care practices [23].CDSTs are designed to provide clinicians with intelligently filtered knowledge and patient-specific information to support the decision-making process due to abundant health data resources and high-performance algorithms [24].CDSTs can improve medical decisions by assisting mainly in diagnosing patients [25], making predictions on patients' prognosis [5], and creating treatment plans [26].
Despite an enormous body of computer science research aiming at advancing technical dimensions [6], their actual value for improving clinical workflows remains uncertain.This is pertinent to the complexity that arises from the wicked nature of decision-making tasks in healthcare [10].To enhance the effectiveness of CDSTs, Sittig et al. [7] proposed that the system implementation should not be interruptive and intrusive to current clinical scenarios.Similarly, Musen et al. [8] criticized that technical advantages are easy to fail when they need to be transformed into affordable and useful CDST applications.More recently, Yang et al. [9] argued that the design and deployment of CDSTs should match to characteristics of the clinical context, such as the workflow pattern and the collaborative nature in healthcare.Collectively, prior studies suggest a lack of considerations on HCI and a need to resketch the user-system interactions when developing these computational tools for healthcare.
In the HCI community, research on CDSTs has mainly focused on two aspects.The first is improving the adoption and trustworthiness of intelligent decision support.This type of research addresses several critical design issues using, among others, more explainable AI frameworks [12], more precise data visualizations [13], and better presentation of information [27].The second strand of research draws on the clinical appropriation of CDSTs to investigate the system to be assistive and collaborative to the clinical routine [28].Some early studies have explored HCI design strategies to fit intelligent systems into various clinical tasks.For instance, VizCom leverages distributed information systems and integrated communication interfaces to enable collaborative diagnostic works in the intensive care unit [14].Simulator is a tabletop application used to shape the collaboration in hearing aid tuning by helping the patient and the clinician build joint decisions on the diagnosis and treatment actions [29].CORE-MI is an automated feedback system that uses interactive visualization and reward models to help psychotherapists reflect on their performance in motivational interviews [30].Unremarkable AI generates information slides based on subtly embedded machine prognostics to support doctors' collective decision on heart pump implant [9].
Recent studies have also drawn attention to conversational agent-based user interfaces for promoting human-AI collaborations of decision making in medical routines [11], [15], [17], [31].In the following subsections, we survey the development of conversational interfaces and relevant applications designed specifically for healthcare settings.

B. CONVERSATIONAL INTERFACES FOR CLINICAL ROUTINES
Conversational interfaces, such as a chatbot, can facilitate the application of CDSTs with many promises, such as building trust [32], engaging users [33], increasing transparency [16], and guiding the workflow [19].As a new generation of AI systems, chatbots commonly use natural language processing to understand user inputs and adopt text messaging applications to provide feedback through the nuances of human language [34].This kind of conversational intelligence has been primarily developed to help patients receiving health care services.For example, Mandy is an agent-based mobile app that simulates a clinical interview to collect patient narratives of illness and background information [17].Quro utilizes a personalized chatbot interface to support selfdiagnosis [35].Shihbot is an intelligent conversational system embedded in social media to promote the search of sexualrelated health information [36].Moreover, a variety of studies have deployed chatbot features to facilitate effective health interventions that encourage patients to quit smoking [37], increase physical activity [38], or control weight [39].
The implementations of conversational interfaces can also be beneficial to medical practitioners.Kazi et al. [18] integrated terminologies and domain knowledge resources into an interactive tutoring system for medical students.Similarly, Tanana et al. [40] designed and evaluated a textbased conversational system called Clientbot to serve as a tool for junior therapists to enhance interviewing and counseling skills.Besides the training purposes, a growing body of research has focused on assisting clinicians with decision-making by creating collaborations between AI and humans based on conversational intelligence.An early example by McSherry [16] is a conversational case-based reasoning system that serves as a non-obtrusive CDST to fit into clinical routines seamlessly.Recently, Palanica et al. [11] conducted a survey to understand doctors' opinions about using chatbots for healthcare.Their results suggested that rather than replacing medical practitioners, conversational intelligence should be designed to be assistive to physicians as routine technologies.
Informed by these studies as mentioned above, in this paper, we investigate the process of interweaving conversational intelligence into the clinical routine to assist and collaborate with doctors in their decision-making processes.Our research was conducted in the specific medical context of occupational health consultation.

A. OCCUPATIONAL HEALTH PHYSICIANS AND OCCUPATIONAL HEALTH CONSULTATIONS
According to [41], occupational health can be described as a multidisciplinary field of healthcare that aims to prevent workplace hazards and to support safety-and health-oriented working cultures.There have been various OH services investigating the maintenance and improvement of employees' health and working capacity.For instance, digital health technologies have been increasingly applied to facilitate health surveillance [42], vitality promotion [43], and illness prevention [44].In many industrial countries, another primary OH service is the diagnosis and treating occupational diseases and work-related injuries [45].In this context, occupational physicians are playing an essential role.
Occupational physicians are doctors specialized in conducting the multifactorial assessment of work and health risks and in developing rehabilitation strategies and reintegration-to-work plans [45].Their daily tasks involve creating the diagnosis, prognosis, and treatment plan for sick workers, based on the triage intake and OH consultation [41].In this case, occupational physicians focus on matching individuals to healthcare services and helping them in a sustainable return to work [41].In the Netherlands, for example, OH physicians have a 30-minutes OH consultation with the patient within six weeks after receiving the absence and sickness report [20].
The OH consultation is an information-heavy clinical routine work [46].As shown in Figure 1(a), the physician would interview the patient, take notes, check historical data, find relevant information (e.g., knowledge and protocols), discuss with the patient, and write a consultation report, which contains the diagnosed issue, the prognosis, and treatment suggestions [20].

B. CONVERSATIONAL INTERFACE-ASSISTED OH CONSULTATION
Over recent years, electronic health records and few CDSTs [47], [48] have been adopted in OH consultations [49].
Previous studies [50]- [52] have indicated the potential of technology as a helpful addition to improving medical consultations.Our research investigates the evolution of CDSTs based on conversational intelligence for OH consultations.We hypothesize that conversational agentbased interactions could help doctors in using decision support systems during consultation meetings as unobtrusive virtual assistants.To this end, we conceptualize the CI (conversational interface)-Assisted Consultation as a new form of the clinical workflow (see Figure 1(b)), facilitated by a web-based dashboard with a chatbot assistant (namely ConsultAI).In a CI-assisted session, ConsultAI can unobtrusively record the conversation, extract valuable information, provide suggestions, and present them through the chatbot interface as text messages to help the occupational physician in taking diagnostic decisions and creating treatment plans.Our concept and prototype development involved a set of design activities with multi-stakeholders, including endusers (i.e., occupational physicians), system developers (i.e., data scientists, software developers, designers), and research and management teams (i.e., product managers, research consultants).The co-design process has been reported in [53].
Figure 2 shows the detail of the ConsultAI interface, divided into two major parts.On the left it presents a summary of the patient, including the personal profile, self-reported complaints, work absence history, and employment information.On the right it integrates a chatbot-like conversational assistant that can interact with the physician to provide decision support during the consultation.According to the progress of the consult, the conversational assistant can present relevant information as instant messages, including AI-based prognosis and diagnosis, medical domain knowledge, OH guidelines, etc. Regarding the conversational mechanism, we learned that a healthcare consultation meeting requires the doctor to involve enormous mental efforts in communicating with the patient [50].As a concurrent practice, the system interaction of the chatbot may need to be designed in order to be easy to use.Therefore, inspired by [2], [54], we developed the following two types of chatbot interactions to investigate how different conversational mechanisms could influence the effectiveness of the decision support.
• Chatbot with proactive interaction: Figure 1(c.1) shows that in the proactive mode the chatbot would generate decision-support information based on the conversation between the doctor and the patient.Then such information would be converted into text messages and sent to the doctor automatically throughout the consultation.
• Chatbot with the on-demand interaction: Figure 1(c.2) shows that in the on-demand mode the chatbot would be able to receive questions entered by the doctor.Based on analyses of the received questions, the system would send messages to the doctor with related information and suggestions useful for decision making.
Although out of the scope of this study, we briefly describe a possible implementation of ConsultAI (Figure 3), including four major components.The first should leverage automatic speech recognition to acquire data based on the doctor-patient conversation unobtrusively.Speech recognition technologies have been widely investigated and used in a variety of healthcare practices, such as access to health information [55], clinical documentation [56], health data collection [57].The second component, similar to [57] and [58], should utilize natural language processing methods (i.e., text summarization algorithms) to extract the clinical facts and some essential information from the doctor-patient conversation.The third part of the proposed system should query related information from OH databases and develops confidence-based recommendations using the multimodal data fusion approach [60], which has been validated extensively in enabling datadriven healthcare decisions [61].The last part of the system should arrange the retrieved data into text messages and present them in real-time to the doctor through the chatbot assistant.In this case, the decision-support information can be ranged from medical knowledge and OH protocols to the AIbased diagnosis, prognosis, and treatment plan.

IV. USER STUDY
In this section, we describe the user study and the evaluation of the proposed CI-assisted consultation strategy, including the research questions, the study design and setup, the recruitment of participants, study procedure, data collection and analysis.

A. RESEARCH QUESTION
We conducted a field study consisting of a 30-minutes occupational health consultation.ConsultAI was applied as a research probe to facilitate the realization of a CI-assisted consult.The technical fidelity was addressed using the Wizard-of-Oz approach [22].This type of user evaluation method has been long recognized as a practical approach to verify a new design concept of intelligent user interfaces and conversational agent-based applications [62].Based on this user study, we aim to answer the first research question: • RQ1: Whether and how does the ConsultAI system support doctor's workflow during a conversational interface-assisted consult?
To explore the effects of the chatbot interactions (Figure 1(c)), we asked the recruited OH physicians to use two different types of CIs.The comparison between their subjective experiences in using the proactive and the on-demand interaction modes was made to investigate our second research question: • RQ2: Whether and how do interaction styles of the ConsultAI chatbot influence doctors' experiences during a conversational interface-assisted consult?
An additional aim of this study was to gain in-depth insights into design challenges for the improvements of CIs to be integrated into OH services as a clinical routine.

B. STUDY DESIGN
We conducted the user study in collaboration with a Dutch OH service provider.Since legal regulations do not allow to experiment prototypes during real OH consultations, following the methodology in [9], one actor was asked to impersonate a sick employee and participate in a meeting with the occupational physicians.During the consultation, we randomly assigned the recruited doctor to use one version of the CI.The other version was then introduced in the follow-up interview.Therefore, we collected their feedback on the two chatbot interaction mechanisms (proactive vs. on-demand) for a qualitative evaluation [63].
We developed the medical case for the study based on the following steps.Firstly, we involved one researcher in playing the role of a sick employee (i.e., 'patient') during the consultation.We built the patient profile based on the researcher's physiological characteristics and past working experiences.Secondly, a senior occupational physician helped to create the medical case after selecting the "somatic symptom disorder" as the intended disease.Somatic symptom disorders are difficult to diagnose due to the physical symptoms that trickily connect to emotional distress [64].As such, we assumed that doctors would benefit from the assistance of a decision support tool.Thirdly, in order to create the medical simulation case, the doctor collected information from several different real cases without disclosing any identifiable personal data and created the medical content of the chatbot taking into account a routine medical examination consisting of 1) getting acquaintance with the patient, 2) diagnosing the potential diseases, 3) making a prognosis, and 4) creating an action plan for possible treatments and return to work.Lastly, we consolidated the medical simulation case, developed conversation scripts for the study, and finalized all the details with the doctor.

C. SETUP
The experiment was carried out in two separate rooms.In the first room, a real OH consultation room (Figure 4(a)), the occupational physician conducted the medical consultation with the 'patient', assisted by a computer with the ConsultAI system.In a second room (see Figure 4(b)), two researchers (an OH physician and a design researcher) monitored the progress of the consult through a voice-based Skype call that was unnoticeable to the participant.In return, they sent messages to the ConsultAI system as a chatbot assistant, facilitated by a real-time messaging API.

D. PARTICIPANTS
Eight occupational physicians participated in our study.They were unaware of the study's goal, with different age, gender, and working experience.Moreover, participants had a different experience in using CDSTs.This helped us in gathering a wide range of user experiences.Participants' characteristics are summarized in Table 1.These participants are referred to as P1 to P8.In the study, P1 to P4 experienced the CI based on the proactive chatbot, whereas P5 to P8 used the on-demand chatbot.The limited number of participants is justified by the severe shortage of occupational health physicians [45].In line with [9], [29], [65], we collected qualitative insights regarding our proposed design solutions and research questions based on the field study.

E. PROCEDURE, DATA COLLECTION AND ANALYSIS
The experiment was initiated by an introductory session with the doctor, in which the ConsultAI system embedded with the selected type of chatbot and the study procedure were explained, without disclosing the research questions.
Afterward, the medical consultation meeting took place.At the end of it, the doctor who participated in the study was asked to fill out a post-questionnaire.To conclude the experiment, a face-to-face semi-structured interview was conducted.
The post-questionnaire was used to evaluate user experiences during the CI-assisted consultation.Similar to [66], we examined user experiences with conversational intelligence concerning three aspects: trust, user satisfaction, and intention to use.The questionnaire was designed with three subscales using items adapted from [33] to measure trust, [67] to measure user satisfaction, and [68] to measure intention to use.All the subscales were 7-point Likert rating scales (from 1 being a negative experience to 7 being a positive experience).The questionnaire responses were analyzed using SPSS software to calculate median (MDN) and interquartile range (IQR), as well as to conduct non-parametric comparisons.To examine the overall user experiences, we applied the one-sample Wilcoxon signed rank test to compare the questionnaire results of each subscale against the median value (4) of each scale.The Mann-Whitney U tests were performed to understand the impact of the chatbot interaction styles on the user experience with CIs.Here, we compared the questionnaire scores between the participants with the 'proactive' chatbot (P1-P4) and with the 'on-demand chatbot (P5-P8), across three subscales.
The interview took approximately 45 minutes per participant.We followed a scripted protocol and included open-ended questions about the user experiences, potential benefits, and perceived challenges of using ConsultAI.We then presented the interaction mechanism of the ConsultAI chatbot that the participant had not used.Next, we asked the participant to compare the two chatbot modes and elaborate on their reasons.During the interview, we also asked the participants to explain some interesting statements that emerged during the interview.All the interviews were audiorecorded, reviewed, and summarized into transcripts for the thematic analysis [69].We transformed the segmentation of the interview transcripts into quote statements and labeled them.We then measured the labeled statements using inductive coding to identify recurring clusters with emergent themes [70].

A. HOW CONSULTAI SUPPORTED OH CONSULTATION
The first aim of our study was to assess whether and how ConsultAI supported OH physicians during the consultation meeting.Figure 5 shows that our participants experienced our design positively, with reasonably high scores on the subscales of trust (MDN = 5.36, IQR = 1.32), user satisfaction (MDN = 4.50, IQR = 2.75), and intention to use (MDN = 5.34, IQR = 2.50).Furthermore, one-sample Wilcoxon signed rank test indicated that the questionnaire results of this study were significantly higher than the neutral score when it came to the subscales of trust (p = 0.012) and intention to use (p = 0.035).While participants scored their user satisfaction higher than the neutral, such difference was not significant (p = 0.288).To summarize, the quantitative findings suggest that the ConsultAI system was deemed as a reliable tool to support the OH consult.Occupational physicians were motivated to use this system in the future.In addition to the questionnaire responses, the interview data revealed numerous instances in which ConsultAI helped the doctors during consultation meetings.Next, we report these examples in three clusters: (1) easing the way of accessing information, (2) guiding through the consultation process, and (3) offering references to medical decisions.

1) EASING THE WAY OF ACCESSING INFORMATION
As reported by our participants, ConsultAI reduced their workload by helping them in finding information quickly.Firstly, the OH physicians liked that they could get an overview of the patient by looking at the dashboard.As P7 stated, "I can see the job information at a glance, such as working hours, company size, work experience, and so on…" Our participants considered this feature helpful for efficiently initiating the conversation with the patient.
Secondly, we observed that the majority of the doctors used the ConsultAI for finding domain knowledge and necessary information (e.g., guidelines, questionnaires) related to the medical case in the examination.For instance, P6 asked for the link to a medical assessment questionnaire; P4 asked for diagnostic information about the physical symptoms.From the interviews, we learned that participants are used to consulting search engines (e.g., Google) and online services during the consultation to find info for the patient.Most participants preferred getting information based on ConsultAI.E.g, "I ask something to the chatbot, and it gives me feedback based on the right source, this was more intuitive and reliable than the internet" (P7), "It accelerated my work" (P5), "I think it was easier to get the right info without searching for them" (P1).

2) GUIDING THROUGH THE CONSULTATION PROCESS
In the user study, we sent several messages based on a standard OH consultation guideline.These messages involved suggestions such as checking the identification of the patient, collecting information on the patient's general practitioner, and contacting the employer of the patient.From our interviews, we learned that the majority of the participants perceived the abovementioned guidance positively: "This [system] helps to follow the steps and keeps me sharp" (P4).They further indicated as useful to be reminded by these suggestions: "I like the advice such as contacting the employer after the holidays … you might forget something like that" (P8), and "The system reminds me to check the patient's ID, which I always forget.I would ask that more in the future" (P6).Furthermore, few participants also liked the 'mild-tone' of the chatbot in providing guidance.As a result, they did not feel obliged to follow: "If it had controlled my mind during my work, I would have felt forced … Luckily, I didn't feel that because the bot just told me options as my assistant does" (P5).

3) OFFERING REFERENCES TO MEDICAL DECISIONS
The data from the interviews showed that ConsultAI was useful in helping doctors with their decision-making process.As they stated, this conversational intelligence support system was experienced similar to their traditional peer support: "It is a bit like when a co-assistant or senior doctor is participating in the consult" (P3).We found that participants referred to ConsultAI regarding the help provided during the decisionmaking task in two ways.First, when their decision-in-mind was similar to ConsultAI, it supported them in confirming their hypothesis.As P4 described: "I received the suggestions that, at the same time, I had in mind.So, the system confirmed my ideas".Second, when the suggestions received were different, our participants also found it useful in broadening their thought: "The suggestion received about the multidisciplinary exam was good.It was helpful to receive in such a case a suggestion for possible treatments to advise.It helps me think about other possibilities and then make a decision, even though I still kept my own decision in the end." Additionally, some participants believed that alternative suggestions increased their awareness of making decisions carefully when there was an in-doubt situation: "As a senior occupational health doctor, I can't avoid having one line of thoughts to look at the problem … sometimes this is dangerous, but your system reminded me" (P2).

B. HOW SYSTEM INTERACTIONS INFLUENCED OH CONSULTATION
In this subsection, we present the analysis for our second research question about how different system interaction styles influence the subjective experiences with the CIassisted consultation.As shown in Table 2, participants evaluated their user experiences better with the on-demand chatbot than with the proactive chatbot, in terms of the trust (5.36 vs. 5.29), user satisfaction (5.00 vs. 4.00) and intention to use (6.33 vs. 4.67).According to the Mann-Whitney U test, the perceived differences between these two interaction styles were not statistically significant.Although no statistically significant differences were found in the quantitative data, the interview results indicated different user experiences in relation to the chatbot interaction styles.After knowing all the two chatbot modes of operation, all the participants believed that the chatbot with on-demand interactions would be more valuable and assistive than the proactive one.Next, we report qualitative findings on how the chatbot's different interaction styles influenced the doctor's experiences with the CI-assisted consultations.

1) BEING A TASK ASSISTANT WITH ON-DEMAND FEEDBACK MAINLY
There were different levels of system interactivity involved in ConsultAI.In the on-demand interaction mode, the chatbot provided feedback mostly based on questions from the doctor.
In the proactive interaction mode, the chatbot provided suggestions based on information automatically acquired during the consult.Our interviews revealed a high acceptance among the participants towards having a 'passive assistant' for OH consultations.As the most senior doctor (P7) explained: "Doctors always work on their own.It also builds up our personality with strong opinions.So, when I ask a question, the chance of following that question is bigger than receiving automatic advice."As such, our participants considered the system as: 'an intelligent assistant'(P5), 'a collaborator' (P6), 'the smart Wikipedia' (P8).Given the fact that our participants lead the conversation with the patients, they became engaged in thinking with the on-demand system.For instance, "I believe that communication triggers me to think about what I can ask more and check if I miss something.So, I built my decisions step by step" (P7), and "It helps you to think deeper with the information because you can influence the system by asking questions" (P8).In contrast, several participants felt negative by using the proactive system.One example was from the junior doctor, P3, who felt like being watched by the chatbot: "I thought it was a sort of big brother watching me and wanted to correct me.I did not follow anything, so, chatbot, don't bother me!"

2) LIGHTWEIGHT INFORMATION FLOW TO AVOID DISTRACTING THE MEETING
We learned another issue from interviews: some doctors felt overloaded by the automatic ConsultAI system for two reasons.First, they found that it might potentially decrease their work efficiency due to distractions from the proactive yet unexpected information displayed.As P3 described: "It gives me some hints, but I don't know if they are useful or only distracting.I can be swamped, and suggestions without asking can be an unexpected challenge".Some participants also had the same concerns about long-term usage of the proactive chatbot: "If frequent interruptions occur, workers will be more tired at the end and less efficient" (P3), and "I would be annoyed if the suggestions keep popping up" (P5).
In addition to the fear of being distracted, some participants also thought that the proactive chatbot could reduce their communication with the patient, due to the overloaded information flow: "If I check the information frequently, I may lose contact with the patient" (P3).Similarly, P5 stated: "It is nice to have extra info, but it interferes with the interaction with the patient!"In contrast, using the on-demand chatbot, we observed that the participant (P7) also tried to incorporate ConsultAI into the clinical conversation.We found P7 formulating questions, reading the feedback aloud, and analyzing the suggestion with the patient.P7 explained his behaviors as following: "I think the depth of my work is communication.I saw a chance to use interactions with the [on-demand] chatbot to activate my patient and get the trust from her towards finding the solution together!".In line with some earlier studies [50]- [52], this finding suggests that conversational intelligence may potentially be leveraged to enhance the doctor-patient communication, based on ondemand and lightweight interactive CIs for physicians.

C. HOW TO INTEGRATE CI MECHANISMS INTO OCCUPATIONAL HEALTH
In this study, we deployed the ConsultAI as a research probe to elicit doctors' insights into design opportunities to integrate CIs into the clinical routine.According to our in-depth discussions, we identified two main design opportunities for the development of CIs in the OH context.

1) CI AS AN INTEGRAL PART OF OH SERVICES
We mainly employed ConsultAI as an intelligent assistant for the scenario of the OH consult.Our participants believed that not only consultation would benefit from such application but also some repeated coordination tasks, such as standardizing diagnostic reports (P1, P3, P7) and taking over administrative work (P2, 3, 5).It would involve some efforts to advance data collection and management.As P7 described, "If the chatbot can read things I wrote down.It may help me with creating the report based on guidelines, but it will take some integration."Some doctors were enthusiastic to see how ConsultAI could further improve their work efficiency by exposing it to the patients.For example, P2 illustrated a new workflow based on a ubiquitous CI, "I would like to ask the system to send a questionnaire to the patient then put [the patient] in the waiting room … the intake [from the patient] would be presented on my computer later.I think that can at least save half … but maybe one hour a day."Regarding the interface design, we were suggested to integrate chatbot-based features into the current systems.As they stated, the ConsultAI chatbot could be embodied as an additional layer of the OH software user interface, such as a pop-up balloon (P2) or a chat-box (P8).

2) EMPOWER CI-ASSISTED CONSULTS WITH ROBUST DATA INFRASTRUCTURE
Due to the Wizard-of-Oz approach, some participants saw a lack of smartness and inclusiveness in ConsultAI.They pointed out that the quality of data would be decisive in supporting the everyday use of such systems.Participants further encouraged us to expand the coverage of the results with more advanced searching algorithms.As P4 mentioned: "I know you can use 'machine learning' to search articles and new guidelines.The system can be a tool to tell me what is in the protocol and scientific research to help me use new information fast."Moreover, several participants showed their interest in getting data from multiple resources beyond OH services, such as the health data from commercial activity trackers.This would require more work on improving the data infrastructure.However, we believe this also needs to be further initiated, supported, and realized by more in-depth discussions, reflections, and revisions of existing OH regulations and working protocols.

VI. DISCUSSION
Clinical decision support tools (CDSTs) have been studied extensively in assisting a variety of medical practices, such as intensive care [14], clinical implantation [9], mental therapy [30].The current healthcare consultation can be enhanced by adopting intelligent technologies [50].It has been proven that conversational user interfaces can support the application of CDSTs, in terms of increased transparency [16], and optimized workflow [19], improved collaboration [11].In this paper, we have reported an exploratory field study investigating a conversational interface (CI)-assisted consultation, which was designed to use a chatbot assistant to provide real-time decision support to occupational physicians during consultations.Our quantitative and qualitative data analyses suggested that the application of CIs to doctors during the OH consultation could improve their efficiency, primarily through improved information accessibility, guided workflow, and valuable references to decision-making.Moreover, we found that OH physicians preferred to ask the chatbot assistant for suggestions rather than receiving proactive recommendations.Based on our research findings, we now discuss design implications to better leverage conversational user interfaces in the occupational health context for supporting clinical decision-making.

A. FIT CONVERSATIONAL AGENTS INTO OH TASKS AS AN UNOBTRUSIVE COLLABORATOR
Our results revealed that a chatbot with a passive type of interaction could be more efficient in assisting occupational health (OH) decision-making tasks.Some recent studies suggested that CDSTs should be designed to be unobtrusive [9], assistive [11], and collaborative [28].Similarly, our CI mechanism with a passive assistant only provided on-demand feedback and lightweight information to assist with the consult without overburdening the concurrent tasks.Therefore, most of our participants considered it as an unobtrusive collaborator.For instance, P8 confirmed the treatment plan about a multifactorial test with the chatbot; P6 used ConsultAI as a virtual assistant to get information during the consultation in a more accessible way.By contrast, with the proactive chatbot assistant, our participants could not fully decide when and what information to receive.Hence, they felt overloaded and distracted.
Our interview responses suggest that conversational agent can be designed to be integrated and adaptive to doctors' needs in their clinical work routine.For example, the ConsultAI system may be converted into a plugin feature for existing medical software with the option of being activated or deactivated depending on the type of work or task that the doctor is performing.

B. ENRICH CONVERSATIONAL INTERACTION MECHANISMS TO FACILITATE DATA-DRIVEN OH SERVICES
In this paper, we have shown the promising results of using conversational agent-based interactions to facilitate the integration of CDSTs into OH consultations.Our study revealed that the conversational mechanism could be adopted as a facilitator of the interactions between doctors and relevant data under clinical practices.Thus, we were encouraged to incorporate such chatbot-assisted communications further into OH services.
Healthcare practitioners have to work closely with different kinds of data.It has been long advocated for promoting datadriven healthcare services [71].From our study, chatbot-based interactions have shown advantages to help with merging the presentation of data into a conversational flow.As such, the overwhelmed volume of information would be decomposed and may be easy for users to receive and understand during their clinical tasks.Based on this study, we suggest that chatbot-based interactions should be designed as a practical approach to facilitate easy-to-use data-driven healthcare services.

C. LIMITATIONS AND FUTURE WORK
We may need to interpret findings from this study cautiously due to a few limitations.First of all, the study was conducted using simulated medical cases, with limited participants, and based on the one-round of experiments.Therefore, our results might not have been adequate to reveal the effects of CIs in long-term everyday use in real occupational health consults.As an exploratory study, we used the ConsultAI prototype as a research tool to investigate the chatbot's interaction styles for assisting doctors in using intelligent decision support.This experiment was facilitated through the Wizard-of-Oz approach.Therefore, the technical aspects of our design were relatively simple.We primarily investigated doctors' experiences with CIs during OH consultation in this study.How this novel system would influence the patient's behaviors and experiences is still unknown.
Although an intelligent medical decision support system has not been implemented for the current study as in [72] and [73], this work has focused on investigating and selecting an efficient modality of conveying information to physicians by comparing different ways of chatbot-based interactions.In the future, we will focus on implementing a fully integrated and functional ConsultAI system based on our design implications.The system will be used as a new CI-based healthcare application for investigating its long-term impacts on doctors and patients.

VII. CONCLUSION
This paper has presented an exploratory study using conversational agent-based interactions to facilitate intelligent decision support during healthcare consultation.The proposed conversational approach was implemented via using an interactive chatbot assistant, called ConsultAI, that intended to provide real-time assistance to the occupational health physician.We set out a field study with eight occupational health consultations, to investigate 1) the feasibility of ConsultAI in the context of occupational health; 2) the effects of the ConsultAI chatbot interaction styles in influencing the user experience.The quantitative results show that the conversational interface of ConsultAI was perceived positively by physicians in terms of high information credibility and technology adoption.The qualitative findings suggest that the chatbot feature could help with accessing related data, guide the consultation workflow constructively, and provide useful references to decision-making.We also found that the chatbot with the on-demand interaction style was experienced more favorable than with the proactive interaction.We suggest that future conversational interfaceassisted healthcare applications could be designed as an unobtrusive collaborator fitted into existing tasks, as well as a facilitator to promote data-driven occupational health services.

FIGURE 3 .
FIGURE 3. The proposed system design of ConsultAI.

FIGURE 5 .
FIGURE 5. Boxplots of the questionnaire results for the subscales of trust, user satisfaction, and intention to use.The neutral score is indicated in redline.

TABLE II COMPARISONS
OF THE QUESTIONNAIRE FEEDBACK BETWEEN CHATBOT INTERACTION STYLES OF CONSULTAI.