Multimodal Teacher Dashboards: Challenges and Opportunities of Enhancing Teacher Insights Through a Case Study

Teacher dashboards provide insights on students' progress through visualizations and scores derived from data generated during teaching and learning activities (e.g., response times and task correctness) to improve teaching. Despite the potential usefulness of enhancing teacher dashboards, and the respective teaching practices, with rich information regarding students' cognitive and affective states (e.g., cognitive load), few studies on teacher dashboards have considered such information. In this study, we drew on contemporary developments of multimodal (MM) learning analytics and designed an MM teacher dashboard with a notification system. The proposed system: 1) receives data from various sensors; 2) computes relevant cognitive and affective measurements; 3) visualizes the resulting measurements in a clean customizable interface; and 4) notifies instructors during moments of interest, so they may determine an appropriate method to support struggling students. To evaluate our MM teacher dashboard, we first collected multimodal data (MMD), performance data, and video recordings of students' interactions during an in situ study where 26 students engaged with a motion-based learning task. Then, we used our MM teacher dashboard to present the collected MMD and video recordings to 20 experienced teachers and educational researchers and collected qualitative data regarding respondents' insights on the advantages and challenges of visualizing students' MMD. Results showed that teachers found an MM teacher dashboard enhanced with a notification system, useful to complement their pedagogical practices. We offer empirically founded guidelines for design and integration of an MM teacher dashboard with notification systems, aimed to enhance teachers' understanding of students' learning states (e.g., real-time awareness of students' stress).


Multimodal Teacher Dashboards: Challenges and Opportunities of Enhancing Teacher Insights Through a Case Study
Serena Lee-Cultura , Kshitij Sharma , and Michail N. Giannakos , Senior Member, IEEE Abstract-Teacher dashboards provide insights on students' progress through visualizations and scores derived from data generated during teaching and learning activities (e.g., response times and task correctness) to improve teaching.Despite the potential usefulness of enhancing teacher dashboards, and the respective teaching practices, with rich information regarding students' cognitive and affective states (e.g., cognitive load), few studies on teacher dashboards have considered such information.In this study, we drew on contemporary developments of multimodal (MM) learning analytics and designed an MM teacher dashboard with a notification system.The proposed system: 1) receives data from various sensors; 2) computes relevant cognitive and affective measurements; 3) visualizes the resulting measurements in a clean customizable interface; and 4) notifies instructors during moments of interest, so they may determine an appropriate method to support struggling students.To evaluate our MM teacher dashboard, we first collected multimodal data (MMD), performance data, and video recordings of students' interactions during an in situ study where 26 students engaged with a motion-based learning task.Then, we used our MM teacher dashboard to present the collected MMD and video recordings to 20 experienced teachers and educational researchers and collected qualitative data regarding respondents' insights on the advantages and challenges of visualizing students' MMD.Results showed that teachers found an MM teacher dashboard enhanced with a notification system, useful to complement their pedagogical practices.We offer empirically founded guidelines for design and integration of an MM teacher dashboard with notification systems, aimed to enhance teachers' understanding of students' learning states (e.g., real-time awareness of students' stress).

I. INTRODUCTION AND MOTIVATION
A LEARNING analytics dashboard (LAD) is "a single dis- play that aggregates different indicators about learner(s), learning process(es), and/or learning context(s) into one or multiple visualizations" [1].Recent years have seen tremendous increase in the design, usage, and evaluation of different LADs.This article focuses on dashboards that visualize multimodal data (MMD) to support instructors' decision making (i.e., teacher dashboards).
In the context of learning technologies, MMD refer to the collection and integration of student data, derived from multiple modalities, during their participation with digital learning activities.Literature shows that insights extracted from MMD can provide useful information for both learners and teachers [2].Moreover, recent advances in sensors (e.g., eye trackers, wristbands, and motion sensors) empower us to transcend the limits of human observation, by providing real-time visibility of student data, which are normally inaccessible and nonobservable to teachers (e.g., students' cognitive, affective, and physiological processes) [3].This information affords a new vantage from which to explain various aspects of student learning behaviors and processes [4], [5].
In addition, integrating MMD from sensors, and more traditional indicators used to support student assessment (e.g., test and assignment scores), into teacher dashboards (i.e., MM teacher dashboards) provides a more holistic representation of students learning experiences and trajectories and may help guide teaching decisions and facilitate learning.Furthermore, research suggests that it is paramount to understand teachers' pedagogical perceptions in order to facilitate the integration of technological educational innovations into practice [6], [7].However, despite the potential impact of augmenting teacher dashboards with MMD, there is a lack of research focused on identifying the opportunities and challenges that teachers encounter when employing MM teacher dashboards in the classroom.
This study aims to bridge the research gap by identifying empirically based advantages and challenges associated with: 1) visualizing MMD on teacher dashboards; and 2) integrating MM teacher dashboards into pedagogical practice.Specifically, our overarching research question (RQ) asks: What are the main advantages and challenges of using MM teacher dashboards?
To address this RQ, we focus on two main objectives.
O1: Develop an MM teacher dashboard prototype.O2: Investigate educators insights on the advantages and challenges of integrating an MM teacher dashboard into pedagogical practice.
Thus, we developed the MM teacher dashboard, a teacher dashboard prototype with notification system, that leverages students' MMD as they interact with a learning technology (in our study, we used a motion-based learning game).The MM teacher dashboard: 1) receives data from eye trackers, wristbands, web cameras, and the learning activity; 2) computes the student's relevant cognitive, affective, and physiological measurements; 3) synchronizes and visualizes the resulting measurements in a clean customizable interface; and 4) notifies instructors during potential moments of interest (e.g., when a student's data undergo certain phenomena).In a previously conducted in situ study [4], we collected the MMD (from eye trackers and wristbands), learning activity performance data, system logs, and video recordings, of 26 students engaged with a motion-based learning task for learning geometry.We presented the synchronized student videos (playing the motion-based learning game) with the outcome of the teacher dashboard (powered by the students' data) to 20 domain experts (experienced instructors and educational researchers), and following a grounded theory approach [8], we collected qualitative data regarding their insights on the main benefits and challenges of visualizing students cognitive, affective, skeletal, and physiological processes, and the practicalities of integrating such a tool into the classroom.
The key contribution of this article is the identification of a set of implications for consideration in the design and integration of MM teacher dashboards to the classroom.In particular, we offer the following.
1) We present insights from a series of interviews conducted with teachers and educational researchers in which we explore teachers' perspective regarding usage of MM teacher dashboards.2) We report on eight key themes (four advantages and four challenges) that teachers and educational researchers identify during usage of MM teacher dashboards.3) We discuss how to find further knowledge on future design and integration of MM teacher dashboards into the classroom setting.

A. MMD in Learning Technologies and Analytics
Sensors (e.g., eye trackers, wristbands, and movement trackers) enable the observation of interactions and nuances that are impossible to capture using traditional data collection and learning analytics methods (e.g., test scores, log data, and human observation).These data, namely, the learner's cognitive, physiological, skeletal, and affective data, can reveal internal aspects of a student's learning experience, which are undetectable (or difficult to collect) by mainstream analytics.For example, gaze data from eye tracking can help quantify the amount of mental effort (e.g., cognitive load) expended by a learner during problem solving [9], [10], as well as indicate where, and for how long, a learner is directing their attention on-screen [11].Electrodermal activity (EDA) and temperature data from wristband sensors can report on a learner's level of physiological engagement and stress, respectively [12], [13], [14].Skeletal data derived from either motion sensors (e.g., Kinect) or video analysis can indicate the fatigue experienced by a learner, whereas video data can inform on the temporal emotional states expressed as a student interacts with different learning content [15], [16].
Parallel to this, monitoring, understanding, instructing, and evaluating the complex and multifaceted student experience with technology-enhanced learning activities is a difficult task, which requires the consideration of numerous dimensions, such as a student's performance metrics, cognitive, physiological, and affective processes.However, despite this, learning analytics has traditionally been driven by unidimensional data streams [1], which provide a limited view of the learner's experiences.This hinder teachers' ability to gain holistic awareness of the student's learning process, as the resulting analytics neglect to consider, and inform on, several critical aspects of the learner's experience (e.g., related to temporal, cognitive, affective, and psychomotor factors).
MMD refer to the collection and integration of multiple data modalities from both digital and physical dimensions, enabling access and analysis by computational methods [17].Research demonstrates that MMD help to construct a more holistic understanding of the complexities encapsulated within student learning processes [18], [19] which may empower educational stakeholders (e.g., teachers and researchers) with deeper insights upon which to scaffold their teaching practice.Accordingly, several studies have explored the capacity of sensor inclusive MMD to inform on key aspects of student learning processes from numerous angles [5], [20], [21], [22].For instance, Andrade [20] examined the combination of students' gesture/movement, with speech, eye tracking, and video data, to explain students' understanding of the ecological relationships between different animal populations.Researchers have also demonstrated that the combination of kinesthetic, eye tracking, and physiological data [EDA and heart rate variability (HRV)] outperforms individual data streams when predicting students' correctness performance as they engage with motion-based learning tasks [5].Different physiological data streams (HRV and galvanic skin response) have also been used to distinguish students' cognitive-affective states during their engagement with learning tasks of varying difficulty [22].
Collectively, these studies illustrate a synergistic partnership that results from the fusion of multiple data streams (including those from sensors) and its capacity to provide researchers with a richness of information that is bigger than the sum of its parts [23].In turn, this highlights the importance of using MMD to progress our understanding of students' experiences as they engage with learning tasks.However, despite the aforementioned advantages, the transition of using MMD to educational spaces (i.e., the virtual, physical, or hybrid classroom) to support teachers' understanding of their students' learning experiences has yet to occur.In an effort to facilitate this transition, we look to commonly employ educational tools (i.e., teacher dashboards) and explore how their adaptation to include MMD from sensors may be received.

B. Teacher Dashboards
LADs are information visualization tools that present students' real-time and historically contextualized states, thereby making the displayed information available for review and analysis by educational stakeholders [24].Their aim is to provide awareness regarding a learner's or classroom's educational climate and offer opportunities for (self-)reflection and sensemaking, as a means to empower efficient pedagogical decision making to improve the learning process [24], [25].Numerous LADs have been designed and implemented for face-toface [26], remote [27], and blended learning environments [28], [29].Their usage supports teachers [30], students [31], [32], or both [26], though literature shows that LADs have primarily been developed with teachers in mind [1].Most commonly, teacher facing LADs (i.e., teacher dashboards) report on the traditional measurements used to assess student learning, such as a student's progress, answer correctness, knowledge acquisition, and skill development [1], [25], [33].Also, some teacher dashboards provide forums and additional communication channels for direct teacher-student or student-student interaction (e.g., Blackboard).On an individual level, the visualized data help teachers identify at-risk students, whereas at the classroom level, teachers may be made aware of common issues that indicate a need to revisit or revise learning material.
With MMD gaining traction as a valuable way to coalesce the multiple dimensions of student learning and thereby afford greater opportunities to understand the social and cognitive nuances exhibited during learning experiences [17], research has pushed toward MMD visualization (namely the creation of MM teacher dashboards [30]) to ease accessibility and comprehension of the captured data.To this end, research efforts primarily combine systems logs with alternate data modalities [1], such as data from the learning activity [34], [35] and user-entered information [27], [34].For example, early work by Bull et al. [34] integrated students' self-assessment, textual behavior (via chats and forum discussions), and learning artifact data (from multiple choice and open ended test questions) into an open learner model for dashboard visualization aimed to support both teachers and students.In a similar vein, Hu et al. [35] combined system logs and measurements of student interactions with online learning content (e.g., online course material that students chose to engage with, assignment scores, and forum discussions statistics) to build performance predictions and display early warning notification visualizations on a teacher dashboard.Ez-Zaouia and Lavoué [27] created the EMODA teacher dashboard, which displays and contextualizes students' emotional state derived from four different data streams (audio, video, self-report, and interaction traces) in order to facilitate the socioaffective relationships between remote teachers and their students.Despite the documented growth of research on teacher dashboards and recent developments in multimodal learning analytics (MMLA) research (i.e., MMD purposed for learning analytics), the use of MMD to inform both teacher dashboards and decision making is still limited (e.g., is restricted to mainstream analytics, with only a few examples that utilize very limited MMD, such as audio and video data).
Though MMLA is still in its infancy [17], progressive efforts in the direction of MMD teacher dashboards have demonstrated its capacity to aid teachers in enhancing the overall learning quality and performance of their students [35].In a recent literature review on LADs, which surveyed 55 papers, Schwendimann et al. [1] reported only two studies [36], [37], which implemented sensor-driven MM teacher dashboards.Specifically, the MeLOD environment, conceptualized and developed by Fulantelli et al. [36], includes a dashboard for assisting teachers in monitoring and evaluating students during mobile-device-based learning tasks.The MeLOD dashboard integrates and visualizes multiple data streams from students' mobile device sensors (e.g., GPS technology), events pertaining to the communication flow occurring between different students (e.g., learning artifacts expressed as in-application votes and comments), and system logs of student-context interactions (e.g., events between the student and learning application) [36].Finally, the authors in [37] and [38] developed a teacher dashboard to present a real-time synchronized fusion of students' application logs, audio, and kinesthetic data (captured using Kinect sensors) during their interactions with a collaborative tabletop learning activity.Also, in the realm of MMD-based learning tools, the Multimodal Tutor [39] is a system that offers real-time adaptive feedback for the development of psychomotor skills (e.g., cardiopulmonary resuscitation (CPR)).However, it employs a hybrid approach that combines machine learning, artificial intelligence, and human annotation to support analysis and feedback delivery of a learner's task execution.Challenges related to capturing actions and interactions through use of sensors and then translating the resulting MMD into meaningful and actionable insights, such as real-time formative assessment and visualizations for teacher guidance and postreflective reviews, are still under research [40], [41].Nevertheless, to the best of the authors' knowledge, there are no other studies focused on understanding the advantages and challenges associated with using MM teacher dashboards.

III. MM TEACHER DASHBOARD
In order to investigate the research objectives of this article, we first needed to design and develop the necessary artifact.Artifacts correspond to novel designs (e.g., prototype systems and interfaces), which satisfy a specific set of qualities or consist of certain components (such as functionalities and affordances) and that allow us to experiment (e.g., to isolate and test certain components) [42].Artifact use enables us to formulate the necessary conditions by isolating certain functionalities and testing our hypotheses through experimentation.Although artifact use has certain research shortcomings (e.g., specifying a concept through artifact realization forces us to make design decisions, which may affect the produced knowledge), they empower researchers to put design ideas into practice and evaluate them empirically.The produced knowledge may be used to support the design of future artifacts in the form of lessons learned or design implications [43].In this section, we describe our particular artifact that allowed us to utilize students' MMD to power a teacher dashboard.

A. MM Teacher Dashboard
The MM teacher dashboard is an LAD prototype that visualizes a collection of education-related measurements, which are derived from students' MMD.The dashboard supports MMD captured via web camera, wristband (which collects EDA,  blood volume pulse (BVP), HRV, and skin temperature), and an eye-tracking device during students' interactions with digital learning activities.In addition, the MM teacher dashboard is equipped with a notification system, comprised of three different real-time indicators, that informs the teacher when a student may be struggling (see Section III-E).These moments may signify data phenomena, which are worthy of further investigation to understand student needs, as well as when feedback to students may be appropriate.
The MM teacher dashboard is divided into two components: a learner side, which is operated by the student performing a learning activity, and the viewer side, which is monitored and controlled by a teacher who wishes to view the data (see Fig. 1).The learner side consists of the digitized learning activity, web camera, and the sensors worn by the student (eye tracker and wristband).The viewer side is only the MM teacher dashboard GUI that visualizes the MMD.
The MM teacher dashboard has two different start-up modes, either of which is initiated by the teacher through the viewer side: 1) data collection mode (see Fig. 2, blue) and 2) replay mode (see Fig. 2, yellow).Each student has a profile in the system.Once the viewer side has started in the desired mode, the teacher either creates a new student profile (e.g., in data collection mode, with a student unknown to the system) or selects a preexisting student profile (e.g., used by both start-up modes).When a student profile is activated (e.g., new one created or preexisting one selected), a learning session associated with the student is started.In the data collection mode, the video and MMD from students' interactions with the learning content are saved to a database (e.g., a dataset is created or modified) and presented by the viewer side in parallel.This requires connection from at least one learner side, where a student engages with learning content in real time.Alternatively, in the replay mode, the viewer side visualizes a previously populated dataset without a learner side connection.Thus, no students are involved.
During a student's learning session, measurements are continuously calculated from the sensors' raw data and sent to the viewer side to be visualized.The teacher gains visibility into the student's learning experience by viewing: 1) charts of the students' data; 2) a screen capture of the learning activity; and 3) a video of the student performing the learning activity (see Fig. 1, right).Multiple students, each with their own learner side (e.g., learning activity and sensors) can connect to the same viewer side, so that the teacher can monitor several students at Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I DESCRIPTIONS OF THE EIGHT MM MEASUREMENTS VISUALIZED BY THE MM TEACHER DASHBOARD, INCLUDING THEIR DEVICE SOURCE
once.The teacher can access the data in three different session views (i.e., ways the MM teacher dashboard visualizes the data to the teacher), which are explained in Section III-D.

B. Dashboard Devices and Data Collection
The MM teacher dashboard collects students' physiological, affective, and movement data using three devices: 1) a web camera that records facial expressions and full body interactions; 2) a mobile or stationary eye-tracking device to capture gaze data; and 3) a wristband that obtains physiological data (with sensors for EDA, BVP, HRV and skin temperature).The most comprehensive representation of a student's learning experience is achieved using all three devices.However, in the absence of a missing device, the charts associated with the missing data are excluded, and no other aspect of the MM teacher dashboard will be negatively impacted.Also, the MM teacher dashboard records event data (i.e., correctness scores) from system logs.
1) Mobile and Stationary Eye Tracking: A student's gaze data are collected using a Tobii eye-tracking device.During a computer-based task where the student is seated throughout the learning activity, a stationary eye tracker is attached to the bottom of the computer screen.Alternatively, during a motion-based learning task, where students interact with educational content through movement, the student wears a pair of Tobii eye-tracking glasses.Both stationary eye tracker and mobile eye-tracking glasses must be configured with 50 Hz and one-point calibration.In addition, the eye-tracking glasses record the student's field of view via an objective camera built into the nose bridge of the glasses.Video resolution is 1920×1080 at 25 frames/s.
3) Video: The MM teacher dashboard records a student's facial and skeletal data using a front facing Logitech web camera.The camera is installed on top of the screen displaying the learning activity.If the student is engaged in physical activity during the learning task, the web camera films the whole body of the student.However, if the learning task is computer based, only the face of the student is recorded.In this case, the camera is set to record HD video at 10 frames/s, and it must be zoomed-in to at least 200%.

C. Dashboard Measurements
The MM teacher dashboard calculates and visualizes the following eight measurements from the collected MMD: cognitive load, perceived difficulty, information processing index, physiological arousal, physiological engagement, physiological stress, fatigue, and educational-specific emotions (see Table I).

D. Dashboard Session Views
The MM teacher dashboard has three ways to view data (see Fig. 3): 1) single session view (SSV), which displays real-time learning data from a single student's session; 2) all sessions view (ASV), which shows real-time learning data from all connected students; and 3) replay session view (RSV), which allows the teacher to interact with previously recorded session data from a single student session.Each view is divided into panels, with the bottom panel showing the charts of the student's data.The SSV and the ASV have an additional side panel containing a list of student icon buttons representing each connected student (bottom panel, lower left).The teacher can switch between different student session views by clicking on these buttons.Also, there is a button to switch into the ASV (bottom panel, top left).The SSV and the RSV also show a screen capture of the learning activity (i.e., what the student sees, top left) and a video stream of the student performing the learning activity (captured by the webcam, top right).
1) Chart Behavior: For each of the three views, the teacher selects which measurement data to display by opening a pop-up menu in the upper right hand corner of the bottom panel.Each measurement's data are shown on its own line chart, which appears on the main panel at the bottom of the screen.Charts can be changed to show only the last value received in numeric form.Every chart has an information icon, which, when hovered over, provides the teacher with information about that specific measurement and its source device.Charts can be resized, rearranged, and removed on demand.
2) Single Session View: The SSV (see Fig. 3, left) displays the session data belonging to a single student in real time.A video feed of the student during the learning activity streams in real time.This allows the teacher to match the student's actions to the data points in the charts.Learning sessions can be recorded by clicking on the yellow start recording button.A red dot appears on the student's icon button to indicate that the student's session data are being recorded.The recording feature saves all of the values of the student's data (not just the measurements being displayed via charts), as well as their video.The recording can be viewed by the teacher at a later time, using the RSV.
3) All Session View: To view the learning data of all connected students together in real time, the teacher uses the ASV (see Fig. 3, center).After the specific measurements of interest have been selected, the corresponding charts are displayed in a matrix with each connected student represented by a single row, and each variable is displayed in its own column.Due to scalability concerns, videos of the student actions and learning sessions are not provided by this view.

4) Replay Session View:
The RSV (see Fig. 3, right) allows the teacher to interact with the data from a previously recorded learning session belonging to a single student.The teacher can choose to view a recorded learning session from a list of all previously recorded sessions, which is presented upon first starting the MM teacher dashboard, or when the sessions for all connected students have been terminated.A yellow slider at the bottom of RSV is used to control the data.Moving either end of the slider adjusts the window of time associated with the charts and updates the data accordingly.The video still corresponds to the start time stamp of the slider.If notification features (i.e., incorrectness lines and threshold surpassing feature, discussed in Section III-E) are enabled, they also show in the RSV.This way, the teacher can match the critical moments of a student's cognitive, physiological, and affective state to the image that is shown in the dashboard at that time.

E. Dashboard Notification Features
The MM teacher dashboard has three notification features, which inform the teacher when a student's data may be of interest: 1) threshold breach notification, which indicates when a student's data peak above, or falls below, an expected range of normalcy; 2) incorrectness lines, which notify when a student answers a question incorrectly; and 3) student icon overlay, which signifies when the data of a student that is not in current view may require the teacher's attention.

1) Threshold Breach Notification:
The MM teacher dashboard can be configured to indicate threshold ranges for each of the student's charts.When this feature is enabled, dotted red lines representing the upper and/or lower bounds of the threshold range overlay on the line chart (see Fig. 4, top left).If the chart Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply. is in numeric view when a threshold is breached, the color of the offending value is displayed in red for the duration of the breach.This notification allows the teacher to see when, and for how long, certain measurements breach their threshold range.If there is only a single spike in the student's data, the teacher may choose not to take action.However, if the breached value lasts longer, the teacher may wish to intervene.
2) Incorrectness Lines: This feature indicates when a student answers a question incorrectly, by placing a vertical red line on each chart (see Fig. 4, bottom left).These incorrectness lines move across the chart time line in real time so that the teacher may observe the student's cognitive, physiological, and affective states when the incorrect answer was provided.
3) Student Icon Overlay: This feature notifies the teacher when a student may need support by placing a transparent red overlay on the icon button associated with that student (see Fig. 4, right).This is useful when several students are connected to the viewer side and engaged in simultaneous learning sessions as it brings the teacher's awareness to struggling students when the dashboard is not focused on them.The teacher can then determine who, and how, to provide support.If multiple students experience difficulty at the same time, an overlay is placed over each student's button.

F. Dashboard use Case Example
To illustrate how the dashboard may be used in practice, suppose that a teacher is viewing the data for a particular student, Kris, in the SSV, and a different student, Sam, (whose live session is not currently being viewed by the teacher) begins to struggle (see Fig. 5).This is communicated to the teacher through the appearance of a red overlay on Sam's icon button in the side left panel.Upon seeing the red overlay notification, the teacher changes the dashboard to focus on Sam's data.Sam's charts show that she has provided several incorrect responses in a short time.Also, her stress and cognitive load charts have exceeded their respective thresholds.This indicates to the teacher that Sam is experiencing high amount of stress and is approaching (or is already in) a state of cognitive overload.With this knowledge, the teacher may use their contextualized understanding of the situation to determine the appropriate remedial action to pursue.

IV. METHODS
To empirically evaluate the MM teacher dashboard, we first needed to identify a learning technology that is capable of leveraging MMD.To do so, we selected motion-based learning technology due to its ability to capture and account for learner's embodiment with the use of MMD.The overall approach that we adopted for this project is design-based research (DBR) [56].DBR is a systematic but flexible methodology that aims to improve educational practices through iterative analysis, design, development, and implementation, leading to contextually sensitive knowledge and design principles [56].In the context of this study (a cycle of the DBR), we used a realistic and reliable dataset to power the MM teacher dashboard and invited experts to participate in our study.The participants experienced the MM teacher dashboard, and their insights were collected and analyzed following a grounded theory approach [8] that enabled us to address the overarching RQ of this article.Specifically, to investigate educators' insights on the advantages and challenges of integrating a MM teacher dashboard into pedagogical practice.
Grounded theory [8] is a systematic qualitative research methodology, from which knowledge and theories are discovered from empirical data.Rather than framing the research around a set of initial assumptions or a preconceived hypothesis to (dis)prove, the grounded theory approach allows the theoretical narrative to naturally surface from the raw data.This is accomplished through iterative cycles of data collection and analysis as interrelated processes.Each iteration acts to refine, arrange, and connect the emerging concepts and guide successive data collection and analysis iterations.The grounded theory approach is appropriate when the topics of interest center on developing knowledge that is rooted in human experiences, and the preexisting body of knowledge does not adequately represent the phenomenon in question [8].Concerning this study, the corpus demonstrates significant research on LADs [1].However, with respect to MM teacher dashboards, the research aimed to understand the landscape of teachers' perspectives is lacking.Section IV-F describes our grounded theory approach, during which we conducted semistructured interviews and iteratively analyzed the collected data via an inductive category development process [57].

A. Motion-Based Learning Technology and the Dataset That Powered the MM Teacher Dashboard
During winter of 2019, we ran an in situ study [4], in which 26 typically developing children (10 females and 16 males) of ages 10-12 (mean = 10.95 years and standard deviation (SD) = 0.21 years) engaged in a motion-based sorting task for learning geometry.We continuously recorded students' sorting task engagement via web camera and captured data from two devices: gaze data from eye-tracking glasses and physiological data (with sensors for EDA, BVP, HRV and skin temperature) from wristbands.Also, we collected the event data (e.g., response correctness) from system logs.During the current study, we used the dataset from [4] to power the MM teacher dashboard

B. Participants: Learning Technology Experts
In order to recruit respondents who were able to offer insights on how visualized MMD might facilitate the teaching process, we constrained our participant search to include educational researchers and experienced practitioners in the area of learning technology and interface design.This included teachers and academic researchers (masters, Ph.D.s, Postdoctoral, and Professors) with a focus on learning technologies, information technology, and computer science in a learning context.Accordingly, we classify our participants in three roles: teachers, researchers, and dual role (participants with experience as both teachers and researchers).In total, 20 respondents was recruited through purposive sampling [58].The recruitment strategy involved inviting participants through the authors' network, as well as email distribution through various academic channels.The respondent population consisted of five experienced teachers, six academic researchers, and nine individuals with experience in both teaching and educational research.Eight respondents identified as male, while 12 described themselves as female.Their age ranged from 27 to 41 years (mean = 32 years, SD = 5.5 years), They represented seven countries: Canada, the USA, Lithuania, Norway, the U.K., Germany, and the Netherlands.The detailed profiles of the 20 participants can be found in Table IV of Appendix A.

C. Procedure
Our study sessions were conducted by a single interviewer (i.e., the first author) via Microsoft Teams video conferencing platform.They were conducted one-on-one and included the following steps.Initially, the interviewer explained the study's objective to the respondent and initiated an open conversation (i.e., preinterview) to learn about the respondent's background.Next, the interviewer played a 6-min video, introducing the project's overarching RQ, foundational concepts on teacher dashboards, the use of sensors to collect MMD, and practical applications of those collected MMD.Upon completion of the introductory video, the respondent was encouraged to ask questions to clarify the concepts presented.After all of the respondent's questions had been satisfied, the interviewer showed the respondent a 12-min video introducing the MM teacher dashboard (see Section III-A), its three views and notification features, and a collection of use cases demonstrating the different ways the MMD is presented to the dashboard user.Once the video was finished, the respondent was given a second opportunity for questions.Following this, the interviewer conducted a semistructured interview to collect respondent's opinions, preferences, and concerns on visualizing MMD via a teacher dashboard.Both pre-and semistructured interview component employed an interview guide (described in the following section).Transcripts of the recorded interviews were automatically generated by Microsoft Teams.Subsequent manual editing of the interview transcripts was also performed by the first author to ensure quality and correct errors encountered.On numerous occasions, this involved the author revisiting the source video data.Each of these procedural steps, and the utilized interview guide, are explained in more detail below.

D. Data Collection: Interview Guide
Motivated by our RQ, the authors developed an interview guide (see Fig. 8 in Appendix C) containing a collection of curated open-ended questions to lead the interview process.The interview guide contained two parts.The preinterview consisted of nine open questions purposed to learn about the respondent's research or teaching pedagogy-based background, as well as their experience with dashboards, familiarity with MMD, and use of sensors.The main interview questions were made up of 15 open questions aimed to investigate the benefits and challenges of visualizing MMD on teacher dashboard, using the MM teacher dashboard to anchor this exploration to a tangible real-world artifact, thus making the questions more accessible to the respondent.The main interview questions were grouped into three categories related to various aspects of the MM teacher dashboard (e.g., views and data comparison, notifications, and general questions).

E. Interview Protocol
Individual semistructured interviews were performed by the first author, in April and May of 2022, according to the aforementioned prepared interview guide (see Appendix C).Due to the current COVID-19 situation, the Microsoft Teams video conferencing platform was used to conduct, digitally transcribe and record, the interviews.Each interview (i.e., includes preinterview) lasted between 34 and 64 mins (mean = 46 min and SD = 8.44 min). 1

F. Data Analysis
Content analysis of the interview transcripts was conducted by the following process.
1) Coding: Initially, the first author examined each transcript twice to obtain an overall impression of the data.Next, an initial round of open coding was performed, where three transcripts (one from teacher, educational researcher, and a participant with experience in both areas) were selected at random, and repeatedly analyzed in depth, in search of quotes describing aspects related to the benefits and challenges of sensemaking MMD to inform decision making and learning design.An initial set of naturally emerging central themes (e.g., advantage, challenge, anecdote, and feature note) and related codes (e.g., improved dialog:advantages, instructor buy-in:challenge, and suggested:feature note) was identified.
A preliminary codebook was produced, which categorized the resulting themes and related codes.To ensure clarity, all authors discussed the central themes and codes, merged categories, and distilled definitions, until a refined version of the codebook was reached.Then, three new transcripts (one from each respondent role) were selected, and the original author identified and labeled relevant quotes according to the updated codebook.Multiple codes were able to be assigned to the same block of text.During the second coding step, extra care was given to ensure that the revised codebook provided adequate coverage of all benefits and challenges encountered within the newly selected transcripts.No changes were made to the codebook upon the completion of this step.In the final step of the coding process, all transcripts (including the original three which formed the basis for the preliminary codebook) were coded.The finalized codebook can be found in Fig. 7 of Appendix B. Though all authors supported the coding process, coding was conducted by a single author.As such, inter-rater reliability was not calculated.
2) Extraction of Main Benefits and Challenges: The author then transformed the coded dataset into a collection of six tables: one table of advantages and one table of challenges, corresponding to each of the different roles (teacher, researcher, and dual role).For each table, rows mapped to codes and columns mapped to respondent IDs.Then, for each table row, the author identified and summarized the benefits and challenges for the associated code.Summaries, including supporting respondent IDs, were documented in a new column appended to each table.This resulted in six additional columns, with a distilled list of benefits and challenges for each code.The original intention was to extract challenges and benefits related to each code for each respondent role.However, based on their direct experiences with instructors and within the educational domain, researchers often spoke from a teacher's perspective during the interview process.As a result, when comparing the main themes across respondent categories, it became apparent that there is no difference on the benefits and challenges across the three roles.Therefore, after much discussion, authors made the decision to consolidate the benefits and challenges derived from the different respondent categories.This is reflected in the results presented in the following section.Fig. 6 illustrates our grounded theory process.

V. RESULTS
We report on two primary themes: the advantages and the challenges of MM teacher dashboards, as identified by educational experts.We organize each theme in findings subsections, and each subsection is organized by: 1) providing a descriptive subtitle of the finding; 2) presenting an overview of the finding; 3) presenting vignettes as descriptive narrative from our interview sessions; and 4) offering a reflection of the vignettes and our finding with respect to the overarching RQ of this work.

A. Advantages of MM Teacher Dashboards
Respondents identified numerous advantages of using an MM teacher dashboard in practice, each derived from the capacity to provide teachers with increased learning state awareness of their students.
1) Learning State Awareness: All respondents regarded increased learning state awareness as a central advantage of using MM teacher dashboards to support their teaching practice.Respondents acknowledged that they currently have a fragmented understanding of how their students experience the learning process (through mainstream dashboards in online settings, or via "teacher sensing" when physically interacting with students), due to limitations on the times and sources of student data from which they derive their conclusions.Namely, they are primarily constrained to post factum conclusions based only on what they can directly observe or be told by their students.
"For now, the learning process is like a black box for me and I only have a partial understanding of students' experience after collecting and analysing their data [data associated with their learning outcomes]."(Quote 1, D2).
Visualized MMD was identified as a tool to empower teachers by providing access to additional data streams, and temporal transparency of the students' learning state, which would allow teachers to better understand students' academic strengths, weaknesses, interests, and triggers and draw connections to the learning content.
"The data may provide some insights into why students are or aren't performing.Maybe it [the activity is] just not grabbing their attention.Maybe their cognitive load is too high and it's wearing them out or something.Maybe it's too easy for them.Either way, being able to view that data can help you say 'OK, this is why my student is performing this way', and then in the moment you can catch what exactly is triggering your student."(Quote 2, R2).
In addition, respondents appreciated the potential of visualized MMD data for helping understand less expressive (verbally or through gesture and facial expression) students, as illustrated by the following quote: "I have some students that are not very talkative, and I would love to know what's happening in their brains.And this would be like, maybe a sneak peek."(Quote 3, D4) Respondents claimed that teachers could leverage this raised awareness of a student's experience within the learning process, for improved inquiry with students, more appropriate feedback delivery, and decision making within learning design.We describe each of these identified advantages in the following subsections.
2) Evidence for Discussion With Students: One advantage that was identified by numerous respondents was the MM teacher dashboard's affordance to improve inquiry between teachers and students by providing empirical data for discussion.Respondents claimed that combining the qualitative insights observed during students' learning session with the objective and quantitative insights from the MMD would help teachers convey their messages to student and engage them in discussions about their own learning.
"I can see how valuable having that pure data is because people can't argue with the pure data."(Quote 4, T5).
Respondents connected enhanced teacher-student dialog to several positive outcomes.For example, when a student is improving in their learning tasks, the MMD could be used to identify and verbally recognize the student's achievements, which is important in developing a student's self-confidence and helping change a student's negative mindsets regarding their own learning abilities.This was highlighted as especially significant for academically at-risk students, who have a history of receiving primarily negative feedback regarding their learning.
"For the kids who are really struggling, the kids who are always being told and always being given the feedback, purposeful or not, that they're struggling, and they don't do well, and schools are hard place for them and school is hard, and they get that feedback so much that when you can give them feedback that's 'Hey, you did great.You did awesome.Look at you.And look here's the data.I'm not ******* you'.Like, they need that more than any kid... Like we have to show them 'You can do it, you know, this is a strength for you.' That's awesome."(Quote 5, T5).
Building on this, respondents claimed that the direct teacherstudent dialog could center on exploring strategies purposed to get struggling students back on track and foster student agency in developing their own learning path through the identification of individualized adjustments and preferences.This is illustrated in the following quote.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

"It would also be good for the students to see it [the MMD] themselves. And to be like, 'OK, what can we do to make this better for you if you're having issues?' I personally believe it's really, really good to include the kids in their learning and there's a lot of kids that would be like, 'Oh, OK, well, that was what was going on'. I could see kids really being, like, involved in this."
(Quote 6, T4).
Improved teacher-student dialog was also extended to entire classrooms, as an entry point for teachers becoming more cognizant of class needs in real time.For instance, in reference to the notification system, T5 mentioned "I could easily see a situation where a lot of kids are hitting their thresholds and then the buttons are all going red.But I think that's also really good data for a teacher to say 'OK, the task is above everybody's line right now, maybe we should pause and have a discussion about it and then come back to that task'."(Quote 7, T5) 3) Evidence for Informed Learning Design: Another central advantage of using the MM teacher dashboard was its capacity to inform and guide teachers in making evidence-based learning design decisions.Respondents discussed the benefits of using students' MMD as a proxy for the appropriateness of the learning content (with respect to difficulty and induced stress) and considering their observations and insights in design of future learning activities.This is supported by the following quotes.

"If they [the students] are all struggling at the same time, if they are in the same place, in whatever activity they're in, then that probably means there's something that needs to be done about the activity. Like, maybe it's too difficult or stressful. So, it [the MM teacher dashboard] gives you an indication of if the learning content is too difficult or if there are any changes you need to make." (Quote 8, T3). "I think this is useful if the event has already happened and you can go through it [MMD]. It is much easier since you can make notes and you can identify the points where the students had difficulty in terms of your teaching. It [The MMD] can be useful for redesigning the task to make it easier or more difficult based on like, how the children were experiencing it [the learning activity]." (Quote 9, D5).
In addition, several respondents discussed using an MM teacher dashboard to identify which aspects of a learning activity have positive or negative impact on students' engagement metric, and then using that knowledge to intentionally design or revise their learning activities to exploit or avoid those elements, respectively.
"If you look at engagement, it is very important from the classroom perspective.It is always important to keep all the students engaged.So if they are really getting engaged in a certain activity, at a certain point, that might be useful to know, and incorporate such things in other activities...for learning activity design and redesign.The opposite is also true." (Quote 10, D5).
Respondents also indicated that observing students' longitudinal MMD (i.e., history of data via replay mode) may help track students' progress and inform the need for revisions of individual (i.e., personal learning plan) or class educational development plans (i.e., course curricula) to facilitate improved learning experiences.

"What is very important is longitudinal data and how there is, or is no, change over time. So this option to compare previously recorded MMD would allow to have this longitudinal data, and then teachers could determine if improvement is needed or if things seems as expected, and adjust lesson plans accordingly...Then if we think about instructional design or curriculum design,... it is important to find the most efficient ways to design
the curriculum so that it meets the students' needs.And if you are a good teacher, you're reflecting all the time.You're updating and you're trying to make lectures better.You're trying to adjust the assignments.And so knowing how well your students progress over time, seeing where the red flags are, and fixing your assignments based on that, or maybe providing the students with more input beforehand so that they struggle less...I think that's very important.So seeing this kind of data would be very useful for teachers."(Quote 11, D9).
To support learning design decision making, by way of understanding students' MMD, a few respondents suggested the addition of a customizable report-generating feature to inform teachers on the MMD status of individual students or class as a whole.Reports generated could identify MMD trends, either from a single lesson or a specified length of time (e.g., longitudinally).In addition to helping the teacher save time when reviewing student MMD, respondents stated that access to students' highlighted MMD data trends would expedite their reflection on the reported lessons and contribute to preparation for future learning sessions.
The aforementioned quotes demonstrate that respondents primarily attributed post learning session use of the MM teacher dashboard to support learning design decision making.In this regard, respondents recognized and valued MMD as a potential guide for designing engaging learning activities, moderating difficulty and stress levels, as well as for evaluating student progression with intent to design or revise individual and class curricula.
4) Appropriately Timed and Allocated Feedback: Another advantage was appropriate feedback delivery.Regarding feedback allocation, respondents disclosed that identifying which students to support was valuable, especially with respect to learning the educational support needs of shy or easily embarrassed students that refrain from seeking help when they are in need. "

It [using the MM teacher dashboard] would be easier for teachers to understand what's really going on in the classroom and who is really struggling." (Quote 12, D6).
"Maybe a student says they are fine or are not asking for help, but then you can see that they do need help, so you can help them without them having to ask." (Quote 13, T3).
Furthermore, respondents indicated that several students could experience difficulty in their learning tasks concurrently, and therefore, comparing students' real-time MMD could help determine an order for offering help.

"...[D]oing a comparison could be helpful too, in the instant if you are deciding between which student to help [...] But then looking at the data you can see that one is more stressed or finding it more difficult. So you can go help them and leave the others." (Quote 14, T3).
With respect to feedback delivery timing, respondents claimed that the MM teacher dashboard and notifications features could inform teachers when to offer students help.

"...[I] f I'm a teacher and I have this dashboard then I can monitor when they [the students] are doing some learning task and then be able to see exactly when do I need to intervene. And especially I like the thresholds because it [makes] it easier to see when their values exceed." (Quote 15, R3).
Furthermore, respondents described different situations where the MM teacher dashboard could foster proactive feedback delivery (e.g., teacher offers the student help before/without the student asking) and praised this in contrast to the reactive approach, in which assistance is offered in response to students' requests.This is illustrated in the following two quotes.In the latter, D4 highlights that learning state awareness and proactive feedback delivery enables a more balanced distribution of feedback among students.
"I get the notification that some kid's perceived difficulty is just going through the roof, then I can then do another lap around the room and target that kid.So, like, 'Okay, what's going on?Why is this so hard?How can I help you understand this thing better?', which might allow me to capture some of those difficulties faster than waiting for the kid to tell me, right?And that's being proactive, as opposed to reactive.That would so amazing."(Quote 16, T1).
"... this [current method] isn't the greatest system for me, because it is reliant on them telling me that they need my help and this system [the MM teacher dashboard] would be more of like...they wouldn't actually have to raise their voice.I would be able to help the people who are just in a corner not doing anything because I would notice.So that would be super helpful because it would be more equal."(Quote 17, D4).

B. Challenges of the MMD Teacher Dashboards
Here, we present the main respondent-identified challenges of using an MM teacher dashboard.
1) Technical Ability and Support: The devices connected to the dashboard are uncommon in children's learning environments and the majority of teachers reported little or no experience using them.Accordingly, respondents expressed concern regarding the level of technical difficulty associated with the practical aspects of implementing an MM teacher dashboard and its required sensors.
"For me it seems like, difficult to get the equipment on a child.. Like, to learn how to use the sensors, inform kids how to interact with them, to record this information.It's a high degree of technical understanding."(Quote 18, R1).
Respondents also mentioned that due to prior lack of exposure to sensors and their produced data, teachers may exhibit selflimiting beliefs regarding their own technical ability to master using novel technologies, such as an MM teacher dashboard.
"Maybe some teachers don't think they can learn how to operate these technologies.It might be scary for them to learn about these sorts of new things..." (Quote 19, T3).
Correspondingly, the need for special training to inform on understanding and deploying the initial technical infrastructure, device connection, dashboard use, and trouble shooting was identified as a potential challenge by several respondents.

"[I] f you're using this in a classroom environment, how would we [teachers] set it up? Is this the teacher's job? Who teaches them how to do this? And what if they have troubles.. Who helps them?
Where is the support?" (Quote 20, D5).
R5 indicated the need for technical support during MM teacher dashboard use and expressed concern over the availability of IT support at their institution.
"Teachers have no idea how this technology works, about how to fix or set it up...You need to have a team of IT people to support this at all times.It's very hard in the school setting."(Quote 21, R5).
2) Educational Data Literacy: One main challenge identified by respondents was that teachers may lack the knowledge required to understand and interpret the MMD visualized by an MM teacher dashboard.The majority of respondents reported unfamiliarity with the MMD measurements (e.g., cognitive load, perceived difficulty, etc.) displayed on the MM teacher dashboard, as well as their importance to, and influence on, children's learning.Without these important educational data literacy skills [59], teachers may not know which measurements to turn on when starting a learning session in order to monitor/investigate their target interest, or what actions to take to remedy students' learning experience.In support of this, respondent offered the following quotes.

"I think one of the biggest challenges here is the dependence on teacher's digital skills, because if they're not very comfortable with figuring out data or analysing charts and things like that, then it [the MMD] is basically of no use to them. They won't use it [the MM teacher dashboard] because they don't get it." (Quote 22, D6). "A big challenge I see is that the teacher needs to make her decision on what type of data they want to show. So, they must know what type of data is most insightful for them to understand the student experience. And for this they need to have kind of training beforehand." (Quote 23, D2).
Compounding this issue, respondents also raised concern over the possibility of teachers not knowing which actions to execute in response to the different MMD scenarios that students experience in their struggles.

"And what would you expect the teacher does with this information? You know, because I come from the assumption that the teacher does not have the literacy in knowing how does cognitive load or how much does stress, you know, influence the learning task. This is very, very specific, you know." (Quote 24, D1).
Taking a more extreme stance, one respondent questioned the appropriateness of how teachers employ the MMD.She indicated that the data could be used to affirm teachers' negative preconceived notions of children's behaviors and abilities.And that this would perpetuate teachers' "comparing students against a status quo," rather than facilitating learning through individually tailored support, or assessing students based on individual improvement.
"Your dashboard has innocent intentions but also the ability to go very wrong...The MMD will be used to reaffirm what teachers already think they know about their students.We can use this Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
tool to collect more data to also reaffirm things like, this kid has problems or difficulty solving problems or sitting still...So it's used to compare and rate against the norm, instead of to help personalised learning" (Quote 25, T2).
To alleviate these challenges, several respondents mentioned the importance of teacher training to better understand how to harness the potential of an MM teacher dashboard.However, these suggestions were often coupled with concerns of time as a limited resource.
"My concern is just for the very beginning, that could be really challenging for us [teachers].And we need a technology knowledge level a bit higher which means extra training, which takes extra time."(Quote26, R4).
3) Imposed Cognitive Load: Increased cognitive load caused by the use of MM teacher dashboard was one of the main challenges identified by respondents.Specifically, teachers may become perplexed by the mental effort required to observe, understand, and interpret the vast amount of data displayed on the MM teacher dashboard.In reference to teachers viewing students' prerecorded MMD via RSV, R1 said "It's a lot of information at one time so the cognitive load of the teacher is something to be aware of."(Quote 27, R1).This sentiment was echoed by D2, "Showing so many things at once might not be useful, you know, because it's too confusing" (Quote 28, D2).
Building on this, respondents agreed that combining the pressures of a real-time teaching environment (where teachers already split their attention in several directions), with the complexities of using an MM teacher dashboard to monitor the MMD of multiple students, may further burden teacher's cognitive load.Supervision of a hybrid learning environment requires a high degree of alertness, ability to multitask, observe and interpret MMD, and react quickly (e.g., with feedback delivery and troubleshooting) to the situations and data of multiple students, in order to keep pace with the ongoings on both the screen (MM teacher dashboard) and the classroom in real time.This was supported by the following quotes: "I'm concerned about the teacher getting their own cognitive overload of having to process so much data as well as, fielding questions and managing the class."(Quote 29, R2).
"I think this will be very useful, but the problem is basically the amount of information you show to the teacher, because this is a serious question about the teacher's cognitive load.Like, how much information can they digest...especially in real-time?You know, teachers have to juggle so many things already.It's difficult to manage it all in real-time."(Quote 30, D1).
However, respondents did note that the rise in cognitive load may vary depending on the learning environment and how an MM teacher dashboard is integrated into that environment (e.g., mix of remote and in-person learning, single student use in regular classroom, several students using it at once, etc.).
To mitigate increased cognitive load associated with real-time usage, several respondents noted the value of implementing a feedback mechanism in the MM teacher dashboard to inform the teacher of appropriate actions to execute to remedy students' unfavorable learning states.

"[W]hen you have to manage a classroom, you know, to orchestrate the classroom, you need to teach. So, this may be way too much to do for a single person to manage. And if we really make teachers centred, we would probably remove all the unnecessary things and really focus on what matters in this moment
. So to making it useful for the teachers, we should really try to make more actionable recommendations.Perhaps have the MM teacher dashboard give the teacher one suggestion at a time on how to help students."(Quote31, D1).
The aforementioned quote demonstrates that though the respondents anticipate that teacher's cognitive load may increase due to MM teacher dashboard usage in various settings, respondents support the notion that simplifying the dashboard visuals, as well as including directive feedback functionality (e.g., action and order of actions) for the teacher to consider, may alleviate some cognitive burden experienced by the teacher.
4) Instructor Buy-In: Respondents also identified teachers' attitudes regarding the use of sensors to capture and record student data as a major challenge.Although only a few respondents themselves demonstrated a negative disposition toward augmenting their teaching practice with MMD, many respondents raised the possibility that other teachers may exhibit attitudes of data mistrust, skepticism of practical implementation, and resistance to learning new technologies (i.e., the MM teacher dashboard).
With respect to data mistrust, respondents questioned the MMD accuracy, its susceptibility to external factors, and raised concerns about taking the data at face value.
"We have to consider how the external factors, like social interaction, influence the student's learning state.But then it's really hard to know whether the metrics shown on the dashboard are due to the learning task itself, or perhaps like, from interactions between peers."(Quote 32, D2).
To mitigate MMD mistrust, many respondents stressed the importance of taking time to acclimatize to use of an MM teacher dashboard.For instance, D3 stated: "We shouldn't believe all the data we see.This type of system and MMD are very new to the children and teachers as a generation.We need time to get used to it.So it should be used for a few years to understand how it works,... like, its accuracy and so the teacher can trust in the system" (Quote 33, D3).
Alternatively, some respondents demonstrated confidence in the MM teacher dashboard's ability to deliver accurate information but skepticism toward the practical applications of using MMD "in the wild." "I am sure the technology works, but I am not convinced that it works in practice.I mean, the technology works properly, but I am unsure of its usefulness" (Quote 34, D4).
Speaking from over a decade of experience as a high school IT instructor and tech facilitator, T1 (a self-declared "early adopter of new technologies") emphasized the challenge of "demonstrat[ing] the value of how new technologies can improve learning" when they are introduced to new users (i.e., teachers).

"It's the transition from the early adopters to the mainstream teachers where you really have to show the value of how it [the MMD] will improve learning... A lot of teachers don't want to change their practice. They want to do the thing they're doing because that is what has worked for them.... And they won't change from that, because it's too much work. I know it sounds a little bit jaded, but it's the biggest hurdle we've come across." (Quote 35, T1).
The need to soften teacher's resistance to using MMD, by informing teachers on how a tool like the MM teacher dashboard, can deepen a teacher's awareness of their student's learning experience, and ease the teacher's role was echoed by several respondents.
"Teachers are set in their ways and not open to change...but their perceptions need to be changed.When it comes to new technologies, they need to open up to consider adoption... it's about improving teaching methods for the teacher as well, but a lot of time those new technologies are pushed away because teachers don't realise this...So it's important to inform them and show them the actual potential offered by the MMD, and how it might save their time."(Quote 36, D9).
"Teachers want proof.We want to know how this will improve our jobs, and make it easier for us to work with students and do assessment.So if you can show how this tool improves student learning and how it improves our jobs and helps facilitate student learning, that's when you've got a really good argument to bring sensor data into the classroom."(Quote 37, T1).

VI. DISCUSSION
This study investigates the challenges and advantages of integrating an MM teacher dashboard into pedagogical practice, through the lens of teachers and educational researchers.From the data analysis, we identified four overarching advantages: 1) enhanced learning state awareness; 2) enriched teacher-student discussion; 3) informed learning design; and 4) appropriately timed and allocated feedback, and four underlying challenges: 1) low technical ability and need for IT support; 2) insufficient educational data literacy; 3) imposed cognitive load; and 4) poor instructor buy-in, which are important for ed-tech designers and technology facilitators to consider when implementing and integrating MM teacher dashboard into the classroom.These advantages and concerns produce positive and negative influence on teachers' ability to coordinate and conduct learning activities (i.e., which includes lesson planning, execution, and post lesson reflection, and is referred to as "orchestration load" [60] when considered at the classroom level).In this section, we expand upon each theme and present a collection of empirically derived implications for design and practice drawn from the results.

A. MM Teacher Dashboard Advantages
Our findings showed that respondents recognized and appreciated the potential of MM teacher dashboard to augment the teacher-student relationship through deeper awareness of students' learning states, enhanced teacher-student interactions (e.g., inquiry and feedback delivery), and more conscientious learning design decision making.
With respect to heightened teacher awareness of students' learning states, an MM teacher dashboard may uncover opportunities for teacher-student inquiry and discussion on an individual student, group of students, or entire class level.Teachers and students could explore the learning experience together, using the MMD as an artifact to inspire conversations.This may afford students a more holistic exploration and understanding of their own learning experiences and offer them new insights into their own learning dispositions (i.e., habits, strengths, weaknesses, and adversities).This enriched awareness (for both teachers and students) may facilitate more meaningful teacher-student dialog, as well as contribute to students' improved understanding of the feedback they receive.In turn, involving students through MMD-led inquiry may facilitate student accountability and empower students through a strengthened sense of self-agency.However, we caution that although an MM teacher dashboard affords unprecedented transparency into students' learning states, it is important that teachers consider the MMD in combination with their own contextualized knowledge of each student and the information shared by each student.With appropriate rationalization, teachers can utilize the MM teacher dashboard to enhance their teaching.Therefore, instead of attempting to automate or outsource teaching, our findings propose that MM teacher dashboards should be used to empower teachers' decision making.
Teacher dashboards provide a means to monitor and evaluate the educational climate of classroom environments and identify student needs, thereby supporting teachers' decision making and feedback delivery processes [30], [61].Specifically, the MM teacher dashboard enables real-time monitoring of students' states at a class level, as shown on the ASV.In this way, the MM teacher dashboard helps teachers manage the learning environment in a better manner than without using a dashboard, thereby easing the orchestration load by providing "awareness of what is happening in the classroom" [60].Moreover, the data overview provided by the dashboard can inform on the appropriateness of the learning activity ("core activities" [60]), thereby offering support for real-time regulation of the task ("emergent activities" [60]).Hence, depending on the activity's design, a teacher may make modifications in real time based on the data shown.In this way, using an MM teacher dashboard supports flexibility.Both control and flexibility are design principles attributed to mitigating orchestration load [60].Building on this, to support delayed regulation (e.g., class-level data-driven decisions to change the upcoming lesson), future design of the MM teacher dashboard could adapt the RSV to visualize all students in a given session (similar to the ASV, but available after the learning activity has completed).Also, our findings showed that respondents appreciated the enhanced learning state awareness enabled by the use of the MM teacher dashboard (in particular its MMD-based notification system) as a directive to support teachers with appropriately timed and allocated feedback delivery.However, several respondents expressed the need for guidance on how to react to the MMD (i.e., what is the meaning of the different learning states expressed by students, and what actions should the teacher consider taking?), to which we offer two possible solutions.Previous studies have indicated the importance of both technology adoption and educational data literacy.Technology adoption refers to the process of incorporating new digital tools and technologies into the classroom to enhance the learning experience.The effective Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
use of technology requires teachers to have a deep understanding of all three knowledge domains (as per TPACK [62]), namely, content knowledge, pedagogical knowledge, and technological knowledge.Educational data literacy [59], on the other hand, refers to teachers' ability to sensemaking and use data for informed decisions about teaching strategies and student progress.From our findings, the introduction and efficient use of MM teacher dashboards requires increased teachers' data literacy competencies.Thus, initiatives to improve teachers' educational MMD literacy skills must be put into practice (discussed at length below).In addition, prior works [5], [21], [41] illuminate the use of MMD-driven feedback to facilitate learning.
Building on this, future MM teacher dashboard design may look to the implicit human-computer interaction (iHCI) paradigm for inspiration.This is when a system implicitly infers users' preferences and needs based on their behavior, rather than explicit communication [63], [64].For example, extending MM teacher dashboards from awareness platforms to teacher feedback recommendation systems by integrating MMD-based feedback actionables that address student learning states (e.g., offering positive reinforcement, or scaling the content difficulty) and are displayed to teachers via an MM teacher dashboard in real time.As such, our second suggestion is to extend MM teacher dashboards from awareness platforms to teacher feedback recommendation systems by integrating MMD-based feedback actionables that address unfavorable student learning states.Furthermore, and as suggested by respondents, such feedback could also be presented in a sequenced manner to help teachers decide which order to address student concerns.The recommendation for MMD-based priority sequenced feedback is crucial in scenarios where feedback delivery is time sensitive and a low frequency of support cues is required [65].However, similar to the case of MMD-led discussion and inquiry, and inline with the control design principle mentioned above, feedback suggestions must be considered in parallel with teachers' contextualized understanding and awareness of student/class on-goings, indicating that a teacher's decision must always supercede that of the system.

B. MM Teacher Dashboard Challenges
Shifting focus to the identified challenges, respondents stated that the sheer amount of MMD presented by the MM teacher dashboard may present difficulties (e.g., imposed cognitive load, time required to review the MMD, and unreasonably sophisticated educational data literacy skills).In addition, they added that the MM teacher dashboard would offer more value with respect to learning design, if a cumulative view of students' learning experience with longitudinal MMD trend identification is included.Thus, we offer two result-driven (and respondentsuggested) design implications, which may ease teachers' evaluation of the MMD when used in evidence-based customization of instruction and curriculum design.First, MM teacher dashboards should generate comprehensive custom reports that summarize students' progress over time (as seen in [21]).Second, they should include a view designated to communicating long-term data trends.These additions may be time savers as teacher could expeditiously access the MMD once a learning activity has concluded, assess the progress of students (e.g., determine if they are progressing through the learning material at the expected rate and identify learning patterns), reflect on, and revise learning decisions quickly.Also, the addition of a comprehensive report may allow teachers to review the MMD repeatedly and at their own pace (i.e., facilitating postlesson reflections).
Moreover, it is paramount that teachers are in support of technology-enhanced learning tools in order for them to achieve sustainable acceptance and classroom adoption [21].However, according to respondents, external and internal challenges related to technical ability and support, educational data literacy, imposed cognitive load, and instructor buy-in may hinder the widespread adoption of MM teacher dashboard into learning spaces.These concerns echo the works of Ertmer [66], who identifies first-order (e.g., technical ability and MMD literacy competency) and second-order (e.g., instructor buy-in) barriers as challenges to the integration of technological educational innovations, such as an MM teacher dashboard, into practice.Moreover, she argues that such challenges may be tightly coupled due to their "continual interactions" [66].For example, the amount of cognitive load (i.e., mental effort) required to utilize an MM teacher dashboard increases, as teachers' educational data literacy decreases.The contrapositive is also true (imposed cognitive load decreases as educational data literacy approaches fluency.)Thus, introducing ways to improve educational data literacy will also help mitigate cognitive-load-related issues (e.g., orchestration load) and vice versa.Similarly, with respect to instructor buy-in, respondents expressed concerns with data mistrust.This issue may also be addressed through improved educational MMD literacy.In light of these dependencies, we offer the following collection of design implications and process related measures to collectively address the respondent-identified challenges.
1) Demonstrating Evidence of Value: Teachers must accept, support, and be willing to actively engage with MM teacher dashboards in order for them to transition into learning environments.Our findings uncovered teachers' sense of skepticism toward practical implementation, resistance to learning new technologies, and data mistrust.Collectively, these lead to low teacher buy-in.These second-order barriers derive in part from teachers' inherent beliefs regarding how they envision teaching, learning, and knowledge acquisition to be [67] and are often difficult to detect, measure, and overcome.Consequently, they deter the integration of educational technologies [66].
However, empirical evidence holds much weight in this regard.Thus, in addition to formal pilot studies, empirical tests, and system revisions based on teachers input, demonstrating MM teacher dashboard's potential to enrich learning experiences through teacher peer-led sessions, which exemplify meaningful and effective MM teacher dashboard use cases, may encourage change in teachers' perspective and address these barriers [66].These demonstrations should simulate realistic learning experiences (e.g., be practiced in approximated scenarios, as opposed to learning the system's core functionality without real-classroom context) to address issues of skepticism regarding practical implementation.Inviting teachers to actively participate in discovering the capabilities of the MM teacher dashboard and their sensors encourages mastery of the involved technology and empowers teachers in their pedagogical practice.
Furthermore, to alleviate issues of data mistrust, teachers need access to important information about the data (e.g., what data are being collected, how they are analyzed) and space to grow trust in such a system, which necessitates time to familiarize, adjust to, and habituate using an MMD teacher dashboard, as well as for curriculum development purposes [66].Finally, instructor buy-in undoubtedly hinges on the ability of MM teacher dashboard to integrate smoothly into teachers' already established pedagogical practice [21].Design efforts (i.e., minimalism [60]) must ensure a low degree of complexity associated with using an MM teacher dashboard, meaning that dashboards (and their associated sensors) must be easy to use (i.e., multifaceted minimalism by simple interface design, easily understandable data, intuitive features) and quick to set up.
Parallel to this, the request for assistance with first-order barriers [66], such as device set up and on-going technical support, can be largely attributed to the fact that the devices (i.e., eye trackers, wrist bands, webcam, and motion sensors) used by our MM teacher dashboard for data collection are uncommon in learning spaces, and so teachers lack the relevant experience or necessary skills needed to set up, maintain, and troubleshoot the devices.However, previous research [66] suggests that such barriers can be reduced given additional resources, such as specialized training, support, and time to adjust.Thus, with respect to teacher's technical support and abilities, we offer the following respondent-inspired design guidelines and process suggestions: 2) Hands-on Device Training: Rather than learning from standard user manuals, teachers need comprehensive and practical training instructing on how to operate the devices.This includes learning proper device setup (e.g., helping students put on the devices and connecting the devices to an MM teacher dashboard), calibration (if required, i.e., eye trackers), operation, and troubleshooting common device related issues as they arise (e.g., devices incorrectly worn and devices disconnecting from the computer).Hands-on training would enhance teachers' technical competencies, confidence, and likely reduce (or eliminate) the need for tech support.In addition, such hands-on experience may demonstrate the plausibility of classroom usage, thus addressing concerns of practical implementation (i.e., instructor buy-in).Moreover, hands-on training should be a reoccurring endeavor to ensure that teachers maintain and continue to hone related technical competencies [67].
3) Independent, Agnostic, Plug-and-Play Device Connection: Our MM teacher dashboard is device independent, meaning that if a device type is not connected, the associated measurements are excluded from the charts view and the MM teacher dashboard continues to operate.Device independence simplifies the dashboard setup process, as the teacher may choose to connect only the devices associated with their desired measurements.It also minimizes the need to troubleshoot defective devices in real time, as they do not disrupt usage.In line with this, we suggest that MM teacher dashboards implement a plug-and-play device connection infrastructure to simplify the setup process.Moreover, employing a device-agnostic design approach (e.g., compatibility across heterogeneous device brands without requiring adaptations) may: 1) enable connections from less accurate but more affordable devices (e.g., common eye trackers used in online games or cameras that compute metrics' approximations); and 2) eliminate the need for device calibration (e.g., self-calibrating eye-tracking device [68]).In addition to lowering the financial barrier of device cost (which also contributes to skepticism of practical implementation), the aforementioned recommendations may reduce teachers' needs for ongoing technical support and high technical ability.
The amount of mental effort required to observe, understand, and interpret the extensive amount of data displayed by the MM teacher dashboard may jeopardize teachers' cognitive load, which has been used in part to measure orchestration load [69].These issues are further compounded by teachers' lack of knowledge regarding educational MMD.To address these interconnected concerns, we make the following suggestions.
4) Measurement and Notification Customization: Our findings indicated that respondents valued the MM teacher dashboard's high degree of customization (e.g., selectable metrics, resizable charts, adaptable layout, and disableable notifications), specifically with respect to simplifying the appearance and behavior of the dashboard by reducing the amount of information (MMD and notifications) shown.In addition, teachers' appreciation for customizable features was derived from their need to adapt to their diverse teaching profiles (e.g., student demographics and subjects) and dynamic teaching circumstances that a one-size-fits-all MM teacher dashboard may not accommodate.Based on this, we recommend designing MM teacher dashboards to be configurable with respect to which data to display, how to visualize it (e.g., line chart, numeric representation, etc.), as well as which notification features to enable.This will allow teachers to contextualize the MMD and adapt it to their instructional needs in ways that a one-size-fits-all design would not.Streamlining the dashboard's appearance and behavior enables teachers to create an interface and feature set with the teachers' desired level of minimalism, so they may focus their cognitive resources on the information that matters.Furthermore, we extend this customization to include notification features, which reduce the constant need for teachers to visually poll the MMD, but may be irrelevant or distracting in some circumstances.Revisiting the iHCI paradigm, for example, future design could include an adaptive user interface, where the MM teacher dashboard modifies its layout, functions, and features according to the teachers' prior behavior and context, may offer a more natural user experience, and enhance teacher's usability and orchestration experiences.

5) Educational MMD Training:
To properly analyze and interpret the dashboard visualizations, it is imperative that teachers have high MMD literacy competencies.This includes a generic understanding of what each measurement represents (e.g., cognitive load is the amount of mental effort expended during a learning activity), how manipulating the learning activity might influence a measurement (e.g., introducing an element of speed may increase students' cognitive load), the measurement's influence on students' learning behaviors (e.g., cognitive overload may attribute to an increase in incorrect answers), and the potential remedial actions to address the states of different measurements in the educational context.However, teachers Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
were unfamiliar with the measurements and possess low MMD literacy.To achieve the necessary educational MMD fluency, we must provide teachers with proper training on the measurements in use, a notion that was echoed loudly by respondents.Strengthening teachers' MMD literacy competencies may assuage the cognitive and orchestration load imposed by the dashboard, while increasing teachers' technical confidence.In addition, teachers possess contextualized knowledge of their students, which may not/cannot be reflected by the MMD (e.g., teacher may be aware of a student's issues at-home).It follows that insights derived strictly from MMD may be misleading if not considered in parallel with the teachers' personal knowledge.This furthers the importance of educational MMD training as it provides teachers with the knowledge required to confidently sensemake and contextually interpret the MMD.

C. Limitations
The findings of this article aim to identify the advantages and challenges that teachers encounter when using MM teacher dashboards.However, our findings are subject to certain limitations.For instance, because this study was conducted during the COVID pandemic, respondents were shown a comprehensive video of the MM teacher dashboard in action, but were not given the opportunity to use it themselves.Similarly, respondents were unable to try operating the devices in person (e.g., wearing and connecting to the MM teacher dashboard).Results may have been different if respondents had interacted with the MM teacher dashboard and devices.For example, respondents may have reflected differently on their own technical abilities after having hands-on experience.
Our participants came from different corners of the world.As such, they offered insights from heterogeneous teaching practices, which were undoubtedly influenced by cultural and societal norms.For example, several respondents from Canada reported that a common practice in the Canadian school system is for teachers to video record their students (via teacher's personal cell phone) and share these video clips with parents, in order to engage parents in their children's learning experience.On the other hand, two teachers from a European country took a strong stance against sharing students' data with the students' parents and claimed that the data belong to the students and providing it to their parents violates student privacy.This illustrates how cultural-societal norms played a role in participants' contrasting attitudes surrounding data privacy concerns, which may have influenced respondents' concerns regarding challenges (e.g., instructor buy-in).
With respect to reliability, the coding process was iterative with consensus meetings held between the coders, where the coding was discussed.Although we did not conduct any systematic process to assess the reliability of our coding (e.g., calculate Cohen's kappa or a similar index), the process followed provides a degree of reliability in terms of consistency and what Krippendorff [70] describes as reliability-"the degree to which members of a designated community concur on the readings, interpretations, responses to or uses of given texts or data."Finally, we recognize that this article represents a single short-term study, with findings determined from teachers' and educational researchers' first impressions, many of which are new to various aspects of the involved technology.Future work, including revisions of the MM teacher dashboard artifact and implementation of longitudinal studies, is needed to further investigate the challenges and opportunities of MM teacher dashboard.Also, investigating the use of an MM teacher dashboard in different contexts and settings (e.g., learning technologies, students' age, physical, online, and hybrid spaces) should be conducted to address the nuances of these different learning environments and portray a holistic picture of the challenges and opportunities of using MM teacher dashboards.

VII. CONCLUSION AND FUTURE WORK
In this article, we followed a grounded theory methodology to investigated the challenges and advantages encountered when using MM teacher dashboards in learning spaces.We motivated our RQ with relevant literature.Then, we presented the design of a novel MM teacher dashboard with notification features, which we used as a conceptual anchor for MMD visualizations during a series of semistructured interviews conducted with teachers and educational researchers.Our findings identify the emergence of four overarching advantages: increased learning state awareness, teacher-student inquiry, opportunities to inform learning design, and improved appropriateness of feedback allocation; and four challenges: technical setup and support, educational data literacy, imposed cognitive load, and instructor buy-in.The challenges identified by our findings should not be taken lightly.However, they may be mitigated with proper technological training, increased educational MMD literacy skills, and time to develop trust and mastery of the data.With this in mind, our study uncovers the unprecedented potential of MM teacher dashboards to inform, guide, and expedite teachers through numerous aspects of their teaching practice, thus supporting the teachers to perform targeted interactions and classroom orchestration actions [71].In the following, we provide implications for design and practice for MM teacher dashboards to facilitate teachers use.Future research should consider following an educational design research (EDR)/DBR approach [72] by inviting teachers as experts to participate alongside researchers, through iterative cycles of dashboard ideation, design, implementation, and evaluation, with the aim of improving the teachers' user experience and MM teacher dashboard adoption rates.EDR's ongoing teacher-researcher exchange may uncover teachers' various preferences and concerns (e.g., selection of important and reliable measurements, and adoption concerns), thus producing knowledge early enough to negate several potential challenges encountered when MM teacher dashboards are used "in the wild," such as those which respondents identified during the interview phases.This may help facilitate wide-scale integration of MM teacher dashboards.Finally, future research endeavors should focus on integrating actionable feedback for teacher consideration.Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

Fig. 1 .
Fig. 1.(a) Student is wearing eye-tracking glasses and a wristband as they interact with a digital learning activity.MMD is captured via the eye-tracking glasses and wristband worn by the student and video recordings web camera located at the top of the screen.(b) MM teacher dashboard GUI, streaming synchronized videos of the learning activity and the student video recording, and charts of the students' device data.

Fig. 2 .
Fig. 2. MM teacher dashboard start-up modes.Blue: data collection mode requires a student's live connection via the learner side.Yellow: replay mode in which the teacher can review data from a student's previous learning session.

Fig. 4 .
Fig. 4. Three different notification features of the MM teacher dashboard.Top left: the threshold breached notification displays a red dotted line, so the teacher may see when a measurement's data surpass the threshold.Bottom left: incorrectness line notification appears as a vertical red line when a student responds incorrectly.Right: student icon overlay notification puts a transparent red marker over a student's icon button to indicate that the student's data may warrant the teacher's attention.

Fig. 5 .
Fig. 5. Left: The MM teacher dashboard focused on Kris', a nonstruggling student's, data.A red overlay notification over Sam's icon button alerts the teacher that she is struggling.Right: The MM teacher dashboard shows Sam's data, after the teacher has switched to show her data by clicking on her icon button.

Fig. 8 .
Fig. 8. Question template used to guide the semistructured interviews with respondents.

TABLE III CHALLENGES
THAT MAY RESULTS FROM USE OF AN MM TEACHER DASHBOARD, AS IDENTIFIED BY RESPONDENTS Summaries of our findings for advantages and challenges are presented in Table and III, respectively.We use superscripts to denote a researcher (R), teacher (T), and dual role (D) participant, which follows the mapping of Table IV in Appendix A.