Preparing Middle Schoolers for a Machine Learning-Enabled Future Through Design-Oriented Pedagogy

Machine learning (ML) literacy has recently been identified as one of critical skills students need to succeed as future creators and innovators. While the significance of introducing ML basics at kindergarten to twelfth grade (K-12) levels is increasingly acknowledged, there is limited research that focuses specifically on collaborative design of ML applications with middle school students. We posit that engaging young children to co-invent and make concrete prototypes improves their ideas, encourages them to become active participants, and allows them to establish the implications of the technology in their everyday lives. In order to lay the foundation for middle school ML education, we collaboratively designed and prototyped ML applications with 43 eighth grade students (ages 11 to 14) in a Nigerian school. The ideas generated by the students indicate that they began to identify the applicability of ML to their daily lives and as a solution to a plethora of societal challenges. This study provides learners’ input and preliminary insights into approaches that could be adopted to promote ML within the compulsory level of education in an African setting. The research contributes to the limited body of knowledge available on effectively teaching ML to young learners using design-oriented pedagogy, especially in the context of an emerging country.


I. INTRODUCTION
Machine learning (ML) is the science of teaching computers to mimic human abilities and autonomously improve their abilities over time by inputting data in the form of observations and real-world interactions [1]. Over the years, ML has evolved from stand-alone solutions to embedded solutions and has produced technologies integrated into readily available intelligent devices such as mobile phones, tablets, and smartwatches, among others. ML technologies such as Google Assistant, Siri, and Cortana provide evidence The associate editor coordinating the review of this manuscript and approving it for publication was John Mitchell . of advancement in this emerging field. This advancement has become fundamentally ingrained in our society, with impacts ranging from work to daily activities. Applications of ML include image processing, face detection, which is readily available in most smartphones; algorithms applied by various search engines [2], [3]; and email intelligence, which allows intelligent bots to reply to emails, remind users to attach files, and manage work tasks.
Despite ML's ubiquitous impact and societal effects, it has received minimal attention at the K-12 level until recently [4], especially in terms of teaching the basics of ML. According to Carney et al. [5], the K-12 computer science curriculum globally tends to center on rule-based programming, algorithms, and digital literacy [6]. Children's lives are significantly impacted by ML technologies such as Siri, Alexa, Netflix recommender, and Google Assistant, but they lack knowledge of how ML works, thereby deepening digital inequality [2]. For a generation growing up in a datadriven, ML-powered society, understanding the potency of ML is pivotal to being an active participant in society [5], [7], particularly bearing in mind the gap between the fast-paced market demand for artificial intelligence (AI) professionals and the limited number of professionals to fit that demand. Therefore, it is essential to inspire the interest of the younger generation and equip them with the adequate knowledge, skills, and ethical thinking needed to fulfill the demands of a society steeped in ML [8].
Recently, teaching ML at the K-12 level has received attention from researchers, scientists, and technology advocates around the world. The growing attempts to teach ML basics at this level can be seen in the development of resources and approaches to demystify the concept to children of all ages [8], [9], [10], [11], [12]. Despite these advances, ML in K-12 education has not been the subject of sufficient empirical research to understand how the concept can be effectively introduced to students. In particular, research is limited concerning how researchers, educators, and students can collaboratively design resources to popularize ML within the compulsory level of education. According to Schaper and Pares [13], the focus of co-designing is not only to work together towards a product output, but to enhance the insight, design skills, and reflective thinking abilities of the children involved. Based on the premise that engaging children in the design process allows them to express themselves through mediums that are familiar to them and encourages creativity, we hypothesize that such an approach could be valuable to introduce ML basics. In addition, [14] state that ''current educational practices do not sufficiently foster young peoples' creative competencies, owing to strong focus on the transmission of pre-given content knowledge and routine procedures to students.'' As a result, our study aims to bring co-invention culture to Nigerian schools. Research into teaching ML at the middle school level has been plagued by various challenges, most importantly the challenge of simplifying a multifaceted computer science concept to a level comprehensible by children with diverse cultural backgrounds and mental capacities [7].
In Nigeria, the computer studies curriculum presently focuses on digital literacy and information communication technology [15], greatly neglecting how young learners, especially middle school students, perceive, interact, and understand societal changes brought about by ML technologies. In an attempt to prepare children for an AI-enabled future and lay the foundation for middle school ML education, we collaboratively designed and prototyped ML applications with Nigerian middle school students. This research aims to support the students in developing skills and becoming creators and co-designers of their ML applications.
This study also aims to make middle school students aware of the ML technologies they interact with, develop data agency, and become critical players in the present datadriven society. We conducted the pedagogical intervention with middle school students in a Nigerian school to explore students' perceptions of ML as well as the process and applications they could develop through an ML co-creation process.
The following are the research questions considered in this research: • What ML activities did the students carry out, including the ideas generated and applications initiated during the design process?
• How did the students perceive and explain the process of co-designing ML apps? Our research builds on prior studies of children as designers of ML apps [7], and we adopted the research design process of these studies. We believe utilizing a design approach can provide valuable insight in a context with a pronounced digital gap and limited co-design culture. While Agbo et al. [16] and Rinnert et al. [17] have implemented codesign pedagogy in Nigeria, the focus has been on students at higher education institutions (HEIs) and in-service teachers respectively. The present research represents some of the first co-design interventions with middle school students in Nigeria, specifically in relation to ML. In addition, as the teaching of ML continues to be implemented in K-12 schools globally, this study considers an African context to increase awareness, develop learners' literacy, and gather perspectives to understand suitable approaches to effectively promote ML in Nigerian schools. The dearth of studies on the use of the co-design process to introduce ML at the K-12 levels, particularly in the African context, makes this work necessary.
This paper introduces ML and the study's aims in Section I. In Section II, the literature review and conceptual framework are presented. Section III describes the tools used in the study, while Section IV describes the methodology and related approaches used in the research. Section V details our findings, including excerpts of our discussions with the study participants. In Section VI, we discuss our findings in relation to the existing literature. We conclude with limitations and recommendations for future research in Section VII.

II. LITERATURE REVIEW A. TEACHING MACHINE LEARNING IN K-12 SCHOOLS
The application of technology has become pervasive at all educational levels, including K-12. Many of these technologies include AI and ML applications, which involve concepts often not entirely familiar to the students using them. However, ML has been taught for a while at HEIs, both in standard classes and for independent studies. Research studies have also been conducted concerning the teaching of ML in universities. For instance, Martí-nez-Tenor et al. [18] taught ML in robotics interactively. Additionally, a mixture VOLUME 11, 2023 of instructive and constructivist methodology was adopted to apply ML methods with Lego®. Mindstorms robots in a cognitive robotics course. However, teaching ML at levels below HEIs is still unusual.
Introducing ML at K-12 levels helps young learners gain basic technological literacy that will assist them in understanding and appreciating the world they live in [19]. According to Pena [20], early introductions to the fundamental workings of ML can support children's understanding of the external world and their capacity to solve related problems. Pena adds that such early interventions can help prepare children for educational leadership positions and furnish them with basic innovation skills that they can apply early in their careers. Through playful learning combined with personalized guidance in an expert-apprentice environment, students can explore their interests and become strong advocates for their own learning [21].
Studies have emerged presenting workshops and initiatives that have attempted to introduce the basic principles of ML to K-12 learners. In Brazil, an introductory course was developed to teach ML in alignment with the K-12 guidelines for AI lessons [22]. The workshop culminated with the development of an image recognition model for recycling trash using GTM. In Finland, a similar workshop applied design-oriented pedagogy. Hands-on exploration of ML-based technologies helped K-12 learners formulate different design ideas to provide solutions to the challenges of daily life [23]. In Germany, Scheidt and Pulver [24] created Any-Cubes, a prototype toy that combines deep-learningbased image classification with machine-to-machine communication through the MQTT protocol, allowing children to delve into and understand ML and Internet of Things technology in a fun and intuitive manner. According to the findings, children were able to move beyond the original application and begin to use the cubes in novel ways. Other studies focusing on teaching ML in K-12 classrooms include [25], [26], [27], and [28]. All of these initiatives revealed how the entry barrier to technical computational practices can be lowered and the computational literacy of children can be improved.

B. PEDAGOGICAL FRAMEWORK
This research adopts a constructivist, design-oriented pedagogical approach. The constructivist theory argues that, for learning to be fulfilling, the learning process should be student-centered, and there should be social interaction among peers participating in the construction of knowledge [29]. It also frames the role of experts as that of guardians in the assimilation and construction of knowledge [30], [31]. Papert argues that all builders need building materials, and with adequate materials for the construction of knowledge, when children are allowed to act as designers, builders, and programmers of artifacts, they can learn essential skills in computational thinking, making, and action in the world [32], [33], [34]. Papert further stressed that novel ideas can be born when children are made active participants in designing and building artifacts they can ponder and share with others [35]. Instead of transmitting existing content, design-oriented learning centers activities in which students connect with the world around them through artifacts they co-create [36]. As shown in Figure 1, Vartiainen [36] further asserts that design-oriented learning involves participatory learning, technology, and co-design.
It is worth noting that there are various models of instruction, some of which are based on a fully prescribed, build-a-thing task-based approach. Many of these models are made up of laborious scripted tasks and instructions from an instructor who teaches students how to re-discover some unifying principle [33]. These models of instruction arrange curriculums into sequenced tasks to complete, and the assessment of learners is often based on their ability to recall what they were taught [37]. Schmidt et al. [38] argue that educational institutions that adopt these models of learning only train their students to perform tasks that are being automated by computers and robots.
In contrast to these fully prescribed models of learning, design-oriented pedagogy centers on open-ended, real-life learning tasks [36]. In these types of tasks, emphasis is on the fact that there are no ''right'' or ''wrong'' answers; instead, students are allowed to come up with different answers they consider meaningful to the questions and challenges they encounter [39]. This approach leads students to conceptually understand the question or challenge as well as the answers they discover [40].
In design-oriented pedagogy, students are guided to understanding by sharing various tasks involved in the ideation and creation of authentic artifacts. This approach plays a vital role 39778 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
in mediating learning and collaboration [41]. As described by Papert, design-oriented pedagogy emphasizes a playful, creative process in which students use various materials (e.g., drawings and programs) along with conceptual and external representations to communicate with each other [42]. These processes of creating external representations, coupled with assisted refinement of the ideas that students generate, can increase understanding of the complex and abstract content domain being represented [43].
Furthermore, design-oriented pedagogy encourages students to engage in problem-solving activities that are closely related to those of expert communities [39], [42], [44], [45]. While carrying out these activities, students are expected to engage in various group discussions, sharing what they know, making decisions together, assigning roles, and bearing the burdens of responsibility to advance their ideas and encourage teamwork [44], [46]. To fully harness expert-like practices, students also need to work together with experts [44]. Guided participation or cognitive apprenticeships are an important aspect of this type of collaboration. In such collaborations, the expert provides encouragement along with the means and metacognitive support for the students to solve problems or complete tasks that they do not possess the necessary skills to complete independently [47], [48].
In the course of such expert-novice collaborations, students tend to acquire the culture, values, and skills that characterize the expert community [49]. Through intuition rather than explicit teaching, the students acquire expertise and strategies that experts have mastered and internalized with years of practice [50]. With this perspective, students may also engage in computational practices and discover applications of science and engineering to real-world problems. Such collaborations may also motivate students, reorient their thinking around computational design or what it means to be a computer scientist, and reduce entry barriers to core computing practices.
Although there are several studies on design-oriented approaches to learning and how the framework can be applied in various pedagogical contexts, only recently has research been conducted on ways in which children can become designers and creators of ML applications. Some of these studies present ML workshops and projects in which children generated ideas and then designed and built models of those ideas [23], built models of their physical activity [51], pictured intelligent devices and toys of the future [52], and investigated the workings of object recognition through drawings [53]. All of these studies have highlighted the benefits of making students active participants in learning processes that position them as subjects and teachers of ML systems, not just as the teaching objects of conventional pedagogies.

C. DEMOCRATIZING ML
The study of AI has been restricted to the expert domain in the past. Despite its widespread applications in daily activities, most users of AI feel intimidated by it [54]. In recent years, continuous efforts have been made to reduce the entry barrier into the world of AI for nontechnical learners. The democratization of AI technologies has also been helping to lower this barrier and has paved the way for children to communicate with machines in diverse ways, not only through popular rule-based programs such as Scratch, but also through natural language, gestures, and other ML-enabling technologies and interfaces [2], [52], [55]. Such technologies include Google Teachable Machine [56], Machine Learning for Kids [19], and Learning Machine Learning [57]. In this study, we engaged learners using Google Teachable Machine and Machine Learning for Kids.

III. ML TOOLS ADOPTED IN THIS STUDY
Several ML tools have been utilized to teach students at the K-12 levels, as shown in Sanusi et al.'s [58] study. This section briefly describes the two ML platforms adopted in this study.

A. GOOGLE TEACHABLE MACHINE
Google Teachable Machine (GTM) is a web-based tool developed by Google Creative Lab that offers a no-code and lowcode approach to modeling and building ML applications. GTM connects advanced neural network algorithms with a simple, user-friendly graphical user interface that enables anyone, even children, to make fast and easy ML models, allowing middle school learners to explore the working principles and basics of ML.
GTM was initially launched in 2017 [56]. The first version supports only image classification projects using a webcam as a data collection tool. In 2019, GTM 2.0 was released, which extended its project support categories to include image, audio, and human poses. The Images tool trains an ML model to classify images. The training data can be captured by a webcam or uploaded to the GTM environment from the user's device. The Audio tool similarly trains a model to classify different sounds, such as different people talking, or distinguish between background noise and an intended audio sound. Finally, the Poses tool employs PoseNet for real-time human pose estimation and identifies different bodily movements. The ML models produced in these project categories can be exported in TensorFlow and Javascript and can be embedded in web apps, mobile apps, among others.
GTM has gained traction recently with various software engineers and researchers. For example, Payne and her research team at MIT used GTM as part of an open-source curriculum that applied a hands-on learning model to teach middle school students the basics of AI [55]. In addition, GTM has been used to introduce students across grade levels, from early childhood education to high school, to AI and ML concepts such as image recognition and pattern recognition [59], [60]. VOLUME 11, 2023

B. MACHINE LEARNING FOR KIDS
Machine Learning for Kids (ML4Kids) is a web-based educational tool specifically designed to teach young learners and novices about ML. ML4Kids helps children create projects and build applications with the ML models they train. This tool was developed by Dale Lane using APIs from IBM Watson. It provides an easy-to-learn, guided environment for training ML models to recognize text, numbers, images, or sounds. To train an image model, the training data can be captured from a webcam or a web address or drawn by the user. For an audio project, training data can only be captured from the user's device microphone. To train a model that recognizes text and numbers, training data can be captured using the keyboard.
ML4Kids has furthered its goal of introducing coding to children by adding ML models trained to coding platforms such as Scratch and App Inventor. Furthermore, trained models can be exported in Python for developers to use in Python-enabled systems. Since its official release in 2017, ML4Kids has been adopted for ML teaching by thousands of schools, coding clubs, and families worldwide [19].

IV. METHODOLOGY
This study employed a design-based research (DBR) methodology to achieve results that create functional knowledge, have a direct impact, and are not dissociated from the problems and issues that arise in practical use [61]. The DBR method is essential to understanding how, when, and why educational innovations, such as the ML intervention this study deployed, work in practice [62]. Primarily, this framework helps set a premise to develop technological tools, learning environments, pedagogical models, social practices, and design principles that direct, inform, and enhance both practice and research in the study of learning [63]. For the pedagogical design applied in this study, we focused on the design processes involved and emphasized that tasks carried out had no right or wrong answers. This approach was intended to advance general knowledge about ML education and derive novel conclusions in a naturalistic environment to support and enhance the understanding of learning [64]. As Schoenfeld [65] explains, ''the products of well-conducted design experiments are improved interventions and improved understanding of the processes that result in their productiveness.'' The intervention of this study is of immense benefit to the research community, especially due to its African context. The pedagogical design process as applied in this study is shown in Figure 2.
Design-oriented pedagogy is based on three pillars: participatory learning, technological infrastructure, and codevelopment as a powerful social innovation that underlines pedagogical principles [45]. Based on these values, we provided learning activities, technologies, and pedagogical processes that engaged students as users, testers, informants, design partners, and meaning-makers in cutting-edge research practices and development experiments [66], [67]. Doing so utilized an emerging trend that emphasizes collaborative learning and a creative and robust design process. Our choice of approach is backed by metadesign, a framework that creates a social and technical infrastructure that enables end-users to be actively involved in the iterative process of system development [68]. A distinct advantage of design-oriented pedagogy methods is that they also allow students' perspectives to be considered when introducing AI and ML to K-12 classrooms [7]. Considering that students quickly identify mismatches in interventions, such methods and collaboration with researchers may also increase the chance that the designed intervention will turn out to be practical and relevant for students [69].

A. CONTEXT AND PARTICIPANT ANALYSIS
This case study is centered on teaching ML to middle school students without any prior knowledge of ML. Generally, middle school students in Nigeria are familiar with computers, mobile phones, the internet, and other technological devices and can use them with ease [70]. They possess appropriate communication skills such as writing, reading, listening, and speaking. They can, therefore, reason, grasp the knowledge taught, and reflect on their experiences. The study was carried out in a Nigerian military school. The participants in the study were 43 eighth grade students (ages [11][12][13][14] and their computer science teacher. The design and implementation of the study was planned jointly with the school to fit into the school's activities.
The hands-on workshop sessions were also designed to fulfill the objectives [22] specified in Table 1. These objectives were adapted from the K-12 Guidelines for Artificial Intelligence, particularly Big Idea 3 -Learning [2] and guidelines for AI literacy [71].
As shown in Figure 2, to achieve these objectives, preworkshop sessions and three workshops of about two to three hours each were conducted. At the first pre-workshop session, we conducted an orienting task that we called the white paper task. In this task, learners were asked to express their prior knowledge of ML and their thoughts on teaching a computer by drawing or writing on a white piece of paper. We emphasized that there were no right or wrong answers, and all thoughts and ideas were appreciated. A presurvey was also administered to capture their perceptions and background knowledge of ML.
In Workshop 1, after we reviewed student submissions from the white paper task and pre-survey, we presented an introduction to ML and how it is present in our everyday lives, including some useful ML products such as smartphone face recognition, Google Assistant, and Apple's Siri. We further asked students to imagine the future of these AI agents. The students were grouped to interactively discuss different situations in which they thought machines  could learn, including the kind of information that could be collected and what it could be used for. We recorded this discussion session for research purposes. In the last session of Workshop 1, we introduced the GTM and ML4Kids platforms and provided guidance on how to use these platforms. To conclude the first workshop, students were given individual homework to search for and identify everyday problems that could be solved using ML-powered technologies.
Prior to the commencement of the second workshop, we selected feasible ideas that could be prototyped into web-based ML applications. We divided Workshop 2 into two sessions (Workshop 2A and Workshop 2B) that were conducted on different days. In Workshop 2A, pupils engaged with GTM, while in Workshop 2B, they engaged with ML4Kids. After classifying the students into co-design teams based on their interests, we gave them a design template to record the functions of the app, the kind of data collected (image, sound, or pose), and different categories the model should recognize. The students then collected data and trained their models on GTM and ML4Kids while we provided support when needed. To conclude the session, the participants were asked to present their developing thoughts and sketch the design interface for their own apps.
Finally, Workshop 3 focused on testing the prototype apps designed before the session. The students were also briefed on the app development process and given feedback regarding their design plans. We concluded the session by interviewing each student team to gather their perspectives and reflections on the design process, including how their understanding of ML concepts had evolved.

B. DATA COLLECTION
Based on an earlier study, data were collected throughout the phases of the research study [23]. These data included the white paper task at the beginning of the project, the students' group conversations, their design ideas and drawings, the co-designed apps, and the focus group interviews at the end of the design experiment. Building on the previous empirical research, our interview questions included general and follow-up questions addressing each student's background, their conception of ML, the collaborative design procedure (how they conceived their app ideas, their thoughts about the designing and learning procedures, the teamwork structure, challenges in how the app operates, and new concepts), and data agency (how they felt about using MLbased automation in their everyday lives). Additionally, the average time each group took to complete each subdesign task was recorded. Furthermore, we closely observed the students during the activities and took photographs of some activities carried out throughout the design process. During the workshops, various qualitative data (audio-taped interviews, images of activities, observations, and notes) were collected.

C. DATA ANALYSIS
Our data analysis relied on a qualitative content analysis [72], and we specifically adopted the approach of Vartiainen et al. [23]. We started with the students' drawings and descriptions of their thoughts about ML (through the white paper task), analyzing the kinds of design ideas initiated and the rationales behind the ideas. We carefully examined students' accounts of their experiences with the main stages of the co-design along with their results. In addition, we observed how the student ideas improved throughout the developing process of the design phases. We examined the design idea descriptions, the use or basis of the application in question, and the model type to be adopted (sound, image, or poses). We further examined the improvements that students identified as necessary while testing their applications. Finally, the audio-taped interviews and group discussions were transcribed, and excerpts were used to report the students' reflections on the process.

D. ETHICAL CONSIDERATIONS
Using appropriate ethical principles to protect humans who participate in qualitative research is of utmost importance in a research study [73]. In this study, ethical concerns associated with the data collected during the workshops, both through interviews and surveys for demographic characteristics, were given due consideration. Before the pre-workshop session, the students were given a consent form for their parents to sign. At the pre-workshop, we retrieved the consent forms. After retrieving the signed consent forms, we gave the students an assent form to fill out and further explained to them that participation in the program was voluntary. We also emphasized that they could opt out at any phase of the study. We gave copies of the signed forms to the students. Throughout this paper, the anonymity and confidentiality of the participants are preserved by giving each student a pseudonym and concealing other forms of identity in the data presentation.

V. RESULTS
This section presents in detail the proceedings of the workshops, the activities carried out, and our findings in the course of the workshops.
• RQ1. What ML activities did the students carry out, including the ideas generated and applications initiated during the design process?.

A. CONTEXTUALIZATION AND EXPLORATION
The analysis of the white paper task revealed that most of the students had no prior experience or knowledge of ML or datadriven design. Typically, students' descriptions and illustrations (as depicted in Figure 3) included writings closely related to what they had been taught in elementary computer studies and data processing classes, as well as depictions of smartphones, computers, humanoid graphics, and other internet symbols. There were only a few indicators of ML procedures or concepts. Since the pre-task was performed at home, it is probable that some of the students received assistance, particularly since we considered some responses to likely go beyond the knowledge of the students themselves. For example, one student drew an illustration of deep learning, while another drew a computer used in a hospital for data collection. Our assumptions were confirmed during the interviews; most students stated that they had no prior experience with or knowledge of ML, and some revealed that their elder siblings assisted them in the task. The white paper assignment, which is typical of an expert-novice study, served both research and learning purposes by prompting students to communicate their previous knowledge through writing or artwork.
Workshop 1 began with a lecture presentation to introduce ML and its presence in everyday life. This lecture achieved our first learning objective (LO1). During the presentation, we demonstrated how Google Assistant and Siri work to give students a practical sense of the topic under discussion. We also invited questions from the students to get feedback on their level of understanding and to shed light on areas that required clarification. The students were later assigned into groups and asked to discuss what they had learned during the lecture as well as situations in which they thought they had encountered ML and how they could use ML, especially in their daily lives. We challenged the students to identify everyday problems they face and develop solutions to these problems. The group conversations were monitored for research purposes, and we asked the students similar questions during the interview sessions. These conversations and interviews showed that most of the students had at some point heard of or utilized ML-based apps such as YouTube, Facebook, Snapchat, and the smartphone facial recognition unlock feature.
In the first workshop, we divided the students into eight groups, with four groups interacting with GTM and four groups interacting with ML4Kids. With the support of the researchers, students were able to explore how image recognition, voice recognition, and pose recognition work by training their ML models. During this stage, we emphasized playful learning and exploration without support. Students were able to explore the image project, audio project, and pose project on GTM. For example, some students trained a model to recognize various emotions and moods through their facial gestures, while others taught the machine to recognize various sign language alphabets, as seen in Figure 4. We concluded the workshop with an individual assignment to search for everyday problems and generate ML ideas to solve these problems.
We identified some instances that indicated students'have begun to recognize the basic principles of ML algorithm such as classification and its application in real world. We believe they understood the concept of classification through their interaction with GTM and ML4kids where they explored image recognition using their faces to ascertain if the learning tools recognizes them differently. Voice recognition was also experimented with various sounds and pose recognition was performed using body gestures. The use of these approaches has been adopted in earlier studies to introduce classification and bias to young learners [7], [74]. Some feedback from how the students think about ML before the first workshop and the discussions within the activities showed conceptual changes that emerge during the activities. For instance, before the first session, students only identified computer, graphics, and internet related symbols as ML while during the conversations before the end of the session, they began to refer to ML-based applications they use and encountered daily. The students also began to develop their understanding of some ML vocabulary such as model, training data, and testing data as they engage with the ML tools. 39782 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.

B. IDEA GENERATION AND DEVELOPMENT
The goal of the individual assignment given at the end of Workshop 1 was to stimulate critical thinking, encourage students to brainstorm design concepts, and support the young learners in creating bridges between their formal and informal learning [45]. The students' ideas included applications for school security and utility (e.g., a gate pass application and a noisemaker detector), financial services (e.g., a currency converter), audio identification (e.g., a solfa music note translator), home automation (e.g., an automatic door opener and an image recognition burglar alarm system), and many more, some of which were not feasible (e.g., a system that diagnoses a patient without the help of a doctor). An attendance tracker (a system that marks students' attendance using the faces of students in class) was another innovative idea the students thought of. The idea of attendance tracker shows that the young learners were beginning to imagine a truancy monitoring system to curb the growing problem of truancy in Nigeria [75]. The design of such a system could be a panacea to the increasing number of truant children in the country. The quote below was from the student that presented the idea of noisemaker detector.
It was my idea. I thought it would be useful in different ways. . . apart from you being in a class and people are making noise, and it reveals the people that make noise, it can also be used in to detect a familiar voice at various places, such as homes or workplace.
In contrast to the white paper task in which students displayed no experience with or knowledge of ML, the review of participants' ideas shows the benefits of a playful and informal hands-on exploration of GTM and ML4Kids, after which students exhibited knowledge of and experience with ML. Students' multi-modal interactions with GTM and ML4Kids contributed to these design ideas and spurred them to reflect on daily activities and situations through a new lens. Also, we observed that adults solving assignments for young learners can inhibit their ability to think, ideate, and communicate their ideas. We assume this position because some of the students could not explain their responses to the white paper task, which suggests that they might have been helped by family members or guardians, since it was a takehome task.
Having laid a background of what ML is in the first session and had students engaged with ML related activities, they had benefited from the knowledge that inspires them to brainstorm ideas to address societal challenges. The ideas presented by the students suggests that the intervention has great influence on their thoughts if juxtaposed with preconceptions. We assume this position based on how they could not even distinguish what ML applications are VOLUME 11, 2023 and where they are applied before the commencement of the sessions. Developing students understanding to generate ideas on how ML can be used for social good is key to preparing and motivating them for further learning of ML.

C. IDEAS TO EXPLICATED DATASETS AND DESIGN INTERFACE
After thorough analysis and consideration, we selected four student ideas (the currency converter application, the gate pass application, the noisemaker detector application, and the musical instrument identifier application) to prototype as web-based ML apps. We assisted the students in reengineering some ideas to make them feasible, considering their level of expertise, the ML technologies available, and the timeframe. The four ideas were implemented on both platforms -GTM and ML4KIDS. The collection of data and model training were completed during Workshop 2, which was conducted in two phases, Workshop 2A and Workshop 2B. This workshop achieved Learning Objective 2 (LO2).
During each of the two phases of Workshop 2, students were asked to discuss the operation of the application, the nature of data, and how to collect data in a guided environment to better articulate the idea for the ML application. The students trained various models according to their group's design idea. The currency converter and gate pass were based on the image tools in GTM and ML4Kids. The currency converter recognized dollar notes and converted them to their Nigerian naira equivalent. In comparison, the gate pass automated gate access to the school through facial recognition. The data for training the currency converter was collected from the internet through the Google search engine, while the students' faces were used for training the gate pass. Finally, the noisemaker detector and musical instrument identifier were based on the sound tools in GTM and ML4Kids. The students recorded their voices to train the noisemaker detector model and downloaded sounds of different instruments from the internet to train the musical instrument identifier model.
The workshop was fully hands-on as students trained their models to produce their desired results. We monitored the students as they brought their ideas to life and had fun in the process. A few critical ML concepts were explored in an efficient and contextualized manner. For instance, since ML involves learning from data, we emphasized the centrality of data and the importance of training as the students trained their models and explored their ideas through the ML platforms. Responses from students during the final interviews highlighted meaningful moments from making and testing these applications: Interviewer The testing session provided valuable feedback for the students. The participants could instantly get feedback on the implementation of their design idea and the performance score of their application, i.e., they could test the quality of their data. Although they could see their design in action, it did not always work as expected. This gave them practical experience with how data is vital to getting accurate results in ML models and also helped to achieve Learning Objectives 3 (LO3) and 4 (LO4). The students' emerging understanding of the influence of data on ML algorithms and applications was revealed in the conversation we had with Kunle and Somadina.
Interviewer Also, the GTM and ML4KIDS platforms did not work as the students expected, and on some occasions, navigating the platforms required assistance from the experts and facilitators in the classroom. We asked the students if they requested help at any point during the workshops. The excerpt below is from the conversation between the interviewer and Bolanle: Interviewer: Was there any situation where you needed help in the course of modeling this application?
Bolanle: Yes Interviewer: In what situation was that and why? Bolanle: When we were putting the data into classes. We needed clarification as to how much data we needed and number of classes we should use to train our model (Excerpt from the group creating the currency converter using GTM) The clarification that was sought about the amount of data required to train a model provided us the opportunity to educate the students that the more the data, the better the model will perform optimally. After the hands-on session, the participants were asked to draw their ideas for the interfaces of the ML apps they had created. Many students struggled to externalize their thoughts on this topic due to the poor culture of sketching and drawing in the traditional  educational system. Few groups were able to develop an interface diagram relating to their design ideas, and only some of these interface designs were feasible, as illustrated in Figure 5 and 6. Nevertheless, these illustrations indicated that there was a considerable increase in students' understanding of application development. Based on these neural network models, we prototyped three web-based ML applications.
The way students explicates their design ideas and interface shows increase in their understanding of how ML applications functions. The students were aware of the nature and the kind of data they needed for their proposed solutions. They specifically understood the implications of training data and importance of use of variety of data sets and large amount of data.
• RQ2. How do the students view and explain the process of co-designing ML apps?.

D. APPLICATIONS TESTING AND REFLECTION ON THE PROCESS
We started the third and final workshop by detailing the app development process. Alongside the models trained by the students, we designed and integrated a graphical user inter-face using the Angular Material JavaScript library. We also gave feedback on the performance of the models trained by the students, explained what guided decisions on the application development, discussed the constraints, and later gave students a 1 URL for the site of their application. On the URL, as indicated below, we bundled three models into one website: the currency converter model, the gate pass model, and the musical instrument identifier model. The navigation bar has three tabs where students can navigate through each of the applications. The first tab hosts the currency converter application, which uses the device camera as an input source. It captures in real time the value of the dollar note displayed and provides the equivalent naira value corresponding to that day's exchange rate. The exchange rate data is culled from a third-party service provider using REST API technology. The application displays no dollar note found if it is unable to recognize the image captured. On the second tab, the gate pass application captures students' faces in real time. It identifies the student and displays his or her name to make the school gate open automatically. Finally, the third tab hosts the musical instrument identifier. This application requires the device's microphone for input. It records and identifies a sound played in real time and displays the name of the instrument being played.
To conclude our presentation, we briefed the students on the possibility of improving the web application with more functionalities in the future if they so desire. The students went on to test their applications ( Figure 7) and presented them to their peers and teachers as well as to scientists.
To wrap up the project, we interviewed each group. In this session, we questioned students about the design process, its functionality, and the application. Furthermore, we asked about moments that stood out for them and their thoughts about the team co-design process. Their perspective on teamwork can be seen in this excerpt from one of the interviews: Interviewer: How does working together as a group helped in developing your project?
Evans: Yes, we were able to share ideas on how to build an ML application Toks: Working together in a group helps me perform better Interviewer: What would you say are the advantages of working as a group?
Evans:.Groupwork bring about shared responsibility and allows brainstorming. More so, if there is a mistake in group work, there is a possibility that one person will detect where the mistake lies in contrast to working alone without someone to collaborate with.
Jide: We are good at different things, working in a group helps us to complement one another which was what helped us in the training process of our application The students' perspectives about working as a team confirmed that providing tasks and activities that enables collaboration among students can promote learning of ML. This is consistent with existing study that proposed collaborative approach should be encouraged among young learners to effectively teach AI in Nigerian schools [8].
While sharing their perspectives on how the aim of their project was achieved, students were able to explain using some ML-related terms and concepts. For instance, students said, ''my model worked after inputting several data,'' and, ''better training is required to get my app work well.'' This vocabulary suggests that the students were able to demonstrate some level of data-driven reasoning and design after the workshops.
Finally, we asked them what their takeaways from the entire process were. We had the following conversations with the group that co-designed the noisemaker detector application using GTM and the group that developed the gate pass application using GTM: Interviewer Overall, all lessons that the students learned, ranging from collaborative skills to data-driven reasoning capabilities and essential digital skills, are vital to their development in the 21st century. The students displayed confidence and enthusiasm in taking the knowledge they had acquired to the next stage by teaching others and doing more work with the platforms independently. This attitude was reflected in our discussion with Melvin.
Interviewer: Is there any other thing you would like to add? Any questions?
Melvin: There should be more ways we can make this thing better. . . I will introduce how computers learn to my friends and siblings. I will use Teachable Machine and ML4Kids applications at home.
(Excerpt from the group that developed gate pass using GTM) The reflection at the end of the sessions is an indication that students learn about the concepts of ML (e.g., classification algorithm and bias) as well as societal implications which 39786 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.  makes a connection with two of the five AI4K12 big ideas [2]. The two ideas include: Computers can learn from data (Big Idea #3) and AI applications can impact society in both positive and negative ways (Big Idea #5). The students were able to demonstrate understanding of ML concepts through their experiences with the co-design process. This is in tandem with constructivism approach which focuses on how students create (construct) knowledge from their own experience [76]. Based on the premise that constructivism enriches the students thought processes and engagement on design process, we believe our approach provide changes in their understanding of ML in comparison to their prior experience. Figure 8 provides the summary of our findings.

VI. DISCUSSION
ML has grown to become an integral part of our daily activities. Hence, it is important for children who grow up using ML-enabled devices to understand the working principles of ML. Empirical findings have begun to emerge in literature exploring pedagogical activities that support young learners in becoming inventors, designers, and developers of MLbased apps. Such studies include [23], who demonstrates an important step towards understanding how to support middle school students in becoming co-designers and creators of their own ML applications. However, the findings presented in Vartiainen et al.'s [23] study can be considered limited in terms of geographical location and equal access to new technology. Besides, since ''the extent to which studies can be replicated in a field is one of the most important standards for reliability'' [77], our study focuses on a different context: eighth grade students in Nigeria.
ML literacy is now considered relevant and valuable for young learners regardless of age and grade level. Drawing on the prior definition of AI literacy by Long and Margeko [71], we conceptualize ML literacy as a set of competencies that enables individuals to evaluate ML technologies; communicate and collaborate effectively with ML and use and understand the societal effect of ML. One way of ensuring equitable access to ML is by devising ways to involve students in the design process and relate their ideas to societal issues. We have reported on the collaborative design process of ML applications with middle school students in a Nigerian school. The findings of the present study indicate that generating ideas and designing ML applications with middle school students significantly increased their understanding of ML and its societal implications. These improvements to students' understanding seem to stem from the effectiveness of the co-invention process and the creation of concrete prototypes based on the students' ideas [14], [78]. Additionally, the approach used provided students with the opportunity to address real-world challenges and envision how ML could improve society.
Our findings is consistent with past research which adopted similar approach in a developed region [23]. While both studies showed the effectiveness of engaging young learners with co-design process, we observed the solutions they were thinking of stem from their social context. Below, we further discuss our findings in relation to the research questions.
Our first research question was, ''What ML activities did the students carry out, including the ideas generated and applications initiated during the design process?'' The analysis of the students' ML design ideas revealed that children with no prior knowledge of ML can generate valuable ideas that address everyday problems. The young learners involved in our study utilized voice recognition, image recognition, PoseNet, and gestures through ML applications to identify challenges that could be solved, specifically relating to finance, security, and education. Analyzing the ML application ideas the students put forward (e.g., the gate pass and the currency converter) during the ideation stage further reinforces the need to empower young children with ML skills.
In practical terms, the currency converter is very relevant for the Nigerian system, as ''the rate of exchange of the Nigerian naira to the US dollar has always been a critical part of the management of the Nigerian economy from the early 1970s when Nigeria changed its currency from pounds and pence to naira and kobo'' [79]. The gap between the official and parallel market exchange rates of the naira keeps widening as citizens exchange in outrageous rates. There are double standards regarding the exchange of currency. It is worth noting that young children are already exploring solutions to curb the numerous challenges citizens face in doing business and completing transactions. The gate pass application demonstrates another way in which children are already concerned about how to make people's lives more secure. Nigeria is faced with a myriad of security issues, ranging from armed bandits to terrorists who invade schools and abduct pupils and staff members. The students were clearly thinking about how to protect schools and allow learning to continue by preventing unwanted visitors from entering school premises. The ideas generated by these students show that they have begun to identify how ML can be applied to their daily lives and solve societal challenges. We were able to achieve the learning objectives of the workshops by developing the students' conceptual and procedural understanding of ML during the codesign process. Evidence of achieving the learning objective began with students understanding how ML is present and useful in their everyday lives and advanced to labeling data for training purposes, exploring how data influences ML algorithms, and examining how training impacts ML models.
In regard to our second research question, which addressed the students' perceptions of the design process, the findings indicate that the children were quite appreciative of the opportunity to invent and design ML applications. The students showed great enthusiasm and now believe that they can become builders and owners of ML applications in the future. The co-design process also generated curiosity about ML among the students. By introducing ML ideas with an emphasis on the applicability of ML to students' lives, the workshops fostered students' interest in ML and related fields.
The study revealed that although the society in which children live directly impacts their reasoning and idea generation [80], playful learning and interactions with technological tools can assist them in creatively imagining possibilities beyond their current experiences and can strengthen learners' creative abilities [23], [81]. The playful exploration of ML and learning platforms such as GTM and ML4Kids assisted the learners in creating solutions to problems beyond the scope of their prior experiences. These interactions with multimodal technologies helped reduce the entry barriers for engaging in some of the core practices of computer science [23] and supported the children in a tangible, personalized exploration of abstract ML concepts. These findings further corroborate the purpose and importance of a social and technological environment that supports an interest-driven exploration of ML [23].
Our study suggests that a co-design pedagogy could be effective in teaching ML to middle school students. More studies are required to validate these findings, especially in the African context. Finally, this study provides learners' input and preliminary insights into approaches that could be adopted to promote ML within the compulsory level of education in an African setting. As our society becomes more sophisticated and data-driven, it is crucial to support young learners in developing computational fluency and data-driven reasoning [34], [82]. These skills will enable them to fully participate in society and develop a more complete understanding of the world they live in. This study contributes to the limited body of knowledge available on how to effectively teach ML to young learners using a design process, especially in the context of an emerging country.

VII. LIMITATIONS AND FUTURE RESEARCH
The limitations of this study were the timeframe of the workshops, the technological tools available, and limited data. We were unable to implement the graphical interfaces designed by the children and instead produced a basic application interface. Nonetheless, the study attempted to achieve its learning objectives and answered its research questions. We observed during interviews that once equipped with basic computing competencies, the children were able to give better and more informed explanations of their experiences with ML. Despite the limitations listed above, the study revealed that fundamental concepts and procedures of ML design need not be exclusive to experts or HEI learners; middle school students can also participate. Considering the paucity of research on teaching ML in K-12 classrooms, this study has demonstrated ways in which children can develop ML literacy, actively contribute to ML designs, and understand the applications of ML in their daily lives. This research is crucial as the role of ML continues to surge in governance, business, and citizenship [83]. This study strengthened the data agency of young learners and piqued their interest in becoming responsible designers and makers in the age of ML. Looking ahead, we propose further empirical and co-design research into making children actual developers of ML applications and contributors to developing the technological tools used in teaching ML in K-12 classrooms. Besides co-designing with children, introducing foundational ML algorithms to middle school students is also important. This goal can be achieved by developing curricula and instructional materials characterized by hands-on activities, innovative technology, and intuitive examples to facilitate student learning and understanding of ML basics [10]. Additionally, future work could further consider how students from different regions process learning resources in order to understand how the uniqueness of each social context can be leveraged to best introduce ML to middle schoolers [84].
ISMAILA TEMITAYO SANUSI is currently an early stage Researcher with the School of Computing, University of Eastern Finland. He is active in research with papers presented at international conferences and has published in academic journals. His research interests include technology education for K-12, educational technology, computing education, technical and vocational education and training (TVET), and ICT for development. His current research interests include artificial intelligence and machine learning education for young learners.
JOSEPH OLAMIDE OMIDIORA received the B.Tech. degree (Hons.) in computer science from the Modibbo Adama University of Technology, Yola, Nigeria. He is currently pursuing the joint master's degree (Erasmus Mundus) in GrEen NetworkIng and cLoud computing (GENIAL). He is currently an Erasmus Scholar with the Faculté des Science et Technologies, Université de Lorraine. His research interests include machine learning, data science, cloud computing, and the IoT. He shares a passion for multidisciplinary research focusing on the intertwining of the IoT, machine learning, and cloud computing.
SOLOMON SUNDAY OYELERE received the B.Tech. degree (Hons.) in computer science from the Federal University of Technology, Yola, Nigeria, the M.Sc. degree (Research) in computer and systems engineering from the Ilmenau University of Technology, Germany, and the Ph.D. degree in computer science from the University of Eastern Finland, Joensuu, Finland. He is currently an Associate Professor with the Luleå University of Technology, Sweden. His research interests include mobile and context-aware computing, smart learning environments, and pervasive and interactive systems. His current research interests include developing smart technology and games to support education and healthcare.
HENRIIKKA VARTIAINEN has worked as a responsible researcher in several multidisciplinary projects focusing on, for example, technology education, co-design in school context, designoriented pedagogy, and 21st skills. She is currently a Senior Researcher and a University Lecturer with the School of Applied Educational Science and Teacher Education, University of Eastern Finland (UEF). Her current research interests include learning machine learning through co-design and on the ways to support children's data agency. Her work on designoriented pedagogy has received the Doctoral Dissertation Award, in 2014, by the Finnish Educational Research Association (FERA) and the Young Researcher Award of UEF, in 2015.
JARKKO SUHONEN received the M.Sc. and Ph.D. degrees in computer science from the University of Joensuu, Finland, in 2000 and 2005, respectively. He is currently the Research Manager with the School of Computing, University of Eastern Finland. He has published over 120 peer-reviewed articles in scientific journals, conferences, workshops, and chapters of books. His research interests include online and blended learning, design science research in educational technology, computing education, and ICT for development. He has acted as a reviewer in several scientific journals and international conferences.