Belonging in Engineering for Black, Latinx, and Indigenous Students: Promising Results From an Educational Intervention in an Introductory Programming Course

Contribution: This study demonstrates the efficacy of an ecological belonging intervention in a first-year engineering programming course to increase belonging for Black, Latinx, and Indigenous (BLI) students and close academic equity gaps. Background: Introductory programming courses are often challenging for students and can shape belonging in engineering. BLI students may be particularly susceptible to interpreting struggle as confirmation that they do not belong in predominantly white spaces, which can negatively influence academic outcomes. Research Questions: “What are the effects of an ecological belonging intervention on BLI students’ feelings of belonging within their first-year engineering course?” and “What are the effects of an ecological belonging intervention on BLI students’ performance on a weekly computer programming assignment?” Methodology: The intervention was implemented with 691 students in Spring 2022 and was designed to normalize the struggle to address threats to belonging and close equity gaps in BLI students’ academic performance. A pre-/post-semester survey measuring belonging was analyzed using repeated-measures ANOVA, and pass/fail academic records were analyzed using logistic regression. Findings: The targeted belonging intervention for BLI engineering students can help to address issues of isolation and academic confidence that negatively impact individuals’ sense of belonging and academic performance.

Belonging in Engineering for Black, Latinx, and Indigenous Students: Promising Results From an Educational Intervention in an Introductory Programming Course

I. INTRODUCTION
C OMPUTER programming concepts have become essential to the engineering profession and are part of requisite courses for most degree fields [1], [2].However, the process of learning programming concepts is often challenging for students as it requires identifying the problem and key features, formulating a solution strategy, translating the plan into the appropriate syntax, applying logic correctly, and debugging errors within the code in order for a script to run successfully [3].Additionally, these skills are unique to programming [4], and students are often expected to learn these metacognitive skills alongside syntax and content [5].Because of these challenges, the first programming course engineering students take can be a key signal about their abilities to succeed in engineering and their feelings of belonging within their degree program [6], which in turn have been shown to predict motivation, engagement, and academic performance in STEM majors [7], [8], [9].
Alongside these general challenges students face in learning computer programming, students who are systematically excluded and marginalized face additional threats to their belonging above and beyond their peers [10], [11], [12].The exclusion and marginalization that Black, Latinx, and Indigenous (BLI) students experience is well documented in engineering [13], [14].The barriers BLI students experience include feelings of isolation, invisibility, lack of belonging, and imposter syndrome which can adversely affect BLI students' academic performance [15], [16], [17], [18], [19].BLI students are also uniquely affected by pervasive whiteness in engineering culture, particularly at Predominately White Institutions (PWIs), which account for the most common and largest engineering programs in the United States [20].BLI students report a variety of negative experiences, including unwelcoming classroom environments, imposter syndrome, macro-and micro-aggressions, systemic racism, and stereotype threat in their engineering experiences [21], [22].
Two factors, in particular, appear to motivate attrition from the engineering and programming fields: 1) stereotype threat and 2) lack of social belonging.As described by Steele and Aronson [23] and Spencer et al. [24], stereotype threat occurs when a member of a stereotyped group unconsciously devotes part of their working memory to behaving in ways that do not confirm negative stereotypes, imposing a significant cognitive burden on the student.In engineering programming contexts, "geek culture" can create dissonance between BLI students' salient identities and the stereotypical image of a programmer-a smart "techy" White or Asian man with poor social skills [25], [26], [27], [28].BLI students may face racial stereotypes implying that they are less competent in subjects and skills pertinent to engineering and programming, which can undermine their academic success and reduce their persistence in STEM fields [19], [29], [30], [31].These stereotypes, which are inconsistent with the role of a "good" engineer, force them to grapple with the exclusionary effects of systemic racism and often require self-reauthoring to conform within engineering spaces [17], [32].
Social belonging, "a sense of having positive relations with others," refers to feelings of being accepted, supported, connected and is often reduced by the experience of being marginalized [33, p. 1447].Having a strong sense of belonging is associated with retention, persistence, academic adjustment, and academic achievement [11], [18], [34], [35], [36], whereas belonging uncertainty detracts from students' ability to engage with and benefit from learning activities [37].Belonging differences by race and gender exist across STEM broadly and engineering in particular [38], [39], [40].These differences have been linked to lower self-efficacy [41], and greater barriers to success in engineering [19], [42].As a result of these factors and others, BLI students comprise just 16.5% of all bachelor's degree earners in engineering [43] and demonstrate higher rates of attrition from engineering and STEM overall [18], [44], [45].
Due to these layered threats to belonging, introductory engineering courses that include computer programming are a focal area for promoting equitable course outcomes.As BLI students continue to represent a larger share of the national populace [46], [47], higher education must address barriers that hinder opportunities by improving the sense of belonging among these groups.Several studies have tested interventions to address stereotypes and support belonging for BLI students generally [12], [33], [48], for women in engineering [49], and for BLI students in biology and women in engineeringfocused physics courses [37].Many of these interventions describe general experiences for all students in college and are administered individually.This intervention is specifically contextualized to the struggles students experience in the target course, is delivered during class, and reshapes the social environment.This focus on the interaction between individuals within the classroom environment (i.e., students, teaching assistants, and instructors) makes the intervention ecological in nature.The present study extends and refines earlier work on an ecological belonging intervention in physics and biology [37] to the context of an introductory engineering programming course.

A. Study Context, Purpose, and Research Questions
This study was conducted at a large, Midwest public R1 institution.Students interested in pursuing engineering degrees are admitted generally to the first-year engineering program and after a year of common science, mathematics, and engineering courses, they select and matriculate into one of 16 engineering degree programs.This study took place in Spring 2022 in a second semester, required first-year engineering course.This course focused on engineering decision making and data analysis using a common engineering tool, MATLAB.The institution is predominately White (54%), and BLI students make up 7% of the engineering undergraduate enrollment along with 16% Asian American, 4% multiracial, and 18% international students [20] (Note, the percentages reported have been rounded to the nearest whole percent to anonymize the institution).
The analysis of institutional data revealed a consistent equity gap in BLI students' performance in the course of 0.44 points on a 4.0 grade point average scale.This equity gap persisted across the previous four years and was the main reason for choosing this course for a belonging intervention.The same analysis did not reveal a gender equity gap.The purpose of this study was to test the efficacy of an ecological belonging intervention in this course context for BLI students.Six sections of the course participated in the study; three in a "business as usual" condition (i.e., control; n = 331) and three who received the ecological belonging intervention (n = 360).Changes in participants' sense of belonging and grades in a MATLAB assignment were compared across experimental groups (students who received the intervention and those who did not) and across race/ethnicity (BLI and White/Asian students) in a 2×2×2 repeated-measures ANOVA (RM-ANOVA) for the first research question and with logistic regression for the second.The following research questions were addressed.
RQ 1: What are the effects of an ecological belonging intervention on BLI student's feelings of belonging within their first-year engineering course?RQ 2: What are the effects of an ecological belonging intervention on BLI student's performance on a weekly computer programming assignment (using MATLAB software) in the course?

II. INTERVENTION FRAMEWORK
The ecological belonging intervention employed extends the work started in [37] into an engineering context.This baseform intervention was adapted to the context of a required first-year engineering programming course (refer to [50] for details on the process of contextualizing the intervention for this course).Binning's ecological approach was developed from prior social belonging interventions (refer to [51]), which taught students that adversity in college is both normal and surmountable.The ecological approach attempted to instill the same message, not just within individual students, but within the social system of the classroom.Namely, rather than being delivered in a lab setting as in prior work, the ecological approach targeted carefully selected populations-classrooms with specific, known academic equity gaps in performance by race/ethnicity, gender, or first-generation college student status.The intervention was delivered in one class session and was designed to establish a classroom norm that adversity in the course is common and normal, and these struggles tend to be temporary and surmountable with time and effort.
The ecological belonging intervention took approximately 40 min to administer and consisted of five activities.First, the instructor described common challenges that students have likely encountered and may be navigating (e.g., adjustment to college, time management, increased workload and challenge of course material, and finding internships/co-ops).Then, students individually wrote about one of their current struggles and considered how it might be addressed and overcome in the future.Next, these challenges were normalized by presenting a collection of stories from students who previously completed the course.These stories were designed to convey an arc of a challenge, the realization that a challenge is normal and surmountable with sustained effort, and the resolution of that challenge.After these stories were presented, students actively practiced and reinforced the message that struggle is normal and surmountable through discussion in groups of three to five students.They discuss two questions designed to give them exposure to diverse perspectives from their peers, allowing them to gain insight into their partners' knowledge and experiences [52].
The intervention was grounded in the theoretical considerations of Walton and Cohen's [12] concept of belonging uncertainty and Steele's [53] concept of stereotype threat.As defined by Walton and Cohen [12], belonging uncertainty occurs when members of stigmatized groups enter an environment in which they may not be able to develop firm social bonds due to stigma, resulting in hypervigilance, anxiety, and sensitivity to negative social dynamics.This psychological state manifests as an unconscious "broad-based hypothesis that 'people like me do not belong here"' [12, p. 83].The unconscious hypothesis primes the student's attention, drawing it to potentially threatening cues that would otherwise pass without notice (i.e., hypervigilance).
Research demonstrates that BLI students disproportionately experience belonging uncertainty [54], which (re)produces disparities in college performance [55].Specifically, belonging uncertainty modifies students' cognitive processing of academic adversity.Stigmatized students who experience academic challenges (like poor performance on an exam) are more likely to believe that their immutable characteristics are associated with their poor performance [56], [57] and consequently, they cannot become a member of a valued in-group in that academic context.
A second mechanism relating marginalized student identities to underperformance is stereotype threat.Stereotype threat is a risk of confirming negative stereotypes about one's racial, ethnic, gender, or cultural group, which can create a high cognitive load and reduce academic focus and performance [58].In order for stereotype threat to manifest, a student must believe (correctly or incorrectly) that others in their social environment hold a stereotype about their group (e.g., Black students are worse at engineering than White students [58]).When students are uncertain about their belonging in a context, like engineering, they are hypervigilant to cues from their environment that signal if they belong.This state requires a significant degree of attention and stress to regulate behavior in a manner that disconfirms the suspected stereotype [59], [60].This self-monitoring depletes their available working memory for learning [61] and can artificially downregulate task performance by an average of 0.22-0.64standard deviations [62].
Both stereotype threat and belonging uncertainty may be addressed via social belonging interventions.Specifically, the knowledge that others who are "like me" have succeeded within a context that primes belonging uncertainty and stereotype threat has shown promise in countering these marginalizing experiences [63], [64].Walton and Cohen [12] suggested that interventions that decouple cognitive attributions from race/ethnicity and emphasized that belonging uncertainty can come from feelings of challenge (a normal part of the learning process) and that these feelings can change over time are effective ways to address these unconscious states affecting student performance.This decoupling allows students to cognitively reframe their past and current experiences and indicates that feelings of lack of belonging are not diagnostic of their longterm belonging or success in the field.Resultantly, students are more psychosocially resistant to challenge as a signal that they do not belong and are more likely to experience challenge as something they can change.In this way, social belonging interventions, such as the ecological belonging intervention employed in this study [37], are theorized to disrupt the negative feedback loops of belonging uncertainty, stereotype threat, and consequent underperformance.

A. Procedures
The intervention took place during the first week of classes in Spring 2022, during the second meeting of the class.Students in the control condition experienced business as usual and were not exposed to the intervention.The presurvey was made available to students in all conditions during the first day of the class session and closed prior to the introduction of the intervention.The post-survey was opened during the last two weeks of class and closed prior to the beginning of finals week.All students were offered extra credit for completing the survey and were provided other extra credit opportunities if they did not wish to participate in this research study.As part of this survey, participants were asked about their sense of belonging, classroom norms, engagement, career aspirations, learning behaviors, and demographics, including racial and ethnic identity, gender identity, sexual orientation, family background, and dis/ability.Data on course performance, including the two major assignment types, individual MATLAB grades and a team-based project, were also collected to understand the impact of the intervention on closing the equity gap for BLI students in the course.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

B. Participants
A total of 641 students completed the pre-/post-surveys (92.7% response rate across conditions), 307 in the control sections and 334 in the experimental sections.Of these students, 1.7% identified as African American or Black, 5.5% as Latino/a/x, 0.8% as American Indian or Alaska Native, and 0.5% as Native Hawaiian or Pacific Islander.A total of 86.2% of students identified as White or Asian, and 29.1% preferred not to respond.The response options provided a multiselect option, so the percentages listed above may sum to more than 100% (refer to Table I for more information about race/ethnicity and gender).Participants who identified as Black/African American, American Indian, Mexican, Central American, Puerto Rican, Other Latino/a/x, and Native Hawaiian/Pacific Islander were grouped into a larger category (BLI, n = 52) for comparison against the non-BLI group (n = 589).These figures include both students who are U.S. citizens and international students.Although the research teams' preference would have been to model belonging separately for each BLI group, the study lacked the statistical power to do so and made the decision to group these students together within the analysis as, on the balance, assessing the potential efficacy of the intervention outweighed valid concerns [65].
Students were assigned to each section by the university registration process (i.e., not self-selected schedule).As such, this work represents a quasi-experimental research study design.Equivalency between demographic enrollment between control and experimental sections was verified with 5.2% of control sections BLI students, and 8.0% of experimental sections as BLI students.Instructors were recruited to participate in the study as either intervention sections or control sections.Three sections were recruited for treatment sections and three as intervention sections across four instructors.Each section had a maximum enrollment of 120 students and was similar in size.

C. Measures
The first outcome variable, a measure of belonging in the classroom domain, was measured using three items (a = 0.82) that asked participants to agree or disagree with the statements on a 4-point Likert scale: "I feel comfortable in engineering," "I feel I belong in engineering," and "I enjoy being in engineering."The second outcome variable, performance on a weekly MATLAB assignment, was measured using students' grades.The majority of students had high scores on this assignment, with the end-of-semester average above 96% for 70% of students.As a result, scores were dichotomized as pass (n = 561) or fail (n = 43; cutoff at 0.70, equivalent to the C-needed to pass the class).An analysis of overall grades and team-based project grades was also conducted and showed no statistically significant impact.

D. Analysis
To test the hypothesis that the intervention would be significantly correlated with an increase in the sense of belonging among BLI students, but would have no significant effect upon White and Asian students, the research team performed a 2×2×2 RM-ANOVA using the afex package in R [66].Due to the longitudinal structure of the data, time was used as the within-groups variable, while identification as a BLI or non-BLI student and experimental condition (intervention or control) were the between-groups variables.Q-Q plots and skewness/kurtosis tests were used to verify the normality of the dependent variable.To account for unequal sample sizes among groups, Type III sums of squares were used.To examine the effects of the intervention and race/ethnicity on MATLAB grade, a logistic regression was run using the glm() function in R on students dichotomized MATLAB grade (passing grade of C-or higher versus nonpassing grades).Students' grade on the class project was entered as a control, and multicollinearity was ruled out by examining VIF scores (all values below 4).
Both of these analyses were underpowered, owing to the low number of BLI engineering students.Previous work with highly underrepresented groups argues for the validity of such research and the need to consider how traditional statistical norms may hide or prevent the study of "small n" populations [67], [68].To this end, this study uses a higher alpha value cutoff of 0.10 to interpret the findings.This approach attempts to balance statistical considerations with practical and equity concerns for students of marginalized populations.

E. Team Positionality Statement
The research team of faculty, postdoctoral scholars, and graduate students included researchers from higher education, social psychology, and engineering education.One researcher has been engaged with the design and teaching of the course of study, which constitutes the research context.As Black, Latinx, and White scholars, these identities have influenced the teams' engagement with this research and decisions about measurement and the interpretation of results.The research team had regular discussions of their approach to research questions across disciplinary domains, including how the team frames marginalization in engineering and the potential positive and negative impacts of this work for BLI students.
In conducting this research and analysis, the research team has been cautious to interpret the results as promising while also acknowledging the limitation of a quantitative approach to understanding the individual experiences of students within the data and recognizing the need for qualitative and mixed methods research to support the findings described in this study.

A. Belonging × Race/Ethnicity × Treatment
The effects of the intervention were tested, as moderated by race/ethnicity, on students' belonging using a three-factor split-plot (one within-subjects factor and two between-subjects factors) RM-ANOVA.The within-subjects factor was time, which was measured at the start and finish of the semester.The between-subjects factors were racial self-identification as a BLI student or White and Asian student, and assignment to the treatment or control condition.Of the 641 students surveyed, 265 completed both phases of the survey and were used in the RM-ANOVA.There was not a significant amount of missing data (<5%), therefore data imputation was not used.No univariate outliers were detected.Because the within-subjects variable only had two levels, the assumption of sphericity was automatically met.A visual examination of the residual Q-Q plots demonstrated reasonable normality with moderate S-shaped lifting from the central line.
No statistically significant main effects were detected, but a statistically significant three-way interaction between BLI status, treatment condition, and time was detected (F(1,261) = 2.99, p = 0.085, η 2 p = 0.01).This result indicates that BLI students in the treatment group had a statistically significant difference in pre/post-belonging scores compared to the BLI students in the control group.Effect sizes for significant effects were small even with power well below that standard for the detection of effects in social scientific research.The full results of the RM-ANOVA are presented in Table II and visualized in Fig. 1.
All student groups began the semester with average belonging scores above 3.0 on a 5.0 scale.White and Asian

B. MATLAB Grade × Race/Ethnicity × Treatment
Of the 641 students surveyed, 38 did not have grades for the required assignments (MATLAB and class project) or the required demographic information (n = 603, 94%) and were excluded from this model.The effects of the intervention were tested, as moderated by race/ethnicity, on students' dichotomized MATLAB grades (pass versus fail) using logistic regression and controlling for the class project grade (which was most of the remaining score in the course and was completed in teams of 3 to 4).The model was significant, χ 2 (4, N = 599) = 28.25,p < 0.001, explained 11% (Nagelkerke R ) of the variance in MATLAB grade, and correctly predicted 92.4% of cases.The odds of a passing grade increased by 80% (OR = 5.01, 95% CI [0.25, 3.13]) for BLI students in the intervention condition.BLI students in the intervention condition had an average MATLAB grade (M = 0.91, SE = 0.05) 0.25 points higher than BLI students in the control condition (M = 0.66, SE = 0.04), while MATLAB grades held steady for White and Asian students across conditions (control: M = 0.93, SE = 0.02; intervention: M = 0.93, SE = 0.02 (refer to Table III for full results and a visualization of the interaction in Fig. 2).

V. DISCUSSION AND IMPLICATIONS
The ecological belonging intervention had promising, but small effects, in addressing declines in BLI student belonging over the semester.White and Asian students entering the course had an average sense of belonging of 3.17 on a 5-point scale, which did not decline significantly over the semester, regardless of treatment or control assignment.In contrast, BLI students had an incoming sense of belonging scores like White and Asian students (3.17 on a 5-point scale) but experienced a statistically significant decline in belonging over the Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.course of the semester in the control condition but not in the treatment condition (2.90 and 3.14, respectively).A general decline may not be unexpected as students normalize the role of being an engineer and the difficulty of learning new concepts.However, the decline in sense of belonging may have been higher for BLI in the control section than their White and Asian peers because of the additional stereotype and social belonging present [21], [22].
Based on these results, the ecological belonging intervention is hypothesized to work as a protective factor for this sense of belonging loss.Conveying the message that adversity is normal, surmountable, widely experienced, and manifests in different ways for all students can reshape the context of struggle for students in engineering courses that may threaten students' developing sense of self and their potential to succeed [37].This message can support a reframing of students' interpretation of common struggle within this gateway programming course.Instead of struggle as a signal to BLI students that they may not belong in engineering, which erodes social belonging, struggle can be reframed as something that is not predicated upon students' minoritized status.This reframing of struggle can also address stereotype threat that is primed in STEM contexts for BLI.
Significant research has supported the connection between stereotype threat and increased cognitive load reducing academic performance [61], [69], [70], [71], [72].This study also provided evidence that the sense of belonging may be an important intermediary in addressing equity gaps in student performances where negative stereotypes exist about a group, an area that had not been directly tested previously.The intervention also appeared to close the academic equity gaps for BLI students on individual MATLAB programming assignments in the course.However, this intervention did not close the academic equity gap completely; the team-based course project still showed differences in academic performance, which indicates a need to further probe the team dynamics and equity of that project on students.Other research has shown the experience of students in diverse teams can have negative impacts on students through microaggressions, everyday experiences of racism, and teaming behaviors that emphasize engineering products over inclusive practices [73], [74], [75].These marginalizing experiences may create additional barriers for addressing stereotype threat and belonging uncertainty within the team project.
These results have implications for both research and practice in engineering education.It is important to acknowledge that student outcomes in introductory engineering programs are not a direct result of student ability (a deficit-based approach to considering differences in academic performance).No data support the existence of differences in student abilities by control and treatment groups in this study, nor does admissions data indicate significant differences between groups.Instead, evidence suggests that the classroom environment is a key feature that interferes with students' abilities to achieve their full potential.Consequently, it is essential that engineering education research addresses students' sense of belonging as an intermediate factor in addressing equity gaps in student learning and academic performance.

VI. LIMITATIONS AND FUTURE WORK
The above results should be interpreted with caution and interpreted as promising rather than conclusive because the data analyzed are based on a pilot study with small numbers of BLI students.The results indicate promising statistical significance in two of the three-way interactions: 1) Belonging × Race/Ethnicity × Treatment and 2) MATLAB Grade × Race/Ethnicity × Treatment.However, the impacts on closing the overall academic equity gap were not found.The intervention did not address academic equity gaps in the team project that was a large portion of the grade for the course in the latter half of the semester.Still, the results for the MATLAB programming assignments are promising in demonstrating that the ecological belonging intervention employed Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
has efficacy in addressing at least some of the barriers to success in programming for BLI student engineers.
Additionally, due to the small sample sizes, student responses across Black, Latino/a/x, and Indigenous students had to be grouped together and contrasted with the majority of White and Asian students.These results may obscure the intervention's effectiveness for particular groups and only provide a partial characterization of students' experiences [76].This analytic approach could also reify a suboptimal normative comparison of marginalized groups to the "default" of White and Asian men in engineering [77].Finally, small sample sizes prevented this research from examining the effects of the intervention at different intersections of gender and race/ethnicity.Future work will employ larger samples to replicate this study within the same course.This additional data will provide more robust, sensitive results for intersectional investigation.This work will also investigate how the intervention implementation integrity (perceived authenticity of the faculty facilitator and ability to convey the core message that adversity is normal and surmountable) supports or undermines the effectiveness of the intervention in reshaping classroom norms to better support belonging and address the effects of stereotype threat.This future work will also explore how faculty mindsets about diversity, equity, and inclusion; training; and integration of the intervention implementation affect the effectiveness of this intervention.

VII. CONCLUSION
This study investigated the effectiveness of an ecological belonging intervention customized to a required first-year engineering programming course.To the research team's knowledge, this study was the first to focus specifically on first-year BLI engineering students' experiences following an ecological belonging intervention.The results indicated that the intervention acted as a protective psychosociological mechanism for BLI students' belonging during the semester and that the intervention closed academic equity gaps in individual course assignments.The results of this work emphasized that the theories of social belonging and stereotype threat are important aspects of student experiences to address in practice and that even small changes in the classroom norms, environment, and faculty approaches can support more equitable outcomes for students.The results of this study provide a loweffort, effective intervention to address long-standing equity gaps in engineering courses, which can have downstream effects on retention, engagement, and who ultimately becomes an engineer.As the U.S. becomes increasingly diverse, it is crucial to ensure that BLI students feel they belong in the engineering field and have a welcoming space in which to contribute to creative problem solving.

Fig. 1 .
Fig. 1.Pre-and post-effects of intervention on majority and BLI students.

TABLE II THREE
-FACTOR SPLIT-PLOT ANOVA RESULTS

TABLE III LOGISTIC
REGRESSION ANALYSIS OF TREATMENT AND RACE/ETHNICITY ON MATLAB GRADES Fig. 2. Effects of intervention on majority and BLI students' MATLAB grades.