Introduction

Spurred by federal policies, data-driven decision-making (DDDM) has become a dominant practice in schools. DDDM focuses on identifying a problem, seeking a solution via the use of data, examining the consequences of the decision, and determining next steps (Mandinach & Jackson, 2012). This process helps schools and teachers identify “achievement gaps” and their potential remedies. Valuing and using data to make influential decisions about curriculum, instruction, policies, and practices within schools may seem like commonsense. However, the reality of the application of data-driven decision-making in schools has proven to be deeply problematic. Depending on its application, DDDM can promote deficit thinking about students and families by framing each step of the decision-making process as objective and neutral, rather than deeply situated within a nexus of historical and contemporary forces (Garner et al., 2017; Horn et al., 2015). Given its ubiquity in schools, it is essential to study how educators might harness data-driven decision-making in ways that reframe data and position their use for identifying school practices and policies that (re)produce various forms of inequities (Bowles & Gintis, 2011; Collins, 2009).

Over the past two years, the need for critical data-driven decision-making by educators has become even more visibly pressing. The confluence of a pandemic and calls for racial reckonings in all areas of society have spurred new kinds of conversations and questions among educators and has mandated new lenses through which to use data (Bertrand & Marsh, 2021). Rather than use DDDM to solely identify achievement gaps that demand quick remediation fixes for individual students, we posit that data analysis can also highlight systemic opportunity gaps. This expansion of DDDM to what we call CDDDM (critical data-driven decision making) means including a wider range of data than DDDM typically values, acknowledging the existence of systemic inequities manifesting in schools, and school personnel committing to effecting change where needed, both inside and outside of one’s classroom (Dodman et al., 2021a; Dodman et al., 2021b). This is a tall, but necessary, order for teachers and other school personnel who are often directed to use data, but who are not typically encouraged to ask critical questions about it (Braaten et al., 2017; Garner et al., 2017).

As scholars have more urgently begun to call for greater attention to equity as a driver of data use, we utilized a critical perspective of data to design a professional development intervention to aid educators in strengthening their data use for equity orientations and skills. This meant conceptualizing data use for equity as the integration of data literacy and equity literacy and framing the work by inquiry. Guided in design by effective professional development elements and with an eye toward understanding the intervention’s potential as a model, we engaged with a group of teachers and administrators in one elementary school over the course of a year to study the questions: (1) How does professional learning related to data use for equity affect educators’ awareness, understanding, commitment, and actions related to addressing inequity in their classrooms and school?, and (2) What are educators’ perspectives of the professional development model and their experience? To develop their data use for equity, participants conducted a School and Classroom Equity Audit (collection and analysis of school and classroom level equity data), investigated and acted on revealed inequities, and simultaneously engaged in learning related to culturally relevant and responsive instruction.

Conceptual framework

DDDM scholars have recently called for DDDM with an explicit focus on equity, rather than equity only serving as a potential byproduct of data use (Bertrand & Marsh, 2021; Datnow & Park, 2018). This requires altering, questioning, and deepening educators’ understanding of both data and equity. We have previously published on the necessity of engaging a critical orientation of data-driven decision making (CDDDM) (Dodman et al., 2021a) that interrogates the sociopolitical context in which teaching and learning occur and in which data are created. Unlike the instructional improvement frameworks with foundations in corporate management that underlie much research in data use (Datnow & Park, 2018), the theories informing CDDDM rely upon a conception of teaching rooted in teachers taking an inquiry stance and adopting the role of change agent to work towards a more just, equitable world.

Data can be powerful catalysts for identifying educational inequities and prompting educators to revisit their assumptions regarding teaching and learning (Dodman et al., 2019). However, data are also typically perceived as objective and neutral. Without encouraging critical self-reflection steeped in an understanding of implicit bias and structural oppression, teachers’ use of data may fail to address well-documented causes of achievement gaps and underrepresentation of minoritized groups in areas that contribute to students’ school success (gifted programming, for example). Within the critical DDDM cycle, educators must be prompted to consider the ways in which various economic and social forces such as racism, sexism, or classism can and do influence what problems they identify, what data they collect, how they interpret those data, and what solutions they entertain.

CDDDM makes the interrogation of inequitable conditions explicit. The framework centers the practices, policies, and initiatives of the school itself as potential engines of inequity and does not hesitate to interrogate the deficit assumptions inherent in many educational reforms (Patel, 2018). It is an inquiry stance that asks teachers to make visible whose knowledge is most (and least) valued, whose power (and marginalization) is reinforced by how schools and classrooms are organized, and whose voices are included (and excluded) in decision making at the school. Schools hold great possibility as sites of liberation for students. Because liberatory conditions do not just manifest themselves, educators need preparation and professional development in both data and equity literacies to adopt CDDDM as a working model. We refer to this intersection of data and equity literacies as Data Use for Equity (DUE) and position its development as necessary to, and developed during, engagement in critical data-driven decision making. See Fig. 1.

Fig. 1
figure 1

Data use for equity and CDDDM

Full size image

Data literacy

DDDM in education has become expected practice across the globe (Hoogland et al., 2016; Levin & Datnow, 2012; Schildkamp & Kuiper, 2010). However, DDDM has its critics and its implementation challenges, including the formatting of data, accessibility, perceptions of validity, self-efficacy regarding data, and data analysis training for teachers (Dunn et al., 2013; Kerr et al., 2006). Scholars warn that as educators are directed to use data, in the absence of high-quality data and technical assistance, “data may become misinformation or lead to invalid inferences” (Marsh et al., 2006, p. 5). In an educational accountability environment driven by data, widespread misunderstanding and misuse are problematic and, rather than disrupting inequitable conditions, can reinforce racist, classist, ableist, and sexist policies and practices. Bertrand and Marsh (2015) found that when teachers focused on supposedly “stable” characteristics of students in the context of data use, it led them to hold lower expectations for their multilingual students and those identified with a disability. Similarly, Datnow & Park (2018) described teachers using narrow sources of assessment data to confirm already held assumptions about students’ lives outside of school. Considering the accountability context in which data use occurred, Bertrand and Marsh promoted the potential benefit of “moving beyond a focus on individual teachers to a consideration of the broader social and structural forces that shape teachers’ actions” (p. 889).

Consequently, scholars of DDDM are moving away from narrow definitions of data use characterized by accountability (Datnow et al., 2017; Mandinach & Schildkamp, 2021) and are beginning to consider data and their uses more broadly. Data literacy refers to a teacher’s ability to gather, analyze, and interpret all types of data to make instructional decisions that support learning (Mandinach & Gummer, 2016). These data may include assessment data, student information, or systematically collected evidence from observations and from communication with parents and stakeholders. Increasingly, states are requiring educators and administrators to be data literate. Yet, at the same time they are called on to be data literate, there is evidence that teachers face challenges in their understandings of and skills with data, and that they also face challenges in recognizing inequitable schooling conditions. As a result of being asked to engage in data use for accountability versus data use for equity (Braaten et al., 2017), many educators continue to underestimate the degree to which achievement and opportunity gaps (Carter & Welner, 2013; Ladson-Billings, 2006) are present in their schools and districts and often avoid discussing the structural causes of inequity, instead attributing test score inequities to students’ home lives or their genetics (Gorski, 2016; Pollock, 2004; Rodriguez, 2012; Skrla et al., 2004). Such attitudes, expectations, and behaviors persist, despite the use of data (Gannon-Slater et al., 2017).

Mandinach and Gummer’s (2016) conception of data literacy suggests that, to be data literate, teachers must be able to “think deeply about the problem at hand while using the inquiry cycle to use data to inform a decision” (p. 372). Inquiry is an important addition to the DDDM literature. However, research in inquiry, on school and teacher level DDDM, and the authors’ own experiences engaging teachers with data via inquiry demonstrate that asking questions and thinking deeply can easily be done without problematizing expectations, assumptions, or values. In other words, being data literate may or may not include a distinct and critical focus on equity. When it does not, teachers and schools will continue to use data as a tool of social reproduction rather than as a tool for societal transformation, and they will continue to position achievement gaps as individual failures rather than systemic outcomes. In addition to thinking deeply, teachers need to be thinking critically. An updated perspective of data literacy has been offered by Mandinach and Warner (2021) that infuses a whole-child perspective of data use—culturally responsive data literacy (CRDL). CRDL is complementary to data use for equity in its attention to multiple and varied forms of data. This CRDL update complements the data use for equity framework that we propose, which considers the whole system while also considering the whole child.

Equity literacy

Equitable outcomes and experiences for students means having access to, and benefiting from, resources, instruction, environments, dispositions, and curricula that value students’ identities, cultures, experiences, intersectionalities, and ways of knowing. We adopt Safir and Dugan’s (2021) definition of equity to drive our work which positions each person as a daily actor in working toward equity: “Equity is an approach to ensuring equally high outcomes for all by removing the predictability of success or failure that currently correlates with any racial, social, economic, or cultural factor” (p. 29). Achieving educational equity means more than only focusing on closing gaps in standardized test scores or increasing minoritized populations in STEM careers (an oft cited equity goal). Achieving equity requires that educators examine the systemic and institutional conditions re/producing inequitable outcomes- and even question what we view as successful outcomes. Examination in this way requires equity literacy. Gorski and Swalwell (2015) and Gorski (2016) describe equity literacy as the abilities of educators to: (1) recognize inequity, (2) respond in the immediate term to inequity, (3) redress inequity in the long term, and (4) sustain equity efforts. Equity literate educators reflect critically by “engaging in the sustained and intentional process of identifying and checking the accuracy and validity of [their] teaching assumptions” (Brookfield, 2017, p. 3).

Equity literacy requires the development of:

  1. (1)

    a critical consciousness [the capacity to critically analyze our “social and political conditions, endorsement of societal equality, and action to change perceived inequities” (Diemer et al., 2017)],

  2. (2)

    agentic identity [the extent to which one perceives a sense of causality of the self (Dewey & Knoblich, 2014; Obhi & Hall, 2011)],

  3. (3)

    pedagogical knowledge [knowledge of teaching and learning processes, including classroom management, teaching methods, assessment, learning processes, and student characteristics (Voss et al., 2011)], and.

  4. (4)

    organizational understanding [recognizing that schools are complex human systems where power is distributed unevenly and skilled navigation is necessary (Hargreaves & Fullan, 2012)].

Equity literacy relies on assumptions of teachers as intellectuals (Giroux, 1988) whose classroom positioning primes them as change agents. Pairing equity literacy with data literacy for educators is a critical partnership that accounts for the invaluable role of teachers in systemic change to broaden the engagement of all learners in their education. This study utilized an equity audit tool to act as a catalyzing event for this pairing.

Equity audits

An audit enables individuals to systematically examine records in ways that allow consistencies and inconsistencies to become visible; an audit gives voice to what would otherwise go unnoticed. An audit of equity is a means for identifying patterns of inequity and for implementing strategies to address those patterns (Mckenzie & Skrla, 2011). Historically, equity audits have been used as a way for people to document inequitable social, political, and economic opportunities and outcomes as a way to enforce compliance with civil rights statutes (Skrla et al., 2004). In schools, the idea of the “equity audit” emerged primarily from the work of education leadership scholars Scheurich and Skrla (2003) and Frattura and Capper (2007) who framed it as an important and practical accountability tool for school leaders. It is Frattura and Capper’s audit from which we constructed our current version of a School and Classroom Equity Audit (SCEA) for educator use. The SCEA is used to collect data related to many possible inequities within a school along the lines of race, class, gender, (dis)ability, sexual identity, and language. The audit typically includes achievement, discipline, extracurricular (e.g., student council membership), and staffing data (e.g., teacher licensure). The conductor of the audit disaggregates the data by demographics and intersectionality, identifies where there is disproportionality, analyzes why differences exist, works to devise and implement a solution, and monitors the outcomes (Scheurich & Skrla, 2003). For example, disproportionalities are identified by comparing the percentage of students of color in the school population to the percentage of students of color in the school population who are: (1) in advanced programs, (2) reported for disciplinary action, (3) identified for special education, and so forth. When a disproportionality is identified, the individual or team asks questions of the context and self that can lead to further data collection, such as the kind known as “street data” (Safir & Dugan, 2021), to better understand the experiences of students, families, and/or teachers and the systemic elements and structures that may be contributing to the inequity. Using their new understandings, the individual or team plans and engages actions to address the inequity, closely monitoring the outcomes.

In our previous work engaging individual educators with the SCEA as part of an equity study of their schools, we found that educators enter the equity study process underprepared to use data to name, reflect, and act (Wink, 2011) on inequitable schooling conditions. We found that engagement with the equity audit increases teachers’ data understanding and skill, enhances their awareness of in/equity, and helps to develop a greater sense of change agency (Dodman et al., 2019). This is promising evidence that the audit can act as a triggering event for teachers—one that sets in motion the “noticing and sensemaking” (O’Neill & Cotton, 2017, p. 344) that influences their perceptions, attitudes, and actions related to equity.

Professional development design

To engage our purpose of guiding school-based educators in their efforts to identify and address opportunity gaps within their school as identified via the SCEA, our project goals were to develop professional development experiences focused on data, equity, and culturally relevant and responsive teaching. One assumption embedded in the professional development design was that collaborative work by teachers was an essential feature of effective school change and of their self-efficacy as creative change agents (Butler et al., 2015; Davies, 2013). A second assumption was that while teachers are essential to school change (Datnow, 2020), it is equally important to support teachers in developing the knowledge and skills to function as equity leaders (View et al., 2016; Heck & Hallinger, 2010; Nguyen & Hunter, 2018; Wenner & Campbell, 2017). We believed that a partnership between teachers and administrators based on distributed leadership would have particular efficacy for orienting a school culture toward equity (Diamond & Spillane, 2016; Hargreaves & Fullan, 2012; Stillman, 2011). Relatedly, a third assumption was that this collective of educators and administrators could be a critical mass necessary for whole school change. We relied on Rogers’ (2003) potential of innovators and early adopters to diffuse ideas through a school, as well as the established relationship between awareness and responsibility (Hernandez-Wolfe & McDowell, 2012; Redford & Ratliff, 2016; Suyemoto et al., 2015) to design a model that (a) utilized teachers and administrators who volunteered for the project, and (b) paired the SCEA findings with tools to support action by way of culturally responsive and relevant teaching.

In drawing on the literature on effective professional development, we focused on data, systems, and tools to understand and respond in more ways than only research-based teaching strategy fixes. Our professional development design principles were guided by the following elements: collective, sustained, collaborative, mentored by strong leadership, supportive of teachers, and aligned with the school’s student learning goals (Bean & Morewood, 2011; Desimone, 2009; Desimone et al., 2002; Yoon et al., 2007). Additionally, effective professional development is dynamic, responding to the needs of stakeholders as their learning and context evolves (Darling-Hammond & McLaughlin, 2011; Dillon et al., 2011). As such, we moved beyond the process-product logic typical of professional development research and applied a “complexity theory lens” (Opfer & Pedder, 2011). We wanted to account for teacher emotions as they engaged in the change process (Saunders, 2013) by creating spaces for them to be vulnerable and truthful about what they did not know regarding data and equity issues. In a dialogic relationship, we wanted the teachers and administrators to guide us through their school context (Nguyen & Hunter, 2018), as we supported their equity and data literacies. Their engagement in self-study of themselves, their schools, and their classrooms was a design goal. This meant offering space and support to notice and respond to equity-related questions as we progressed.

Our professional development design included significant time with the university team and with each other to create community and to scaffold the change process. We designed two half-day workshops as teachers concluded the 2018–2019 school year, two full-day summer workshops, four half-day workshops spread across the 2019–2020 academic year, and group tasks in between sessions. All of the half-day sessions took place on school days, and funding was provided for substitutes. Participants were also compensated for their summer time engaging in the two full day sessions.

Participants

In Fall 2019, the research team obtained intramural funding for the project. The research team was connected with the school through a faculty colleague. The suburban Title I school’s student demographics were 77% Hispanic, 12% White, 6% Black/African American, 3% Asian/Pacific Islander, and 2% Multiracial. 19% of students were identified with Autism, Specific Learning Disabilities, and/or Speech/Language Impairments. 6% of students were identified as gifted. 60% of students were identified as English Learners from levels 1 through 6. Finally, 85% of students were considered economically disadvantaged.

After explaining the project and its goals, the principal agreed to school-level participation. She stated that she saw the project as complementary to her school’s typical professional development efforts in content areas such as math and reading. She described the project as an opportunity for educators to learn more about equity, which was lacking in their current professional development designs. The principal and the assistant principal described the teachers in the school as a whole as student-driven and very open to professional development, with the assistant principal characterizing the principal as “incredible” in her support for teachers’ learning. The school’s mission and vision spoke to lifelong learning, respect for each learner, and using “various forms of data collection in order to facilitate optimal learning for every student.” After Institutional Review Board and school district approvals were obtained, the principal sent recruitment materials to teachers in the school. The project had funding to support eight teachers and two administrators. The school’s two administrators (principal and assistant principal), one guidance counselor, two English for Speakers of Other Languages (ESOL) teachers, and four teachers from grades Kindergarten, first, second, and fifth agreed to participate. A tenth participant who taught fourth grade initially expressed interest, but had to decline participation due to personal obligations. Participants spanned multiple grades and instructional positions. All participants were female, eight of the nine participants identified as White and one teacher identified as Black. See Table 1 for participants. During our first professional development session (described below), the participants shared their hopes and worries for the project. Their responses indicated a clear student-centered purpose for their joining the project and an openness to learning about themselves in the process. They worried about the time the project would require between sessions and about how they would potentially implement actions for change.

Table 1 Participants
Full size table

The university team comprised five women, one who identifies as Black and four who identify as White. Four members of the university team were former PK-8 teachers and all members had experience in a wide variety of school settings. All five members teach in graduate programs for practicing teachers, and research in areas related to educational equity, teacher learning, race, and multicultural education. We were cognizant of the potential uneven power dynamic that is often present when external “experts” enter into a school building. It was important to us to talk less and listen more. We crafted tasks that shared information, but enabled participants to construct meaning of it within their school context and personal experiences.

PD sessions overview

Spring 2019

In the first half-day session, we spent three hours getting to know one another and introducing the project. Participants shared the stories of their names, their teaching challenges, and their hopes and worries for the project. They also spent time engaging in a photo metaphor activity to surface and reflect on their own K-12 schooling experiences. We then engaged in dialogue about equity, provided an overview of the project, and introduced the equity audit. We shared the SCEA tool and the ways in which other teachers used the instrument to make changes in their own school settings. Participants had an opportunity to examine the SCEA and determine how it might be revised for their school context. Three days later, we held the second half-day session where participants engaged in activities and dialogue focused on multiple perspectives and the role these perspectives play in critical reflection. Brookfield’s (2017) ideas of critical reflection as “assumption hunting” and the “interrogation of power” guided our design as we prompted educators to reflect on their own experiences as students, identifying connections and disconnections between their own stories and those of the students in their school. Participants were tasked with the goal of completing their SCEA by our next meeting at the end of the month, which they did collaboratively by splitting up the data collection tasks between them. They also relied on the principal for data access and to clarify some of the data categories and meanings.

Summer 2019

Our first full-day workshop at the end of June focused on analyzing the data the participants gathered for the equity audit. We began the session by reading Margaret Wheatley’s essay, Willing to be Disturbed (2009). Because we were acutely aware of the potential advantages of administrators and teachers engaging together, but also sensitive to the power dynamics inherent in those relationships, we also co-constructed norms for our work with one another. Norms included distinguishing discussion versus dialogue to recognize that some of our time would be spent in discussion to come to decisions, but that much of it would be spent in dialogue where our goal was stronger understanding; active participation; creating an environment of trust and honesty, and awareness of positionality. See Table 2 for the co-constructed norms that guided our collaboration. Participants then analyzed the SCEA data they had collected regarding their school and classrooms. To do this, they began the CDDDM cycle: they raised questions of the data, turned the data into information, and asked why we are seeing what we are seeing. Participants identified four inequities they saw as necessary for redress: gender disproportionality in discipline data, English Learner underrepresentation in the gifted program, gender disproportionality in STEM, and high student mobility causing disrupted learning and community. Participants formed groups of 2–3 to further examine each area. Before making decisions about how to use their new actionable knowledge to redress identified inequities, we introduced the ideas of culturally responsive and relevant pedagogies (Gay, 2018; Ladson-Billings, 2021). Participants reflected on and self-assessed their dispositions, knowledge, and skills using the Multicultural Teacher Capacity Scale (Cain, 2015), were tasked with reading No More Culturally Irrelevant Teaching (Souto-Manning et al., 2018) and researching the inequity selected by their group for the next session using academic and practitioner resources.

Our second full day session was in August prior to the start of the new school year. We continued to examine culturally responsive and relevant teaching through activities and dialogue related to the Souto-Manning text, including exploring practical moves to illustrate what such educational practices can look like, sound like, and feel like. We specifically spent time interrogating school/classroom rules, mathematics, and typical “what did you do over the summer?” story prompts. The afternoon was spent developing action plans for the four identified inequities to be implemented in the upcoming school year.

Fall 2019

During the school year, we held four more half-day professional development sessions. The first, in October, focused on re/examining the concepts of “success,” “equity,” and “culture.” Teachers also shared changes to their beginning-of-the-year activities and how they were moving towards more cultural inclusivity. Stressing a strengths-based approach to working with students and families, participants brainstormed assets of their students, families, and school and ways these assets could enhance their action plans for equity. Participants also spent time reflecting on their own goals and growth regarding the Multicultural Teacher Capacity Scale (Cain, 2015) that was introduced in the summer. The session in December continued to reinforce this work and built in the idea of making thinking visible through Visible Thinking Routines (Richhart et al., 2011).

Spring 2020

We reconvened in February to pull the work together, adding additional conversations to cultivate a structural lens on the equity issues being addressed. We used Gorski’s (2020) framework of the deficit lens, the grit lens, and the structural lens to critically reflect on equity issues in schools. We emphasized the fact that we cannot replace equity and justice work with celebrations of diversity. Participants worked in their action plan groups to identify what they learned, what is different now, and the continuing challenges related to the equity issue they identified. In our final meeting of March 2020, we held focus groups and conducted final interviews regarding the project. We also made plans for schoolwide sharing of participants’ actions. Unfortunately, two days later schools closed due to the global COVID-19 pandemic, and participants did not have the opportunity to share their work with the larger school community as part of this project.

Methods

The study employed a mixed-methods, single school case study approach (Yin, 2014). According to Yin (2003), “a case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident” (p. 13). In our case, the contemporary phenomenon was teachers’ knowledge and use of equity data in an elementary school setting. Merriam’s (1988) rule of thumb for deciding whether case study design is a good fit for a given research project requires asking whether or not there are a finite number of people who could be observed and/or interviewed in a finite amount of time in order to gather information about a given phenomenon, thus creating “boundaries” for the case. Working with the educators in this school for an entire school year provided those boundaries for this case study. Our research questions were: (1) How does professional learning related to data use for equity affect educators’ awareness, understanding, commitment, and actions related to addressing inequity in their classrooms and school? and (2) What are educators’ perspectives of the professional development model and their experience?

Data collection and analysis

Data were collected and analyzed from four sources. Data from these sources were used to ascertain the project’s outcomes, but also to guide the professional development decisions during the course of the project.

Multicultural Equity and Agency Survey (MEAS)

A survey was developed and used to assess participants’ perceptions of their roles and responsibilities as teachers and administrators; perceptions of their schools and K-12 students; and confidence, interest in, and skill with data collection and analysis. The survey was completed by each participant during the first professional development session in Spring 2019 and again at the final session a year later in Spring 2020. The survey consisted of 48 items measured on a 5-point Likert-type scale. Items regarding teachers’ perceptions of their roles and responsibilities were based on Cochran-Smith and Lytle’s (2001) conceptualization of an inquiry stance. We also used and adapted items from Lukacs’s (2008) validated Teacher as Change Agent Scale. To ascertain teachers’ perceptions of equity in their classrooms and schools, we included items that directly aligned with equity audit data collection. Finally, the questions related to data collection, analysis, and communication were drawn from review of data literacy literature. Content validity of the survey was enhanced via expert review by a faculty colleague and two classroom teachers. Due to the small sample size, participants’ pre- and post-survey responses were analyzed using descriptive statistics by organizing items into composite variables: sense of agency, general data and data use, general sense of equity, and overall equity. The general sense of equity composite included three items focused on holistic perceptions of school and classroom equity and school policies. The overall equity composite included items related to participants’ more targeted perceptions of equity in their classroom and school related to ability, gender, race, language, and sexual orientation.

Multicultural Teacher Capacity Scale (MTCS, Cain 2015)

The MTCS was used as a self-assessment of participants’ knowledge, skills and dispositions related to multicultural education. The tool includes: (1) dispositional items that ask participants to assess the extent to which they believe themselves to: be socioculturally aware, affirm students’ cultural assets, commit to students’ success, and be agents of change; (2) knowledge items that ask participants to assess the extent that they understand the sociopolitical context of schools, understand the impact of context and culture on students, and demonstrate experiential knowledge of school and students’ communities; and (3) action items that ask participants to assess the extent to which they create classroom community that embraces students, engage in critical reflection to guide practice, foster the sociopolitical consciousness of students, and modify curriculum and pedagogy to confront issues of equity. MTCS responses were analyzed descriptively to determine participants’ self-reported progress in the identified dimensions.

Participant interviews

Data related to teachers’ hardships from the prior school year and their hopes and worries related to the project were collected during the first session as a professional development activity. A focus group was conducted at the end of the project with teacher participants to ascertain their perspectives of the entire professional development process, including what they gained or did not gain from the self-study, barriers to the process and implementation, and facilitators to effecting culturally relevant changes in their classrooms and/or school. Sample prompts/questions included: Over the past year, you have engaged in an equity audit and professional development related to culturally relevant teaching as part of an equity self-study. Thinking about the equity self-study process from the beginning until now how would you characterize the process? What benefits, if any, do you feel you gained? Why? What were challenges for you in the process? Why? How did you overcome those challenges, if at all?

Administrator participants were individually interviewed at the beginning of the study related to their perspectives on their school, teachers, and their project goals. Example interview questions included: What drew you to the project? Why were you interested? What do you hope will be the outcomes of this project? Administrators were again interviewed at the end of the study to learn their perspectives on the professional development experience, including the self-study efforts. Sample questions included: What changes, if any, did you notice during and/or at the conclusion of the project? Based on your goals for the project, how well do you think those outcomes were achieved?

Individual and focus group interviews were transcribed and analyzed inductively for themes (Hatch, 2002) using a team-based approach to code and analyze data. We used Dedoose, a qualitative data analysis program, to house and organize our data for collective access. To determine codes, each team member first read each interview for impressions and noted potential codes and relevant text segments. The team then met and determined initial codes based on those impressions and the research questions. Pairs of team members coded each interview with the team-determined codes. Pairs met to discuss code application and potential refinements to the codes. The entire team then met to discuss application of codes and determine agreement. The codes were then refined for the team and each pair coded their interview again using the refined and expanded codes and sub-codes. Examples of codes included: benefits, power orientation, challenges, project design, views of teachers, expressions of affect, perceived effects, and culture/race awareness. All members read through all coded interview data to identify themes, after which the group met again to identify themes across the interviews and determine findings utilizing the interview data, survey and MTCS results, and archival materials.

Archival materials

Archival materials, including session notes and participant action plans, provided additional data to assess the processes and impact of the professional development and self-study experiences. In partnership with the other three data sources, the archival materials were used to offer examples and provide confirming and/or disconfirming evidence of participants’ experiences during the professional development.

Findings

What we found is just as nuanced as any learning endeavor. To present our findings, first we report on overall themes in participants’ learning and their perceptions. Then we break the findings down by data and equity literacies and the professional development model.

Overall

The participants expressed positive perceptions of the professional development experience. They noted that their perspectives were altered through a new awareness of in/equity. On both the pre/post survey and MTCS, all participants reported enhanced knowledge, skills, and dispositions related to data and multicultural education. See Fig. 2 for the MTCS growth by participant. See Fig. 3 for survey results by participant role (administrators, guidance counselor, and teachers) organized by the composite variables: general sense of equity, sense of agency, general data and data use, and overall equity. Specifically, the survey indicated a noticeable increase in teachers’ sense of agency and a slight increase in general data and data use.

Fig. 2
figure 2

MTCS growth by domain

Full size image
Fig. 3
figure 3

MEAS findings by position type

Full size image

Teacher participants’ scores decreased in the equity variables. Paired with teachers’ comments during the concluding focus group, we see this decrease as a positive and not unexpected outcome. As teachers examined their classroom and school level data in terms of representation (which is not something that they had previously done), their perspectives of equity in those spaces were altered. Their decreased sense of equity generally and along more specific lines as represented in the overall variable indicate a greater awareness of in/equitable conditions in their classrooms and schools. By the end of the project, they were able to recognize inequity that had previously been invisible, and as evidenced by their agency responses, they felt better positioned to act.

The administrators self-reported higher on all variables in the survey and reported an increase in all areas of the MTCS. It is not surprising that their sense of agency only increased slightly as they were already positioned in roles of power at the start of the project. Their wide increase in data and data use is encouraging because of this positioning. As they move forward in using data to make school level decisions, there is hope that they will engage data use for equity broadly; in fact, in her interview at the conclusion of the project, the principal indicated this to be the case when she stated: “I also think it will help people look at data even from our school improvement team, look at data maybe a little bit differently, and challenge some of those things and goals maybe that we’ve already written”. Finally, there was a wide increase in general sense of equity and overall equity expressed by administrators, as compared to the teachers’ responses. We lack data in terms of their own perceptions of this outcome; however, field notes indicate that administrators were often quick to rationalize outlier data that indicated potential inequity and often reassured teachers that their instruction was already strong when it was problematized. This supportive relationship was potentially important to a sense of safety for the teachers, but it perhaps over-prioritized comfort in that service.

To further respond to our research questions regarding data use for equity outcomes for participants and their perceptions of the professional development model, we share findings organized by the elements of data and equity literacy we intentionally sought to foster. Lastly, we speak to the participants’ perceptions of the professional development experience.

Data literacy

The elements of data literacy we sought to intentionally foster are presented in Table 2. This list of data literacy elements is adapted from Mandinach and Gummer’s (2016) framework of data literacy for teachers. For our project, we selected items from their framework that seemed especially important to data use for equity and revised items to better account for the kind of work in which we were asking educators to engage. As an example, we have framed data use for equity as broader than examining student level academic performance data; therefore, we revised items that focused solely on data as performance data and clarified an explicit focus on approaching data and their analyses critically and contextually. The table represents those elements of data literacy that we intentionally planned to address via our professional development activities and also represents the elements about which we have evidence of impact.

Table 2 Data literacy findings
Full size table

On the post-survey, six participants reported increased strengths with general data and data use. The only data-related item that did not report a higher mean was the item: I enjoy analyzing data about my students. This item demonstrated no change. Enjoyability of data engagement has not necessarily been theorized to be essential to data literacy, and our findings support that. Not enjoying data did not seem to hinder participants’ engagement with them, as they seemed to recognize data as a useful tool—an element that has been theorized to be important to data literacy. It is important to note that the third grade teacher, Veronica, was not in attendance at our final meeting and was unable to complete a post-survey; therefore, neither her pre- nor her post-survey responses are included. Two of the participants reported lower general data use perceptions: the kindergarten teacher, Erica, and the first grade teacher, Sara. Erica sporadically attended the professional development as she was on an extended leave of absence during the year for health reasons. If post-scores had jumped significantly higher, that would not have been aligned with her participation in the professional development sessions. Sara’s lower reported confidence, interest in, and skill with data collection and analysis is surprising as she was highly engaged in the sessions.

The data related activities of the equity audit data collection and analysis addressed each data literacy element noted in Table 2. Our field notes indicate engagement in all intended elements over the course of the project. For example, when conducting the equity audit, participants engaged in practice data analysis which led them through how to interpret sample quantitative audit data and how to determine proportionality. Participants then discussed potential sources of data for audit indicators and divided the data collection amongst the group where they had to make decisions about data quality and to disaggregate bulk data that they gathered. This process required participants to engage each element of Use Data.

Due to the nature of couching the data literacy elements in an equity literacy framework, Identifying Problems and Framing Questions, Transforming Data into Information, and Transforming Data into Decisions took on an especially significant role during data collection and analysis. The full day of analyzing the equity audit data was especially powerful in fostering these elements as teachers used data to understand their students’ experiences. During this day, the principal, in particular, who regularly used data at the school level, expressed the following during the session: “I thought I knew the areas [to improve] but I feel different after actually looking [at this data]. More dialogue needs to happen before discussions so that our plan of action is the best choice at the time; it was hard not to jump to a solution. [We need to] slow down and use data for informed decisions.” Here she is indicating changes to how she notices and frames problems. Sara noted “how data doesn’t always reflect what reality is because of how it’s reported or collected.” This kind of comment can provide insight into her post-survey responses as she is now starting to view data more critically and to consider the potentially subjective nature of ‘objective’ data. In one instance, as teachers noticed a disproportionality of students engaged in school clubs and organizations, they stumbled into a probe for causality. In this instance, teachers who were now analyzing data beyond “just scores,” asked questions regarding access. It was revealed that the clubs are offered before and after-school making transportation an obstacle for many families. Their data analysis also revealed a discrepancy between the demographics of students and families and their teachers and district leadership with the majority of students and families being native Spanish speakers while no member of the school board identified as bi/multilingual and the majority of teachers in the school identified as White, native speakers of English. The principal noted she goes to one rural county in Pennsylvania to recruit teachers and thought aloud if she might broaden her recruiting pool.

As participants made sense of their collected data, tensions were also revealed. We noticed three particular tensions related to data literacy. The first is a tension of overgeneralization and resistance to outliers. This was most noted as being engaged by the principal in two specific instances. In the first, she was responding to the data regarding racial demographics of special education enrollment. She commented that “every White kid is in special education.” Although she had data in front of her that said otherwise, she overgeneralized. While the school is a center for students with autism, only 17 students who identified as White received special education services. There were 62 students who identified as White in the entire school. Conversely, during a dialogue regarding discipline data, it was noted that 100% of the school’s discipline referrals were for male students and 70% of those referrals were for students of color. The conversation focused on one African American student who had received more than one referral. However, because the school had such limited numbers of African American students, the principal was quick to rule out potential disciplinary bias. This occurred despite the only teacher of color in the group (Sara) relaying a story about her son (a male student of color) being unjustly accused of assault at school in the same school district. While the principal noted that engaging with data as we were doing had prompted her to reconsider how she frames problems, session evidence revealed the tension between perhaps wanting that to be true and actually engaging in representative actions. In these instances, the data were not enough to counter the protective views she held of her school and her teachers.

The second tension is related to data’s perceived accuracy. When reviewing racial demographic data, a group of participants noticed that many families they knew to be “Hispanic” had identified their children as White on the school forms. The group then started re-identifying the children based on their knowledge of their families. This launched the group into a conversation about why parents may have selected White on the forms. These reasons ranged from perhaps parents were unsure what the racial designations meant to they thought that identifying as White would bring advantages to their child. This situation raises questions about how educators may proceed when they feel data are inaccurate—particularly data provided by parents. Further exploring this tension, at the second to last professional development session, we provided the group with data from an academic research study that found that parents with college degrees and parents without college degrees both reported their children doing homework in a dedicated work space with adult monitoring. The same was found to be reported by parents who spoke and who did not speak English in their home. Participants’ responses indicated disbelief in the accuracy of parents’ responses. Participants started to then ask questions of the data to better understand the sample. Their reaction could be seen as positive movement in asking questions of the data and not accepting what is presented to them as objective truth– movement towards approaching data critically. However, it took the data revealing something that ran counter to deficit beliefs about families to prompt this questioning. The structure of the professional development session intentionally led participants through identifying problems and framing questions, using data, transforming data into action, transforming data into decisions, and evaluating outcomes. Because we couched these elements and actions within a framework of equity literacy, participants’ critical consciousness, agentic identity, and organizational understanding were fostered.

Equity literacy

Elements of equity literacy that we sought to foster are found in Table 3. The following outcomes represent how these elements were demonstrated by participants.

Table 3 Equity literacy findings
Full size table

New and altered lenses

Qualitative data analysis revealed multiple ways that participants experienced new and altered lenses. For example, in speaking about changes she has experienced, the guidance counselor noted: “[I’m thinking now] how can we improve, and how can we work to problem solve, how are we looking at it? I’m looking at it now through the initiative lens, whereas I think before it was more just, like, advocacy for students.” This distinction between advocacy and “an initiative lens” is significant. It demonstrates an agentic move that recognizes one’s influence within a system. It also demonstrates a greater recognition that how we look at a problem affects the solutions we entertain. Relatedly, first grade teacher, Sara, further noted a recognition that her own experiences as a girl of color are not necessarily the experiences of other people of color:

I don’t ever remember me questioning, ‘why are all the princesses White?’ I always saw myself [as though] I could be a princess…So, now it makes me think that maybe others don’t, other kids don’t see it that way. They’re maybe asking those questions. I only looked at it from my lens.

Similarly, fifth grade teacher, Tracey, noted in the focus group that the project pushed her to reconsider what she thought she knew and was doing. She said: “I knew gifted was a problem, but did I really? Was I addressing it or any of us addressing this kind of stuff? No.” For Tracey, the SCEA process meant admitting that what she identified as problematic previously was limited, as was her sense of agency in addressing it.

Using data to start equity conversations primed teachers for further hard conversations

Evidence suggests that the space created by the model enabled participants to share knowledge and build trust in ways not nurtured in typical daily structures. The use of data aligned with learning around culturally responsive and relevant pedagogies fostered confidence in noticing and addressing in/equity. As Tracey, noted, “it’s a lot easier to have conversations about race than before.” During the group’s time with one another, conversations ventured into analyzing school-based cultural celebrations and use of potentially stereotypical texts. In one example, participants voiced uneasiness with a school event that generalized the Latinx students as Mexican, despite the majority of Latinx families being from Central America. They expressed previously not having the comfort nor the vocabulary to confront the inclusion of the activity. Sara shared “I think it made me feel more comfortable to speak out because I know when that whole taco thing came up and the Mexican hat dance and it didn’t sit well with me, but I didn’t really know how to explain why without sounding like ‘I just didn’t like your idea’… no one said anything.” Tracey noted that “none of us felt comfortable enough and now I probably would say something.” The space created to share this discomfort was important to priming the teachers to start and continue conversations that recognize students’ marginalization and work to re-center their identities. It is important to note that development of a critical consciousness and agentic identity was ongoing. Even over the course of a year, teachers were solidly in a space of becoming versus became.

Professional development

Participants expressed a strong satisfaction with the professional development model. Among those elements already noted in the professional development literature, two further elements seemed important to participants’ learning.

Administrative participation facilitated teachers’ engagement

Although we noted earlier that the principal had a tendency to “protect” her school and teachers, she also acted as a powerful enabler. At one point, Tracey noted that their school level conversations were starting to focus on what teachers can do now, rather than continuing to wait for the district or state to change their policies. The principal’s physical and cognitive presence and active engagement in the professional development offered teachers the permission necessary to openly question and share concerns. She was an active participant in identifying inequities and then in addressing her selected inequity (student mobility) with a colleague.

Teachers desired more structure

In the post-focus group, participants expressed wanting greater accountability and structure from the university team. They suggested greater use of scenarios to help them practice what to do and say in tough situations, especially when they identify inequity. Interestingly, at the start of the project, Sara and Katrina (an ESOL teacher) noted their apprehension about talking about race because that would ultimately lead to boxing children into essentialized categories. As a result of this, we spent time examining a more nuanced definition of culture and had multiple and varied conversations as a group regarding culture and race as constructs and how our instructional decisions and expectations are influenced by our cultures and identities. We made intentional effort to move participants away from essentialized notions of culture pedagogy. However, participants still wanted a stronger “if…then” component to the sessions and more targeted “try it” tasks between sessions with stronger accountability.

Discussion

This project intended to foster educators’ critical data-driven decision-making capacity through the integration of equity and data literacies, in a construct we refer to as Data Use for Equity. Our findings indicate that integrating these literacies through ongoing, collaborative professional development can positively impact educators’ engagement in working toward equity with data. However, our findings also demonstrate that such development must continue beyond structured professional development experiences to be part of a larger way of work in a school. Our next phase of study will attend to how this group of educators continued their development beyond our project. Safir and Dugan (2021) argue that working toward equity requires: (1) acknowledging that our systems, practices, and narratives are designed to perpetuate disparities in outcomes for marginalized students, (2) deliberately identifying barriers that predict success or failure and activity disrupting them, (3) consistently examining personal identity, bias, and both personal and collective contributions to the creation and/or reproduction of inequitable practices, (4) Re/allocating resources to ensure every child gets what they need to succeed to thrive socially, emotionally, and intellectually, and (5) cultivating the unique gifts, talents, and interests that every person possesses (p. 8). Our findings include evidence related to educators beginning their development in relation to the first four of these actions. Participants used data to make visible areas of their students’ and families’ educational experiences that were not previously acknowledged by them as equity issues. They began to unpack and reorient unrecognized deficit views (Bertrand & Marsh, 2021). Upcoming publications will report on significant moments of participants’ engagement and the nuance of individual participants’ development.

At the start of the professional development efforts, the principal noted that equity work and dialogue was not typical in their school’s professional development opportunities. For the participants, this was one of the first organized efforts in data use for equity in which they had participated. The findings indicated that by the end of the project, although they felt more prepared, teachers were still uncertain about their skills for change on their own. This was evidenced by Tracey’s use of the language of “probably” in imagining her future actions and the participants’ desire for more structure in the project. As we consider the project’s influence on participants’ commitment to addressing equity in their school, it is evident that they are still developing. This first endeavor was impactful, but not complete. Future work will need to emphasize greater action opportunities throughout the project in perhaps seemingly small ways to foster participants’ feelings of commitment and agency.

Over the course of the self-study, the educators engaged in praxis as “the action and reflection of men and women upon their world in order to transform it” (Freire, 1970, p. 79). Their engagement in critical consciousness development is not a destination, but a journey. These participants did not become data and equity literate as a result of this yearlong endeavor. Rather, they strengthened a foundation on which their journey of becoming can continue. While we had to stop short of continuing with the group due to COVID-19 school shutdowns, following up with the group now that so much has shifted in the landscape of “equity” may offer important insight into how the educators utilized that foundation and worked toward sustainable change in their classrooms and school. Following the teachers into other parts of their day when they engage in data conversations with team-based colleagues would also be an important element by which to understand their broader data use for equity and critical data-driven decision making engagement. It would also allow for more day-to-day visibility of their opportunities for systemic equity-oriented action. Furthermore, the importance of ongoing support to build on and sustain teachers’ efforts should not be overlooked. Research has found that the most powerful professional development, particularly related to issues of equity, is ongoing and embedded in the day-to-day work of educators (View et al., 2020; Darling-Hammond et al., 2017); therefore, we wonder how extending the project beyond the designated school year would have affected participants’ development.

Relatedly, we intentionally spent our professional development time examining data that told a larger systemic story of in/equity in the school. It will be important moving forward to also spend time in greater classroom level data disaggregation. While reporting on and analyzing classroom-level data is a key part of the SCEA, our professional development did not linger with these data. We used the professional development in cultural responsiveness to help participants make connections between the revealing systemic data and their classroom pedagogies, but future work should strengthen this by more targetedly interrogating participants’ classroom level data as part of professional development efforts like the one described in this paper. Informed by their feedback, participants would have benefitted from conversations around topics such as how to identify, disaggregate, and analyze data when all students in one’s class seem to fall into similar identity marker groups in terms of language, race, ethnicity, income, and/or dis/ability. Given their concerns about essentializing children and families, doing so could offer nuance to their lenses and skills when tandemly paired with the greater attention to school-based systemic inequities.

The project’s findings also highlight the recognition that teachers are social and emotional learners. Decades in teacher learning speak to learning as more than simply acquiring knowledge; there is an emotional and desire-related aspect of change (Hargreaves, 1994). These educators all volunteered for the project, and still there was apprehension and pushback when their views of themselves as student-centered were challenged. Implicating themselves in inequitable schooling conditions was hard, even for this group of educators who joined the project with an explicit stated desire of wanting to learn and change their practices. One of the largest obstacles that persisted was a difficulty in recognizing deficit assumptions in relation to their students and families. Similar future work in data use for equity would benefit from taking this potential obstacle into account. Utilizing case studies with carefully facilitated analysis that help teachers make the connection between abstract cases and their own selves and contexts may be one way to address this challenge.

While we do not know the extent of diffusion within the school (Rogers, 2003), this group as a critical mass of administrators, classroom teachers, and specialists does seem to hold great promise for infusing a greater equity lens into their data use. The altered lenses that resulted from collecting, mathematically calculating representation, and analyzing new and varied forms of data offered participants new insights into equity and their agency in addressing their own classroom practices and also influential systemic conditions. The inclusion of the principal and assistant principal as participants alongside the teachers supports findings that speak to the necessity of school administration to equity-related change. In their study of teacher leaders, Jacobs et al. (2014) reported on the difficulty of equity-oriented actions when school administrators did not hold equity-oriented change visions. Our study extends these findings as we reported on the key role that the principal played by not only offering verbal support, but being an active member of the group alongside the teachers. Previous work by the first author (Dodman, 2021) found that the role of the principal in teacher learning and change efforts can determine how the work is taken up or continued by teachers; support in passing for teachers’ change efforts is insufficient. Future professional development efforts intended to promote equity-oriented school change should build on this by planning for the inclusion of participants from multiple levels of the organization.

Because schools hold great possibility as sites of liberation for students, teachers’ engagement in naming and addressing the roles that schools can and do play in crafting and perpetuating marginalization and minoritization of students (and very often themselves as teachers) is necessary to manifest those liberatory possibilities (Au, 2014; Delgado Bernal, 2002; Ladson-Billings & Tate, 1995; Migliarina & Annamma, 2020). While more study is necessary, the findings of this study indicate that the intentional development of data and equity literacies in a professional development model focused on culturally responsive and relevant teaching has promising school-based possibilities.