-
Use and Usefulness of Assessments to Inform Instruction: Developing a K-12 Classroom Teacher Assessment Practice Measure Educational Assessment Pub Date : 2024-02-13 Divya Varier, Marvin G. Powell, Stephanie Dodman, Samantha T. Ives, Elizabeth DeMulder, Jenice L. View
Considerable literature is devoted to teachers’ assessment use to support teaching and learning. The study examined the factor structure of a measure of teachers’ assessment use along the assessmen...
-
Justice-Oriented, Antiracist Validation: Continuing to Disrupt White Supremacy in Assessment Practices Educational Assessment Pub Date : 2023-11-22 Jennifer Randall, Mya Poe, Maria Elena Oliveri, David Slomp
Traditional validation approaches fail to account for the ways oppressive systems (e.g. racism, radical nationalism) impact the test design and development process. To disrupt this legacy of white ...
-
Measuring Item Influence for Diagnostic Classification Models Educational Assessment Pub Date : 2023-08-14 Daniel P. Jurich, Matthew J. Madison
Diagnostic classification models (DCMs) are psychometric models that provide probabilistic classifications of examinees on a set of discrete latent attributes. When analyzing or constructing assess...
-
English Learners and Constructed-Response Science Test Items Challenges and Opportunities Educational Assessment Pub Date : 2023-07-07 Tracy Noble, Craig S. Wells, Ann S. Rosebery
This article reports on two quantitative studies of English learners’ (ELs) interactions with constructed-response items from a Grade 5 state science test. Study 1 investigated the relationships be...
-
How Serious are We About Fairness in Testing and How Far are We Willing to Go? A Response to Randall and Bennett with Reflections About the Standards for Educational and Psychological Testing Educational Assessment Pub Date : 2023-06-19 Guillermo Solano-Flores
ABSTRACT Jennifer Randall’s paper on justice-oriented assessment and Randy Bennett’s paper on socioculturally responsive assessment address fairness in the testing of racially, culturally, and linguistically diverse student populations by providing principles and recommendations for improved assessment practice. I warn about the perils of assuming that principles and recommendations suffice to promote
-
Examining the Question-Behavior Effect in Low-Stakes Testing Contexts: A Cheap Strategy to Increase Examinee Effort Educational Assessment Pub Date : 2023-06-19 Sara J. Finney, Mara E. McFadden
Questioning individuals about future behavior influences the subsequent performance of that behavior. This question-behavior effect may be enhanced when questions include positive self-identity pro...
-
Identifying Problem-Solving Solution Patterns Using Network Analysis of Operation Sequences and Response Times Educational Assessment Pub Date : 2023-06-16 Maoxin Zhang, Björn Andersson
ABSTRACT Process data from educational assessments enhance the understanding of how students answer cognitive items. However, effectively making use of these data is challenging. We propose an approach to identify solution patterns from operation sequences and response times by generating networks from process data and defining network features that extract essential information from them. With these
-
It Ain’t Near ‘Bout Fair: Re-Envisioning the Bias and Sensitivity Review Process from a Justice-Oriented Antiracist Perspective Educational Assessment Pub Date : 2023-06-12 Jennifer Randall
ABSTRACT In a justice-oriented antiracist assessment process, attention to the disruption of white supremacy must occur at every stage – from construct articulation to score reporting. An important step in the assessment development process is the item review stage often referred to as Bias/Fairness and Sensitivity Review. I argue that typical approaches to the item and test review process miss the
-
Fairness in Educational Assessment and the Next Edition of the Standards: Concluding Commentary Educational Assessment Pub Date : 2023-05-28 Joan L. Herman, Jose Felipe Martínez, Alison L. Bailey
ABSTRACT This special issue of Educational Assessment seeks to encourage reflection and discussion around the different assumptions and conceptualizations of fairness in assessment and their potential links to and implications for the next edition of the Standards for Educational and Psychological Testing. In this final commentary, the special issue editors summarize the major points advanced by the
-
Let’s Agree to (Mostly) Agree: A Response to Solano-Flores Educational Assessment Pub Date : 2023-05-25 Randy E. Bennett
ABSTRACT “Toward a Theory of Socioculturally Responsive Assessment” assembled design principles from multiple literatures and wove them into a working definition and a network of empirically testable propositions. The intention was to offer a coherent theoretical framework within which to understand why and how particular assessment designs might work, what actions testing programs should consider
-
Public Opinion About National Large-Scale Student Assessment: A Case of NAPLAN Educational Assessment Pub Date : 2023-05-21 Jihyun Lee, Jung-Sook Lee, Neville Ellis
ABSTRACT This study examines public opinion about the National Assessment Program – Literacy and Numeracy (NAPLAN) in Australia, based on an online survey conducted with a sample of New South Wales (NSW) residents (n = 2,017). Our study participants did recognize the benefits of NAPLAN despite their opinion generally trending toward negative viewpoints of the testing program. The views about the positive
-
Introduction to the Special Issue: Fairness in Educational Assessment and the Next Edition of the Standards Educational Assessment Pub Date : 2023-05-21 Joan L. Herman, Alison L. Bailey, Jose Felipe Martinez
ABSTRACT This introduction provides context for Educational Assessment’s special issue, ”Fairness in Educational Assessment and the Next Edition of the Standards.” The article introduces the topic of fairness by citing a prior Special Issue on which the current issue is built, summarizes the current Fairness Standards of the Standards for Educational and Psychological Testing (2014) and provides an
-
A Comparison of Methodologies for Scaling Longitudinal Social-Emotional Survey Responses Educational Assessment Pub Date : 2023-05-18 James Soland, Megan Kuhfeld, Brennan Register
ABSTRACT Much of what we know about how children develop is based on survey data. In order to estimate growth across time and, thereby, better understand that development, short survey scales are typically administered at repeated timepoints. Before estimating growth, those repeated measures must be put onto the same scale. Yet, little research examines how scaling decisions affect comparisons of growth
-
Response to Solano-Flores: How Serious are We About Fairness in Testing and How Far are We Willing to Go? Educational Assessment Pub Date : 2023-05-15 Jennifer Randall
ABSTRACT It Ain't Near 'Bout Fair: Re-envisioning the Bias and Sensitivity Review Process from a Justice-Oriented Antiracist Perspective was intended to facilitate conversation in the field about the bias and sensitivity process, specifically. I argue that our current approaches rely far too heavily on fear-based notions to the exclusion of justice-based aims. As Dr. Solano-Flores points out, however
-
Toward a Theory of Socioculturally Responsive Assessment Educational Assessment Pub Date : 2023-05-02 Randy E. Bennett
ABSTRACT In the United States, opposition to traditional standardized tests is widespread, particularly obvious in the admissions context but also evident in elementary and secondary education. This opposition is fueled in significant part by the perception that tests perpetuate social injustice through their content, design, and use. To survive, as well as contribute positively, the measurement field
-
Assessing Mathematical Higher-Order Thinking Skills: An Analysis of Turkish University Entrance Examinations Educational Assessment Pub Date : 2023-04-17 Utkun Aydin, Bengi Birgili
ABSTRACT Internationally, mathematics education reform has been directed toward characterizing educational goals that go beyond topic/content/skill descriptions and develop students’ problem solving. The Revised Bloom’s Taxonomy and MATH (Mathematical Assessment Task Hierarchy) Taxonomy characterize such goals. University entrance examinations have been seen as one way of accomplishing these goals
-
The Effects of Providing Students with Revision Opportunities in Alternate Assessments Educational Assessment Pub Date : 2022-10-25 Okan Bulut, Hatice Cigdem Bulut, Damien C. Cormier, Munevver Ilgun Dibek, Merve Sahin Kursad
ABSTRACT Some statewide testing programs allow students to receive corrective feedback and revise their answers during testing. Despite its pedagogical benefits, the effects of providing revision opportunities remain unknown in the context of alternate assessments. Therefore, this study examined student data from a large-scale alternate assessment that allows students to make multiple attempts until
-
The Nature of Cognitive Test Anxiety: An Investigation of the Factor Structure of the Cognitive Test Anxiety Scale Educational Assessment Pub Date : 2022-10-06 Lilla Németh, László Bernáth
ABSTRACT The Cognitive Test Anxiety Scale (CTAS) is a unidimensional scale designed to measure the cognitive aspect of test anxiety. The instrument has been adapted in several countries, and convincing psychometric properties have been found; however, uncertainties remain regarding its factor structure. Therefore, the aim of this study is twofold: to revise the instrument’s factor structure and to
-
Predicting Retention in Higher Education from high-stakes Exams or School GPA Educational Assessment Pub Date : 2022-10-05 Martijn Meeter
ABSTRACT The transition from secondary to tertiary education varies from country to country. In many countries, secondary school is concluded with high-stakes, national exams, or high-stakes entry tests are used for admissions to tertiary education. In other countries, secondary-school grade point average (GPA) is the determining factor. In the Netherlands, both play a role. With administrative data
-
Assessment during COVID-19: Students and Teachers in Limbo When the Classroom Disappeared Educational Assessment Pub Date : 2022-10-03 Lise Vikan Sandvik, Bodil Svendsen, Alex Strømme, Kari Smith, Oda Aasmundstad Sommervold, Stine Aarønes Angvik
ABSTRACT The lockdowns that began during the spring of 2020 changed the conditions for teaching and assessment across the globe. In Norway, schools were closed, and all school activities took place online. Moreover, all final exams were canceled, and all student grading was based on final grading by the individual teacher. Because of this, teachers’ assessment skills became more important. This study
-
To What Degree Does Rapid Guessing Distort Aggregated Test Scores? A Meta-analytic Investigation Educational Assessment Pub Date : 2022-08-25 Joseph A. Rios, Jiayi Deng, Samuel D. Ihlenfeldt
ABSTRACT The present meta-analysis sought to quantify the average degree of aggregated test score distortion due to rapid guessing (RG). Included studies group-administered a low-stakes cognitive assessment, identified RG via response times, and reported the rate of examinees engaging in RG, the percentage of RG responses observed, and/or the degree of score distortion in aggregated test scores due
-
Introduction to Twin Pandemics: How a Global Health Crisis and Persistent Racial Injustices are Impacting Educational Assessment Educational Assessment Pub Date : 2022-08-05 Alison L. Bailey, José Felipe Martínez, Andreas Oranje, Molly Faulkner-Bond
Published in Educational Assessment (Vol. 27, No. 2, 2022)
-
Early Literacy, Equity, and Test Score Comparability during the Pandemic Educational Assessment Pub Date : 2022-08-05 James Soland, Anita McGinty, Allison Gray, Emily J. Solari, Walter Herring, Rujun Xu
ABSTRACT Kindergarten entry assessments (KEAs) are frequently used to understand students’ early literacy skills. Amidst COVID-19, such assessments will be vital in understanding how the pandemic has affected early literacy, including how it has contributed to inequities in the educational system. However, the pandemic has also created challenges for comparing scores from KEAs across years and modes
-
An Intersectional Approach to DIF: Comparing Outcomes across Methods Educational Assessment Pub Date : 2022-08-05 Michael Russell, Olivia Szendey, Zhushan Li
ABSTRACT Recent research provides evidence that an intersectional approach to defining reference and focal groups results in a higher percentage of comparisons flagged for potential DIF. The study presented here examined the generalizability of this pattern across methods for examining DIF. While the level of DIF detection differed among the four methods examined, the pattern in which the intersectional
-
Allocating Resources for COVID-19 Recovery: A Comparison of Three Indicators of School Need Educational Assessment Pub Date : 2022-08-05 Jonathan D. Schweig, Andrew McEachin, Megan Kuhfeld, Lou Mariano, Melissa Diliberti
ABSTRACT As students return to in-person instruction in the 2021–2022 school year, local education agencies (LEAs) must develop resource allocation strategies to support schools in need. Federal programs have provided resources to support restart and recovery. However, there is little consensus on how LEAs can target resources to support those schools most in need. This study investigates the relationship
-
Towards an Antiracist Classroom Formative Assessment Framework Educational Assessment Pub Date : 2022-08-05 Jade Caines Lee
ABSTRACT The purpose of this paper is to theoretically explore how Hip Hop pedagogy can be utilized and implemented in K-12 classroom formative assessment practices. As a conceptual paper, there will be five sections. The first section will explore classroom formative assessment definitions and highlight theoretical frameworks used to conceptualize the pedagogical practice. The next section will begin
-
Indicators of Equitable Civic Learning in U.S. Public Schools Educational Assessment Pub Date : 2022-08-05 Laura S. Hamilton, Julia H. Kaufman
ABSTRACT The “twin pandemics” of racial injustice and COVID-19 have underscored the importance of promoting civic knowledge, skills, dispositions, and engagement among the nation’s young people. Although some evidence has demonstrated that civic-learning opportunities are inequitably distributed across U.S. schools and communities, we currently have limited data that could help inform efforts to promote
-
Two Communities’ Views on Test Fairness Educational Assessment Pub Date : 2022-08-05 Drew H. Gitomer, Emi Iwatani
ABSTRACT The education measurement community has centered the idea of test fairness in both theory and practice. Yet, racial justice advocates in education research and practice (the racial justice community) have consistently critiqued that assessments are hardly fair and play a critical and outsized role in contributing to racial and social inequities in the educational system and larger society
-
The Five Faces of an Assessor: Conceptualizing the Enactment of Teacher Assessment Identity in Vocational Education and Training Educational Assessment Pub Date : 2022-08-03 Julie Klovholt Leonardsen, Britt Karin Støen Utvær, Henning Fjørtoft
ABSTRACT Teacher assessment practice is affected by a complex set of cognitive and affective traits, as well as institutional contexts. There is a dearth of research on sociocultural influences on teachers’ assessment identity. This study presents a model illustrating how vocational education and training (VET) teachers enact their assessment identity across contexts. We draw on interview data from
-
Preferences, Substitutes, and Complements: Mapping the Relationship between Data Sources within Teachers’ Instructional Practice Educational Assessment Pub Date : 2022-06-28 Austin S. Jennings
ABSTRACT The extent to which teachers collect, interpret, and use information from multiple data sources is a key distinction between novice and expert data users. Understanding and exploring this dimension of teachers’ instructional decision making requires a shift in contemporary perspectives toward the interconnectedness of data sources within teachers’ practice of data use. In the present study
-
Representations of Disciplinary Knowledge in Assessment: Associations between High School and University Assessments in Science, Mathematics and the Humanities and Predictors of Success Educational Assessment Pub Date : 2022-06-24 Michael Johnston, Bronwyn Elisabeth Wood, Sue Cherrington, Suzanne Boniface, Anita Mortlock
ABSTRACT In this paper, we report an exploratory investigation of the types of assessment in a variety of high school subjects that best predicted subsequent success at university in those subjects. In social sciences and the humanities, internal (school-based) assessment was a better predictor of university success than external, examination-based assessment, whereas the converse was true for mathematics
-
Improving Measurement of Teacher Performance: Alternative Scoring for Classroom-Based Observational Systems Educational Assessment Pub Date : 2022-06-24 Ryan J. Kettler, Anh Hua, Christopher M. Dudek, Linda A. Reddy, Ilona Arnold-Berkovits, Nicole B. Wiggs, Adam Lekwa, Alexander Kurz
ABSTRACT The study examines reliability and validity evidence of observational systems for evaluating teacher effectiveness and fostering professional development conversations in schools. Specifically, this study compared the Framework for Teaching’s (FFT) validity evidence using the traditional scoring approach with a new composite scoring approach that averages the components nested within each
-
Assessment in the Time of COVID-19: Understanding Patterns of Student Disengagement during Remote Low-Stakes Testing Educational Assessment Pub Date : 2022-06-10 Steven L. Wise, Megan R. Kuhfeld, John Cronin
ABSTRACT The arrival of the COVID-19 pandemic had a profound effect on K-12 education. Most schools transitioned to remote instruction, and some used remote testing to assess student learning. Remote testing, however, is less controlled than in-school testing, leading to concerns regarding test-taking engagement. This study compared the disengagement of students remotely administered an adaptive interim
-
Health Education Teachers’ Assessment Conceptions and Practices: Identifying Assessment Profiles Educational Assessment Pub Date : 2022-04-21 Olli Paakkari, Leena Paakkari, Henna Haapala, Mirja Hirvensalo
ABSTRACT The study explored the latent construct underlying the assessment conceptions and practices of Finnish Health Education teachers (n = 165) in the context of curricula, seeking thereby to identify the teachers’ assessment profiles. Six underlying factors were found to encompass their assessment conceptions and practices, namely Assessment supporting learning, Assessment of working, Self and
-
Developing Test Performance Communication Solutions in a Teacher-Researcher Partnership Educational Assessment Pub Date : 2022-03-17 Chad M. Gotch, Mary Roduta Roberts
ABSTRACT Individual-level score reports represent a common artifact in teacher-parent communication about standardized tests. Previous research has documented challenges in communicating student achievement. Researchers have also leveraged teachers in the process of score report design. Little is known, however, about teachers’ experiences with using score reports in authentic settings. In this study
-
Student Engagement on the National Assessment of Educational Progress (NAEP): A Systematic Review and Meta-Analysis of Extant Research Educational Assessment Pub Date : 2022-03-06 Allison J. LaFave, Joseph A. Taylor, Amelia M. Barter, Ariel S. Jacobs
ABSTRACT This systematic review examines empirical research about students’ motivation for NAEP in grades 4, 8, and 12 using multiple motivation constructs, including effort, value, and expectancy. Analyses yielded several findings. First, there are stark differences in the perceived importance of doing well on NAEP among students in grades 4 (86%), 8 (59%), and 12 (35%). Second, meta-analyses of descriptive
-
The Effect of Device Type on Achievement: Evidence from a Quasi-Experimental Design Educational Assessment Pub Date : 2022-03-02 David Rutkowski, Leslie Rutkowski, Charity Flores
ABSTRACT As more states move to universal computer-based assessments, an emergent issue concerns the effect that device type might have on student results. Although, several research studies have explored device effects, most of these studies focused on the differences between tablets and desktops/laptops. In the current study, we distinguish between different types of devices to better examine the
-
Disrupting White Supremacy in Assessment: Toward a Justice-Oriented, Antiracist Validity Framework Educational Assessment Pub Date : 2022-02-17 Jennifer Randall, David Slomp, Mya Poe, Maria Elena Oliveri
ABSTRACT In this article, we propose a justice-oriented, antiracist validity framework designed to disrupt assessment practices that continue to (re)produce racism through the uncritical promotion of white supremist hegemonic practices. Using anti-Blackness as illustration, we highlight the ways in which racism is introduced, or ignored, in current assessment and validation processes and how an antiracist
-
Investigating the Effect of the Programme of Study on University Students’ Perceptions about Assessment Educational Assessment Pub Date : 2022-01-23 Diana Pereira, Irene Cadime, Maria Assunção Flores, Cláudia Pinheiro, Patrícia Santos
ABSTRACT This study focuses on the effect of the programme variable on the purposes and effects that students associate with assessment, on the assessment methods used and on the perceived use of assessment. Data were collected in five Portuguese Public Universities through a survey (n = 4144) and focus group (n = 250) with students enrolled in different programmes. Findings point to statistically
-
Multimodal Tasks to Assess English Learners and Their Peers in Science Educational Assessment Pub Date : 2022-01-17 Scott E. Grapin, Lorena Llosa
ABSTRACT Traditionally, content assessments have been carried out through written language. However, the latest standards in U.S. K-12 education expect all students, including English learners (ELs), to demonstrate their content learning using multiple modalities. This study examined the performance of fifth-grade students at varying levels of English proficiency on four science tasks that elicited
-
Language Matters: Teacher and Parent Perceptions of Achievement Labels from Educational Tests Educational Assessment Pub Date : 2021-12-29 Francis O’Donnell, Stephen G. Sireci
ABSTRACT Since the standards-based assessment practices required by the No Child Left Behind legislation, almost all students in the United States are “labeled” according to their performance on educational achievement tests. In spite of their widespread use in reporting test results, research on how achievement level labels are perceived by teachers, parents, and students is minimal. In this study
-
Assessing Source Evaluation Skills of Middle School Students Using Learning Progressions Educational Assessment Pub Date : 2021-09-01 Jesse R. Sparks, Peter W. van Rijn, Paul Deane
ABSTRACT Effectively evaluating the credibility and accuracy of multiple sources is critical for college readiness. We developed 24 source evaluation tasks spanning four predicted difficulty levels of a hypothesized learning progression (LP) and piloted these tasks to evaluate the utility of an LP-based approach to designing formative literacy assessments. Sixth, seventh, and eighth grade students
-
A Methodology for Determining and Validating Latent Factor Dimensionality of Complex Multi-Factor Science Constructs Measuring Knowledge-In-Use Educational Assessment Pub Date : 2021-09-05 Leonora Kaldaras, Hope Akaeze, Joseph Krajcik
ABSTRACT Deep science understanding is reflected in students’ ability to use content and skills when making sense of the world. Assessing deep understanding requires measuring complex constructs that combine elements of content and skills. To develop valid measures of complex constructs, we need to understand how their theoretical dimensionality, reflected in the integration of content and skills,
-
Investigating the Effects of Test Accommodations with Process Data for English Learners in a Mathematics Assessment Educational Assessment Pub Date : 2021-09-29 Mikyung Kim Wolf, Hanwook Yoo, Danielle Guzman-Orth, Jamal Abedi
ABSTRACT Implementing a randomized controlled trial design, the present study investigated the effects of two types of accommodations, linguistic modification and a glossary, for English learners (ELs) taking a computer-based mathematics assessment. Process data including response time and clicks on glossary words were also examined to better interpret students’ interaction with the accommodations
-
An Intersectional Approach to DIF: Do Initial Findings Hold across Tests? Educational Assessment Pub Date : 2021-08-22 Michael Russell, Olivia Szendey, Larry Kaplan
ABSTRACT Differential Item Function (DIF) analysis is commonly employed to examine potential bias produced by a test item. Since its introduction DIF analyses have focused on potential bias related to broad categories of oppression, including gender, racial stratification, economic class, and ableness. More recently, efforts to examine the effects of oppression on valued life-outcomes have employed
-
Investigating the Use of Assessment Data by Primary School Teachers: Insights from a Large-scale Survey in Ireland Educational Assessment Pub Date : 2021-06-27 Vasiliki Pitsia, Anastasios Karakolidis, Paula Lehane
ABSTRACT Evidence suggests that the quality of teachers’ instructional practices can be improved when these are informed by relevant assessment data. Drawing on a sample of 1,300 primary school teachers in Ireland, this study examined the extent to which teachers use standardized test results for instructional purposes as well as the role of several factors in predicting this use. Specifically, the
-
The Impact of Disengaged Test Taking on a State’s Accountability Test Results Educational Assessment Pub Date : 2021-07-27
ABSTRACT This study investigated test-taking engagement on the Spring 2019 administration of a large-scale state summative assessment. Through the identification of rapid-guessing behavior – which is a validated indicator of disengagement – the percentage of Grade 8 test events with meaningful amounts of rapid guessing was 5.5% in mathematics, 6.7% in English Language Arts (ELA), and 3.5% in science
-
Using Full-information Item Analysis to Improve Item Quality Educational Assessment Pub Date : 2021-07-22
ABSTRACT Full-information item analysis provides item developers and reviewers comprehensive empirical evidence of item quality, including option response frequency, point-biserial index (PBI) for distractors, mean-scores of respondents selecting each option, and option trace lines. The multi-serial index (MSI) is introduced as a more informative item-total correlation, accounting for variable distractor
-
Beyond Agreement: Exploring Rater Effects in Large-Scale Mixed Format Assessments Educational Assessment Pub Date : 2021-08-17 Stefanie A. Wind, Wenjing Guo
ABSTRACT Scoring procedures for the constructed-response (CR) items in large-scale mixed-format educational assessments often involve checks for rater agreement or rater reliability. Although these analyses are important, researchers have documented rater effects that persist despite rater training and that are not always detected in rater agreement and reliability analyses, such as severity/leniency
-
Anchors Aweigh: How the Choice of Anchor Items Affects the Vertical Scaling of 3PL Data with the Rasch Model Educational Assessment Pub Date : 2021-01-20 Glenn Thomas Waterbury, Christine E. DeMars
ABSTRACT Vertical scaling is used to put tests of different difficulty onto a common metric. The Rasch model is often used to perform vertical scaling, despite its strict functional form. Few, if any, studies have examined anchor item choice when using the Rasch model to vertically scale data that do not fit the model. The purpose of this study was to investigate the implications of anchor item choice
-
Model meets reality: Validating a new behavioral measure for test-taking effort Educational Assessment Pub Date : 2021-01-12 Esther Ulitzsch, Christiane Penk, Matthias von Davier, Steffi Pohl
ABSTRACT Identifying and considering test-taking effort is of utmost importance for drawing valid inferences on examinee competency in low-stakes tests. Different approaches exist for doing so. The speed-accuracy+engagement model aims at identifying non-effortful test-taking behavior in terms of nonresponse and rapid guessing based on responses and response times. The model allows for identifying rapid-guessing
-
Do They See What I See? Toward a Better Understanding of the 7Cs Framework of Teaching Effectiveness Educational Assessment Pub Date : 2021-01-05 Sarah F. Phillips, Ronald F. Ferguson, Jacob F. S. Rowley
ABSTRACT School systems are increasingly incorporating student perceptions of teaching effectiveness into educator accountability systems. Using Tripod’s 7Cs™ Framework of Teaching Effectiveness, this study examines key issues in validating student perception data for use in this manner. Analyses examine the internal structure of 7Cs scores and the extent to which scores predict key criteria. Results
-
Assessing Quality of Teaching from Different Perspectives: Measurement Invariance across Teachers and Classes Educational Assessment Pub Date : 2020-12-18 Georg Krammer, Barbara Pflanzl, Gerlinde Lenske, Johannes Mayr
ABSTRACT Comparing teachers’ self-assessment to classes’ assessment of quality of teaching can offer insights for educational research and be a valuable resource for teachers’ continuous professional development. However, the quality of teaching needs to be measured in the same way across perspectives for this comparison to be meaningful. We used data from 622 teachers self-assessing aspects of quality
-
The Effect of Linguistic Factors on Assessment of English Language Learners’ Mathematical Ability: A Differential Item Functioning Analysis Educational Assessment Pub Date : 2020-12-17 Stephanie Buono, Eunice Eunhee Jang
ABSTRACT Increasing linguistic diversity in classrooms has led researchers to examine the validity and fairness of standardized achievement tests, specifically concerning whether test score interpretations are free of bias and score use is fair for all students. This study examined whether mathematics achievement test items that contain complex language function differently between two language subgroups:
-
The Road to Retention Passes through First Year Academic Performance: A Meta-Analytic Path Analysis of Academic Performance and Persistence Educational Assessment Pub Date : 2020-12-12 Paul A. Westrick, Frank L. Schmidt, Huy Le, Steven B. Robbins, Justine M.R. Radunzel
ABSTRACT This meta-analytic path analysis presents evidence that first-year academic performance (FYAP), measured by first-year grade point average (FYGPA) plays the major role in determining second-year student retention and that socioeconomic status (SES), measured by parental income, plays a negligible role. Based on large sample data used in a previous study, we conducted additional analyses that
-
A Confirmatory Item Factor Analysis of A School Climate Measure for Adolescents with and without Disabilities Educational Assessment Pub Date : 2020-11-19 Graham Rifenbark, Allison Lombardi, Jennifer Freeman
ABSTRACT In this study (n = 5037) we further investigated the construct validity of the Georgia Brief School Climate Inventory (GBS). Despite containing ordinal items (e.g., ordered-responses), the GBS has only been examined using linear approaches. To fill this void, we employed confirmatory item factor analysis (IFA) to examine the factor structure because the observed data are polytomous. Using
-
Using an Index of Admission Obstacles with Constrained Optimization to Increase the Diversity of College Classes Educational Assessment Pub Date : 2020-11-05 Rebecca Zwick, Andrew Blatter, Lei Ye, Steven Isham
ABSTRACT Today, postsecondary institutions in the US typically wish to enroll entering classes that are both academically qualified and diverse. Although the definition of diversity varies from school to school, the challenge is essentially the same: How can academic objectives be combined with goals that involve the composition of the entering class? Many schools have a commitment to facilitating
-
Use of Response Process Data to Inform Group Comparisons and Fairness Research Educational Assessment Pub Date : 2020-09-22 Kadriye Ercikan, Hongwen Guo, Qiwei He
ABSTRACT Comparing group is one of the key uses of large-scale assessment results, which are used to gain insights to inform policy and practice and to examine the comparability of scores and score meaning. Such comparisons typically focus on examinees’ final answers and responses to test questions, ignoring response process differences groups may engage in. This paper discusses and demonstrates the
-
Test Takers’ Response Tendencies in Alternative Item Formats: A Cognitive Science Approach Educational Assessment Pub Date : 2020-08-13 Jung Aa Moon, Madeleine Keehner, Irvin R. Katz
ABSTRACT We investigated how item formats influence test takers’ response tendencies under uncertainty. Adult participants solved content-equivalent math items in three formats: multiple-selection multiple-choice, grid with forced-choice (true-false) options, and grid with non-forced-choice options. Participants showed a greater tendency to commit (rather than omit) responses in the grid items, in
-
Implications of considering Response Process Data for Greater and Lesser Psychometrics Educational Assessment Pub Date : 2020-08-13 Roy Levy
ABSTRACT This paper characterizes the ways in which increased attention to response process data has implications for psychometrics. To do so, this work draws on two organizing frameworks that have heretofore not been associated: evidence-centered design, and the distinction between greater and lesser statistics. Overlaying these frameworks leads to a conceptualization of greater and lesser psychometrics