A recent paper evaluating a new rubric-based graduate admissions approach using generic methods tentatively suggested that its decisions differed noticeably from the previous approach in an unspecified way. Using prior knowledge that the often-stated specific goal was to open a path to increased diversity by reducing barriers to the admission of applicants with low undergraduate grade point averages and graduate record exam (GRE) scores allows simple statistical tests of changes in the distributions of the metrics. The simple tests confirm with good statistical confidence that the barrier-reduction changes were achieved. Nevertheless, the paper’s argument that the de-emphasized tests and grades are not predictive of graduate outcomes is not supported by the prior literature. On technique, although a method used in some of the analyses for dropping data points before running the machine algorithm is likely to bias those results, it helps to clarify why models of the rubric-based system were only weakly predictive.

• Received 8 June 2023
• Accepted 9 November 2023

DOI:https://doi.org/10.1103/PhysRevPhysEducRes.19.028001

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society