Abstract
Intrinsic inequity in assessment refers to sources of harmful discrimination inherent in the design of assessment tools and systems. This study seeks to understand intrinsic inequity in assessment systems by studying assessment policies and associated procedures in residency training, using general pediatrics as a discourse case study. Foucauldian discourse analysis (FDA) was conducted on assessment policy and procedure documents. Two authors independently prepared structured analytic notes using guiding questions. Documents and respective analytic notes were subsequently reviewed independently by all authors. Each author prepared further unstructured analytic notes on the documents’ discourse. The authors then compared notes and constructed truth statements (i.e., interpretations of what the discourse establishes as true about the construct under study) and sub-strands (i.e., themes) that were repeated and legitimized across the documents via iterative discussion. Based on analysis, the authors constructed two truth statements. These truth statements, “good assessment is equitable assessment,” and “everyone is responsible for inequity,” conceptualized inequity in assessment as an isolated or individual-level aberration in an otherwise effective or neutral system. Closer examination of the truth statements and sub-strands in the discourse presented an alternative view, suggesting that inequity may in fact not be an aberration but rather an inherent feature of assessment systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ABP Corporate Policy Applicants with Disabilities. (2021). https://www.abp.org/sites/abp/files/policy-applicants-with-disabilities.pdf.
Accreditation Council for Graduate Medical Education. (2022). ACGME Program Requirements for Graduate Medical Education in Pediatrics
American Board of Pediatrics: Diversity, Equity, and Inclusion (DEI). (2022). Retrieved April 1, 2022, from https://www.abp.org/content/dei
Association of Pediatric Program Director’s Year in the Program: Guide and Resources for the Program Director and Coordinator. (2008)
Bartlett, K., Pentsil, S., Ansong-Assoku, B., & Woods, S. (2013). Faculty development series on assessment in graduate medical education: Interpersonal and communication skills. MedEdPORTAL, 9, 9339
Bergman, E., de Feijter, J., Frambach, J., et al. (2012). AM last page: A Guide to research paradigms relevant to medical education. Academic Medicine, 87(4), 545. https://doi.org/10.1097/ACM.0b013e31824fbc8a
Bhavaraju, V. L., Bartlett, K., Guillot, A., Schumacher, D. The milestone project. (2012). Faculty development series on assessment in graduate medical education presented at: Association of pediatric program directors
Bok, H. G., Teunissen, P. W., Favier, R. P., Rietbroek, N. J., Theyse, L. F., Brommer, H., Haarhuis, J., van Beukelen, P., van der Vleuten, C. P., & Jaarsma, D. A. (2013). Programmatic assessment of competency-based workplace learning: When theory meets practice. BMC Medical Education, 13(1), 123. https://doi.org/10.1186/1472-6920-13-123
Cushman E. (2016). Decolonizing validity. Journal of Writing Assessment 9(1)
DeLisa, J. A., & Thomas, P. (2005). Physicians with disabilities and the physician workforce: A need to reassess our policies. American Journal of Physical Medicine and Rehabilitation, 84(1), 5–11. https://doi.org/10.1097/01.PHM.0000153323.28396.DE
Diversity, Equity, Inclusion: Our DEI Work. (2020). American Board of Pediatrics https://www.abp.org/content/dei
Edgar, L., McLean, S., Hogan, S. O., Hamstra, S., & Holmboe, E. S. (2020). The milestones guidebook. Accreditation Council for Graduate Medical Education
Entrustable Professional Activities Vignettes. (2021)
Equity/Inequity. (2016). Glossary of Education Reform https://www.edglossary.org/equity/
Flores, N., & Rosa, J. (2022). Undoing competence: Coloniality, homogeneity, and the overrepresentation of whiteness in applied linguistics. Lang Learn. Published Online. https://doi.org/10.1111/lang.12528
Foucault, M. (1980). Power/Knowledge. Pantheon
Foucault, M. (1975). The means of correct training. In Discipline and punish. Pantheon Books
Gee, J. P. (2012). An introduction to discourse analysis: Theory and method. Routledge, Retrived from July 17, 2022. http://public.eblib.com/choice/publicfullrecord.aspx?p=201209
Gee, J. P. (2015). Discourse: Big D, Small d.pdf
Gergen, K. J., & Dixon-Román, E. J. (2014). Social epistemology and the pragmatics of assessment. Teacher College Record Voice Scholarsh Education, 116(11), 1–22. https://doi.org/10.1177/016146811411601111
Goldhagen, J., & Waterston, T. (2003). Conclusion: The equity project. Pediatrics, 112(Supplement_3), 771–772. https://doi.org/10.1542/peds.112.S3.771
Harris, P., Bhanji, F., Topps, M., et al. (2017). Evolving concepts of assessment in a competency-based world. Medical Teacher, 39(6), 603–608. https://doi.org/10.1080/0142159X.2017.1315071
Ho, M. J., Yu, K. H., Hirsh, D., Huang, T. S., & Yang, P. C. (2011). Does one size fit all? Building a framework for medical professionalism. Academic Medicine, 86(11), 1407–1414. https://doi.org/10.1097/ACM.0b013e31823059d1
Hodges, B. D., Martimianakis, M. A., McNaughton, N., & Whitehead, C. (2014). medical education… meet Michel Foucault. Medical Teacher, 48(6), 563–571. https://doi.org/10.1111/medu.12411
Holmboe, E. S., & Iobst, W. F. (2020). Assessment guidebook. ACGME
Johnston, J. L. (2014). Resistance is not futile: Foucault and a brave new world of medical education research. Medical Education, 48(6), 554–555. https://doi.org/10.1111/medu.12476
Kenney, J., & Sendelbach, D. (2011). Teaching and assessing professionalism: Faculty development slides. Presented at: pediatrics faculty development series: 2011–2012 faculty promotion of professionalism
Khan, T. H., & MacEachen, E. (2021). Foucauldian discourse analysis: Moving beyond a social constructionist analytic. International Journal of Qualitative Methods, 20, 160940692110180. https://doi.org/10.1177/16094069211018009
Klein, R., Ufere, N. N., Rao, S. R., Koch, J., Volerman, A., Snyder, E. D., Schaeffer, S., Thompson, V., Warner, A. S., Julian, K. A., & Palamara, K. (2020). Association of gender with learner assessment in graduate medical education. JAMA Network Open, 3(7), e2010888. https://doi.org/10.1001/jamanetworkopen.2020.10888
Klein, R., Ufere, N. N., Schaeffer, S., Julian, K. A., Rao, S. R., Koch, J., Volerman, A., Snyder, E. D., Thompson, V., Ganguli, I., & Burnett-Bowie, S. A. M. (2022). Association between resident race and ethnicity and clinical performance assessment scores in graduate medical education. Academic Medicine, 97(9), 1351–1359. https://doi.org/10.1097/ACM.0000000000004743
Kuper, A., Whitehead, C., & Hodges, B. D. (2013). Looking back to move forward: Using history, discourse and text in medical education research: AMEE Guide No. 73. Medical Teacher, 35(1), e849–e860. https://doi.org/10.3109/0142159X.2012.748887
Lefebvre, C., Williamson, K., Moffett, P., et al. (2018). Legal considerations in the remediation and dismissal of graduate medical trainees. Journal of Graduate Medical Education, 10(3), 253–257. https://doi.org/10.4300/JGME-D-17-00813.1
Ling, L., Derstine, P., & Cohen, N. (2013). Implementing milestones and clinical competency committees
Lucey, C. R., Hauer, K. E., Boatright, D., & Fernandez, A. (2020). Medical education’s wicked problem: Achieving equity in assessment for medical learners. Academic Medicine, 95(12S), S98–S108. https://doi.org/10.1097/ACM.0000000000003717
Manthous, C. A. (2014). On the outcome project. Yale Journal of Biology and Medicine, 87(2), 213–220
Martin, R. (1988). Truth, power, self: An interview with Michel Foucault, October 25 1982. In M. Foucault (Ed.), Technologies of the self: A seminar with michel foucault. University of Massachusetts Press
McFarling, U. L. (2022a). ‘It was stolen from me’: Black doctors are forced out of training programs at far higher rates than white residents. Stat News
McFarling, U. L. (2022b). What will it take to level the playing field for Black residents? Stat News
Mickleborough, T. O., & Martimianakis, M. A. T. (2021). (Re)producing “whiteness” in health care: A spatial analysis of the critical literature on the integration of internationally educated health care professionals in the Canadian workforce. Academic Medicine, 96(11), S31–S38. https://doi.org/10.1097/ACM.0000000000004262
Milestones Vignettes for Faculty Development. (2015)
Norcini, J., Anderson, B., Bollela, V., et al. (2011). Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 conference. Medical Teacher, 33(3), 206–214. https://doi.org/10.3109/0142159X.2011.551559
Norcini, J., Anderson, M. B., Bollela, V., et al. (2018). 2018 consensus framework for good assessment. Medical Teacher, 40(11), 1102–1109. https://doi.org/10.1080/0142159X.2018.1500016
Olmos-Vega, F. M., Stalmeijer, R. E., Varpio, L., & Kahlke, R. (2022). A practical guide to reflexivity in qualitative research: AMEE Guide No. 149. Medical Teacher. https://doi.org/10.1080/0142159X.2022.2057287
Program Director’s Guidebook to the American Board of Pediatrics. (2016)
Randall, J. (2021). “Color-neutral” is not a thing: Redefining construct definition and representation through a justice-oriented critical antiracist lens. Educational Measurement: Issues and Practice, 40(4), 82–90. https://doi.org/10.1111/emip.12429
Residency Competency Committee (RCC) Handbook. (2014)
Rojek, A. E., Khanna, R., Yim, J. W., Gardner, R., Lisker, S., Hauer, K. E., Lucey, C., & Sarkar, U. (2019). Differences in narrative language in evaluations of medical students by gender and under-represented minority status. Journal of General Internal Medicine, 34(5), 684–691. https://doi.org/10.1007/s11606-019-04889-9
Sample Residency Mentor Manual. (2011)
Schwartz, A. (2011). Assessment in graduate medical education: A primer for pediatric program directors. American Board of Pediatrics
Stobart, G. (2005). Fairness in multicultural assessment systems. Assess Education Principles, Policy & Practice, 12(3), 275–287. https://doi.org/10.1080/09695940500337249
Sukhera, J., Goez, H., Brown, A., Haddara, W., & Razack, S. (2022). Freedom from discrimination or freedom to discriminate? Discursive tensions within discrimination policies in medical education. Advances in Health Sciences Education. https://doi.org/10.1007/s10459-022-10090-1
Teaching and Assessing Professionalism: A Program Director’s Guide. (2012)
Teherani, A., Hauer, K. E., Fernandez, A., King, T. E., & Lucey, C. (2018). how small differences in assessed clinical performance amplify to large differences in grades and awards: A Cascade with serious consequences for students underrepresented in medicine. Academic Medicine, 93(9), 1286–1292. https://doi.org/10.1097/ACM.0000000000002323
Teherani, A., Perez, S., Muller-Juge, V., Lupton, K., & Hauer, K. E. (2020). A narrative study of equity in clinical assessment through the antideficit lens. Academic Medicine, 95(12S), S121–S130. https://doi.org/10.1097/ACM.0000000000003690
The Program Director Guide to the Common Program Requirements (Residency). (2021)
The Program Coordinators’ Handbook_extracted.pdf. (2015)
The Pediatrics Milestones Project 2015. (2015)
The Pediatrics Milestones Project 2021. (2021)
Unaka N, Lurie, B, Winn A, Myers R, Lieberman R, Grow M. (2019) Associate program directors handbook
Understanding the American Board of Pediatrics Training and Evaluation Process. (2019)
Unknown. (2007). Sample Program Evaluation Survey
Vassie, C., Smith, S., & Leedham-Green, K. (2020). Factors impacting on retention, success and equitable participation in clinical academic careers: A scoping review and meta-thematic synthesis. British Medical Journal Open, 10(3), e033480. https://doi.org/10.1136/bmjopen-2019-033480
Van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Govaerts, M. J. B., & Heeneman, S. (2015). Twelve tips for programmatic assessment. Medical Teacher, 37(7), 641–646. https://doi.org/10.3109/0142159X.2014.973388
Westervelt, M., Billingsley, D., London, M., & Fancher, T. (2021). Three things schools should do to make advancement assessment just. AMA Journal of Ethics, 23(12), 937–945. https://doi.org/10.1001/amajethics.2021.937
Whitehead, C. R., Austin, Z., & Hodges, B. D. (2011). Flower power: The armoured expert in the CanMEDS competency framework? Advances in Health Sciences Education, 16(5), 681–694. https://doi.org/10.1007/s10459-011-9277-4
Wodak, R. E., & Meyer, M. (2001). Methods of critical discourse analysis. SAGE Publications Ltd. https://doi.org/10.4135/9780857028020
Zaidi, Z., Bush, A. A., Partman, I. M., & Wyatt, T. R. (2021). From the “top-down” and the “bottom-up”: Centering Foucault’s notion of biopower and individual accountability within systemic racism. Perspectives on Medical Education., 10(2), 73–75. https://doi.org/10.1007/s40037-021-00655-y
Author information
Authors and Affiliations
Contributions
HA wrote the main manuscript text. LA prepared tables and Fig. 1. All authors conducted study design, analysis, and review and editing of manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Anderson, H.L., Abdulla, L., Balmer, D.F. et al. Inequity is woven into the fabric: a discourse analysis of assessment in pediatric residency training. Adv in Health Sci Educ 29, 199–216 (2024). https://doi.org/10.1007/s10459-023-10260-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-023-10260-9