Skip to main content

Advertisement

SpringerLink
Log in

Implicit versus explicit first impressions in performance-based assessment: will raters overcome their first impressions when learner performance changes?

  • Research
  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

First impressions can influence rater-based judgments but their contribution to rater bias is unclear. Research suggests raters can overcome first impressions in experimental exam contexts with explicit first impressions, but these findings may not generalize to a workplace context with implicit first impressions. The study had two aims. First, to assess if first impressions affect raters’ judgments when workplace performance changes. Second, whether explicitly stating these impressions affects subsequent ratings compared to implicitly-formed first impressions. Physician raters viewed six videos where learner performance either changed (Strong to Weak or Weak to Strong) or remained consistent. Raters were assigned two groups. Group one (n = 23, Explicit) made a first impression global rating (FIGR), then scored learners using the Mini-CEX. Group two (n = 22, Implicit) scored learners at the end of the video solely with the Mini-CEX. For the Explicit group, in the Strong to Weak condition, the FIGR (M = 5.94) was higher than the Mini-CEX Global rating (GR) (M = 3.02, p < .001). In the Weak to Strong condition, the FIGR (M = 2.44) was lower than the Mini-CEX GR (M = 3.96 p < .001). There was no difference between the FIGR and the Mini-CEX GR in the consistent condition (M = 6.61, M = 6.65 respectively, p = .84). There were no statistically significant differences in any of the conditions when comparing both groups’ Mini-CEX GR. Therefore, raters adjusted their judgments based on the learners’ performances. Furthermore, raters who made their first impressions explicit showed similar rater bias to raters who followed a more naturalistic process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Ambady, N., Bernieri, F. J., & Richeson, J. A. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. Advances in experimental social psychology (Vol. 32, pp. 201–271). Academic Press.

  • Brown, M. H., Regehr, G., & Reznick, R. K. (1996). The effect of early performance on examiners’ marking patterns during an oral examination. Academic Medicine, 71(1), S73–S75.

    Article  Google Scholar 

  • Carney, D. R., Colvin, C. R., & Hall, J. A. (2007). A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, 41(5), 1054–1072.

    Article  Google Scholar 

  • Cook, D. A., Beckman, T. J., Mandrekar, J. N., & Pankratz, V. S. (2010). Internal structure of mini-CEX scores for internal medicine residents: Factor analysis and generalizability. Advances in Health Sciences Education, 15(5), 633–645. https://doi.org/10.1007/s10459-010-9224-9.

    Article  Google Scholar 

  • Crossley, J., & Jolly, B. (2012). Making sense of work-based assessment: Ask the right questions, in the right way, about the right things, of the right people. Medical Education, 46(1), 28–37. https://doi.org/10.1111/j.1365-2923.2011.04166.x.

    Article  Google Scholar 

  • Dipboye, R. L. (1982). Self-fulfilling prophecies in the selection-recruitment interview. The Academy of Management Review, 7(4), 579–586. https://doi.org/10.2307/257224

  • Dory, V., Danoff, D., Plotnick, L. H., Cummings, B. A., Gomez-Garibello, C., Pal, N. E., & Young, M. (2021). Does Educational Handover Influence subsequent Assessment? Academic Medicine, 96(1), 118–125. https://doi.org/10.1097/ACM.0000000000003528.

    Article  Google Scholar 

  • Earl-Slater, A. (2002). The handbook of clinical trials and other research. Radcliffe Publishing.

  • Gauthier, G., St-Onge, C., & Tavares, W. (2016). Rater cognition: Review and integration of research findings. Medical Education, 50(5), 511–522. https://doi.org/10.1111/medu.12973.

    Article  Google Scholar 

  • Gingerich, A., Regehr, G., & Eva, K. W. (2011). Rater-based assessments as social judgments: Rethinking the etiology of rater errors. Academic Medicine, 86(10 Suppl), https://doi.org/10.1097/ACM.0b013e31822a6cf8. S1-7.

  • Gingerich, A., Kogan, J., Yeates, P., Govaerts, M., & Holmboe, E. (2014). Seeing the ‘black box’differently: assessor cognition from three research perspectives. Medical Education, 48(11), 1055–1068. https://doi.org/10.1111/medu.12546.

    Article  Google Scholar 

  • Gingerich, A., Ramlo, S. E., van der Vleuten, C. P. M., Eva, K. W., & Regehr, G. (2017). Inter-rater variability as mutual disagreement: Identifying raters’ divergent points of view. Advances in Health Sciences Education, 22(4), 819–838. https://doi.org/10.1007/s10459-016-9711-8.

    Article  Google Scholar 

  • Govaerts, M. J. B., Van de Wiel, M. W. J., Schuwirth, L. W. T., & Van der Vleuten, C. P. M., &amp Muijtjens, M. M. (2013). Workplace-based assessment: Raters’ performance theories and constructs. Advances in Health Sciences Education, 18(3), 375–396. https://doi.org/10.1007/s10459-012-9376-x.

  • Govaerts, M. J. B., van der Vleuten, C. P. M., & Holmboe, E. S. (2018). Managing tensions in assessment: Moving beyond either–or thinking. Medical Education, 64–75. https://doi.org/10.1111/medu.13656.

  • Hatala, R., Ainslie, M., Kassen, B. O., Mackie, I., & Roberts, J. M. (2006). Assessing the mini-clinical evaluation exercise in comparison to a national specialty examination. Medical Education, 40(10), 950–956. https://doi.org/10.1111/j.1365-2929.2006.02566.x.

    Article  Google Scholar 

  • Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32, 676–682. https://doi.org/10.3109/0142159X.2010.500704.

    Article  Google Scholar 

  • Klusmann, D., Knorr, M., & Hampe, W. (2023). Exploring the relationships between first impressions and MMI ratings: A pilot study. Advances in Health Sciences Education, 28(2), 519–536.

    Article  Google Scholar 

  • Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009). Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. Journal of the American Medical Association (JAMA), 302(12), 1316–1326.

    Article  Google Scholar 

  • Kogan, J., Conforti, L., Bernabeo, E., Iobst, W., & Holmboe, E. (2011). Opening the black box of clinical skills assessment via observation: A conceptual model. Medical Education, 45(10), 1048–1060. https://doi.org/10.1111/j.1365-2923.2011.04025.x.

    Article  Google Scholar 

  • Kogan, J. R., Conforti, L. N., Iobst, W. F., & Holmboe, E. S. (2014). Reconceptualizing variable rater assessments as both an educational and clinical care problem. Academic Medicine, 89(5), 721–727. https://doi.org/10.1097/ACM.0000000000000221.

    Article  Google Scholar 

  • Macan, T. H., & Dipboye, R. L. (1990). The relationship of interviewer’s preinterview impressions to selection and recruitment outcomes. Personnel Psychology, 43(4), 745–768. https://doi.org/10.1111/j.1744-6570.1990.tb00681.x.

    Article  Google Scholar 

  • Norcini, J. J., Blank, L., Arnold, G., & Kimball, H. (1995). The mini-CEX (clinical evaluation exercise): A preliminary investigation. Annals of Internal Medicine, 123, 795–799.

    Article  Google Scholar 

  • Norman, G. (2014). The Bias in researching cognitive bias. Advances. Health Sciences Education: Theory and Practice, 19(3), 291–295. https://doi.org/10.1007/s10459-014-9517-5.

    Article  Google Scholar 

  • Shaw, T., Wood, T. J., Touchie, C., Pugh, D., & Humphrey-Murto, S. (2021). How biased are you ? The effect of prior performance information on attending physician ratings and implications for learner handover. Advances in Health Sciences Education, 26(1), 199–214. https://doi.org/10.1007/s10459-020-09979-6.

    Article  Google Scholar 

  • Sumer, H. C., & Knight, P. A. (1996). Assimilation and contrast effects in performance ratings: Effects of Rating the previous performance on rating subsequent performance. The Journal of Psychology, 81(4), 436–442. https://doi.org/10.1080/00223980.1995.9914955.

    Article  Google Scholar 

  • Tavares, W., & Eva, K. W. (2013). Exploring the impact of mental workload on rater-based assessments. Advances in Health Sciences Education, 18, 291–303. https://doi.org/10.1007/s10459-012-9370-3.

    Article  Google Scholar 

  • ten Cate, O., Hart, D., Ankel, F., Busari, J., Englander, R., Glasgow, N., Holmboe, E., Iobst, W., Lovell, E., Snell, L. S., Touchie, C., Van Melle, E., & Wycliffe-Jones, K. (2016). Entrustment decision making in clinical training. Academic Medicine, 91(2), 1–8. https://doi.org/10.1097/ACM.0000000000001044.

    Article  Google Scholar 

  • Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science, 185(4157), 1124–1131.

    Article  Google Scholar 

  • Van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & Van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34(3), 205–214. https://doi.org/10.3109/0142159X.2012.652239.

    Article  Google Scholar 

  • Wood, T. J. (2014a). Exploring the role of first impressions in rater-based assessments? Advances in Health Sciences Education: Theory and Practice, 19, 409–427. https://doi.org/10.1007/s10459-013-9453-9

    Article  Google Scholar 

  • Wood, T. J. (2014b). Is it time to move beyond errors in clinical reasoning and discuss accuracy? Advances in Health Sciences Education: Theory and Practice. https://doi.org/10.1007/s10459-014-9498-4.

    Article  Google Scholar 

  • Wood, T. J., Chan, J., Humphrey-Murto, S., Pugh, D., & Touchie, C. (2017). The influence of first impressions on subsequent ratings within an OSCE station. Advances in Health Sciences Education, 22, 969–983.

    Article  Google Scholar 

  • Wood, T. J., Pugh, D., Touchie, C., Chan, J., & Humphrey-Murto, S. (2018). Can physician examiners overcome their first impression when examinee performance changes? Advances in Health Sciences Education, 23(4), 1–12. https://doi.org/10.1007/s10459-018-9823-4.

    Article  Google Scholar 

Download references

Funding

This research was supported in part by a research grant from the Royal College of Physicians and Surgeons of Canada. The authors would like to thank Lesley Ananny, Scott Rauscher and Katherine Scowcroft for their help on this study as well as the Department of Innovation in Medical Education.

Author information

Authors and Affiliations

  1. Faculty of Medicine, University of Ottawa, 850 Peter Morand Crescent, Ottawa, ON, K1G-5Z3, Canada

    Timothy J. Wood, Debra Pugh, Claire Touchie, Samantha Halman & Susan Humphrey-Murto

  2. Department of Medicine, The Ottawa Hospital, Ottawa, Canada

    Debra Pugh, Claire Touchie, Samantha Halman & Susan Humphrey-Murto

  3. Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Canada

    Vijay J. Daniels

  4. Medical Council of Canada, Ottawa, Canada

    Debra Pugh

Authors
  1. Timothy J. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vijay J. Daniels
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debra Pugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Claire Touchie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Samantha Halman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Susan Humphrey-Murto
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors confirm contribution to the paper as follows: study conception and design: TJW, VJS, DP, CT, SH, SHM; data collection: TJW, VJS; analysis and interpretation of results: TJW, VJS, DP, CT, SH, SHM; draft manuscript preparation: TJW, VJS, DP, CT, SH, SHM. All authors reviewed the results and approved the final version of the manuscript.

Corresponding author

Correspondence to Timothy J. Wood.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wood, T.J., Daniels, V.J., Pugh, D. et al. Implicit versus explicit first impressions in performance-based assessment: will raters overcome their first impressions when learner performance changes?. Adv in Health Sci Educ (2023). https://doi.org/10.1007/s10459-023-10302-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10459-023-10302-2

Keywords

Search

Navigation