Abstract
Artificial Intelligence (AI) strives to create intelligent machines with human-like abilities. However, like humans, AI can be prone to implicit biases due to flaws in data or algorithms. These biases may cause discriminatory outcomes and decrease trust in AI. Bias in higher education admission may limit access to opportunities and further social inequalities, often due to implicit biases in data processing and decision-making. Addressing and recognizing implicit biases in AI is essential to create equal access to higher education admission and opportunities for students. To combat AI implicit biases, it is necessary to monitor and assess their performance and train them using unbiased data and algorithms. This ensures that all students have equal access to higher education and the opportunities it provides them. While the recent studies reviewed the algorithmic approaches to reducing bias, this article focuses instead on exploring the current understanding of the impacts of AI implicit bias in higher education and its implications for admissions. Furthermore, it evaluates the interactions between AI technology and education, specifically in mitigating AI implicit bias algorithms that can be leveraged to achieve inclusive and equitable quality education and promote lifelong learning opportunities for all.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data supporting the findings of this study are as follows:
1. Maslej, N., Fattorini, L., Brynjolfsson, E., Etchemendy, J., Ligett, K., Lyons, T., Manyika, J., Ngo, H., Niebles, J. C., Parli, V., Shoham, Y., Wald, R., Clark, J., & Perrault, R. (2023). The AI Index 2023 Annual Report. AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA. Available online (accessed on November 1, 2023): https://aiindex.stanford.edu/report/
2. Zweben, S, Bizo, B, (2022) Vol. 34, No.5, "CS enrollment grows at all degree levels, with increase gender diversity", Computer Research Association 2021 Taulbee Survey, Available online (accessed on November 1, 2023): https://cra.org/crn/2022/05/cra-2021-taulbee-survey-cs-enrollment-grows-at-all-degree-levels-with-increased-gender-diversity/
3. Baker, R.S., Hawn, A. Algorithmic Bias in Education. Int J Artif Intell Educ 32, 1052–1092 (2022). https://doi.org/10.1007/s40593-021-00285-9
4. Herolad, B., (2022), "Why Schools Need to Talk About Racial Bias in AI-Powered Technologies", Education week, Special report, Available online (accessed on November 1, 2023): https://www.edweek.org/leadership/why-schools-need-to-talk-about-racial-bias-in-ai-powered-technologies/2022/04
5. MacMillan, D., Anderson, N. (2019), "Student tracking, secret scores: How college admissions offices rank prospects before they apply", Washington post, Available online (accessed on November 1, 2023): https://www.washingtonpost.com/business/2019/10/14/colleges-quietly-rank-prospective-students-based-their-personal-data/
Notes
UN SDGs gender equality (Goal 5)—https://sdgs.un.org/goals/goal5
UN SDGs reduced inequalities (Goal 10)—https://sdgs.un.org/goals/goal10
United Nations General Assembly (UNGA)—https://www.un.org/en/ga/
United Nations Educational, Scientific and Cultural Organization (UNESCO)- https://en.unesco.org/
International Telecommunication union (ITU)- www.itu.int/en/Pages/default.aspx
Columbia University Transformation learning technologies lab (TLTL)- https://tltlab.org/
National Institute of Standards and Technology (NIST)- www.nist.gov/
USA Civil Rights Act of 1964—https://www.archives.gov/milestone-documents/civil-rights-act
EU Charter of Fundamental Rights- https://www.europarl.europa.eu/charter/pdf/text_en.pdf
Capture Higher Ed- https://www.capturehighered.com/
Ruffalo Noel Levitz—https://www.ruffalonl.com/
References
2020 Digital Admission Report. (2020). TargetX & mStoner. https://info.targetx.com/hubfs/eBooks/mStoner_TargetX_Digital_Admissions_2020_small.pdf. Accessed 1 Nov 2023.
AI In Education Market Size & Share Report. (2022–2030). AI In Education Market Size, Share & Trends Analysis Report By Component (Solutions, Services), By Deployment, By Technology, By Application, By End-use, By Region, And Segment Forecasts, 2022 - 2030. Report ID: GVR-4–68039–948–8. https://www.grandviewresearch.com/industry-analysis/artificial-intelligence-ai-education-market-report. Accessed 1 Nov 2023.
Al-Alawi, A. I., Naureen, M., AlAlawi, E. I., & Al-Hadad, A. A. N. (2021). The Role of Artificial Intelligence in Recruitment Process Decision-Making. In 2021 International Conference on Decision Aid Sciences and Application (DASA), IEEE, pp. 197–203.
ANSI INCITS 172–220 R. (2007). Information Technology - American National Standard Dictionary Of Information Technology (ANSDIT). https://csrc.nist.gov/Topics/technologies/artificial-intelligence#:~:text=artificial%20intelligence%20AI&text=%22(1)%20A%20branch%20of,learning%2C%20and%20self%2Dimprovement. Accessed 1 Nov 2023.
Artificial Intelligence Market in the Education Sector in US by End-user and Education Model - Forecast and Analysis 2022-2026. (2022). Technavio. https://www.technavio.com/report/artificial-intelligence-market-in-the-education-sector-in-us-industry-analysis. Accessed 1 Nov 2023
Azcona, G., Bhatt, A., Fillo, G. F., Min, Y., Page, H., & You, S. (2023). Progress on the sustainable development goals: The gender snapshot 2023. UN Secretary-General’s High-Level Panel on Women’s Economic Empowerment. https://www.unwomen.org/en/digital-library/publications/2023/09/progress-on-the-sustainable-development-goals-the-gender-snapshot-2023. Accessed 10 Feb 2024.
Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32, 1052–1092. https://doi.org/10.1007/s40593-021-00285-9
Blodgett, S. L., Barocas, S., III, H. D., & Wallach, H. (2020). Language (Technology) is Power: A Critical Survey of “Bias” in NLP. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5454–5476. https://doi.org/10.18653/v1/2020.acl-main.485
Brownstein, M. (2019). Implicit bias. In E. N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Fall 2019 Edn.). https://plato.stanford.edu/archives/fall2019/entries/implicit-bias/. Accessed 1 Nov 2023.
Carnegie Higher Ed. (2020). Prospective Grad Student Survey: Social Media Preferences and Recommendations. https://www.carnegiehighered.com/blog/prospective-grad-student-survey-social-media-preferences-recommendations/. Accessed 1 Nov 2023.
Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20, 1–38.
Chandler, D., & Munday, R. (2020) A dictionary of media and communication. Oxford University Press. https://doi.org/10.1093/acref/9780198841838.001.0001
Chen, P., Linna, W., & Wang, L. (2023). AI fairness in data management and analytics: A review on challenges, methodologies and applications. Applied Sciences, 13(18), 10258.
Cohen, G. (2021). Making Content Marketing Work for Higher Education. OHO Interactive. https://www.oho.com/blog/making-content-marketing-work-higher-education. Accessed 1 Nov 2023.
Dillon, S., & Collett, C. (2019). AI and Gender: Four Proposals for Future Research. Apollo - University of Cambridge Repository. https://doi.org/10.17863/CAM.41459
Dutwin, D., & Buskirk, T. D. (2023). A deeper dive into the digital divide: Reducing coverage bias in Internet surveys. Social Science Computer Review, 41(5), 1902–1920.
Edwards, H (2020). The quest to make AI less prejudiced. Quartz. https://qz.com/1814415/how-tofix-bias-in-ai/. Accessed 1 Nov 2023.
Elazari, A., Mulligan, D., Burrell, J, & Kluttz, D. (2018). AFOG workshop panel 2: Automated decision-making is imperfect, but it’s arguably an improvement over biased human decision-making. UC Berkeley AFOG. https://sites.ischool.berkeley.edu/afog/files/2019/07/AFOG_workshop2018_report_all_web.pdf. Accessed 1 Nov 2023.
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. https://doi.org/10.1038/nature21056
Feathers, T., (2022). This Private Equity Firm Is Amassing Companies That Collect Data on America’s Children. The Markup. https://themarkup.org/machine-learning/2022/01/11/this-private-equity-firm-is-amassing-companies-that-collect-data-on-americas-children. Accessed 1 Nov 2023.
Friedman, B., & Nissenbaum, H. (2017). Bias in Computer Systems. Computer Ethics, 215–232. https://doi.org/10.4324/9781315259697-23
Garcia, J. E., Pereira, J. S., & Cairrão, A. (2021). Social media content marketing strategy for higher education: A case study approach. Smart Innovation, Systems and Technologies, 205, 493–505.
Gilbert, T. K., & Mintz, Y. (2019). Epistemic therapy for bias in automated decision-making. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES '19), (pp. 61–67). Association for Computing Machinery. https://doi.org/10.1145/3306618.3314294
Groeger, L. V., Waldman, A., & Eads, D. (2018). Miseducation: Is There Racial Inequality at Your School? ProPublica. https://projects.propublica.org/miseducation. Accessed 1 Nov 2023.
Gupta, M., Parra, C. M., & Dennehy, D. (2022). Questioning racial and gender bias in AI-based recommendations: Do Espoused National Cultural Values Matter? Information Systems Frontiers, 24(6), 1465–1481. https://doi.org/10.1007/s10796-021-10156-2
Guyon, N., Maurin, E., & McNally, S. (2012). The effect of tracking students by ability into different schools. Journal of Human Resources, 47(3), 684–721. https://doi.org/10.3368/jhr.47.3.684
Herolad, B. (2022) Why Schools Need to Talk About Racial Bias in AI-Powered Technologies. Education week, Special report. https://www.edweek.org/leadership/why-schools-need-to-talk-about-racial-bias-in-ai-powered-technologies/2022/04. Accessed 1 Nov 2023.
ITU. (2022). United Nations Activities on Artificial Intelligence (AI). https://www.itu.int/pub/S-GEN-UNACT. Accessed 1 Nov 2023.
Kleinberg, J., Ludwig, J., Mullainathan, S., & Rambachan, A. (2018). Algorithmic Fairness. AEA Papers and Proceedings, 108, 22–27. https://doi.org/10.1257/pandp.20181018
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074.
Landers, R. N., & Behrend, T. S. (2023). Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models. American Psychologist 78(1), 36.
Lief, L. (2020). How philanthropy can help lead on data justice. Stanford Social Innovation Review. https://ssir.org/articles/entry/how_philanthropy_can_help_lead_on_data_justice. Accessed 1 Nov 2023.
Lievore, I., & Triventi, M. (2023). Do teacher and classroom characteristics affect the way in which girls and boys are graded? British Journal of Sociology of Education, 44(1), 97–122. https://doi.org/10.1080/01425692.2022.2122942
Lohia, P. K., Natesan Ramamurthy, ,K., Bhide, M., Saha, D., Varshney, K. R., & Puri, R. (2019). Bias Mitigation Post-processing for Individual and Group Fairness. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, pp. 2847–2851. https://doi.org/10.1109/ICASSP.2019.8682620
MacMillan, D., Anderson, N. (2019). Student tracking, secret scores: How college admissions offices rank prospects before they apply. Washington post. https://www.washingtonpost.com/business/2019/10/14/colleges-quietly-rank-prospective-students-based-their-personal-data/. Accessed 1 Nov 2023.
Manyika, J., Silberg, J., & Presten, B. (2019). What Do We Do About the Biases in AI? Harvard Business Review. https://hbr.org/2019/10/what-do-we-do-about-the-biases-in-ai. Accessed 1 Nov 2023.
Maslej, N., Fattorini, L., Brynjolfsson, E., Etchemendy, J., Ligett, K., Lyons, T., Manyika, J., Ngo, H., Niebles, J. C., Parli, V., Shoham, Y., Wald, R., Clark, J., & Perrault, R. (2023). The AI Index 2023 Annual Report. AI Index Steering Committee, Institute for Human-Centered AI, Stanford University. https://aiindex.stanford.edu/report/. Accessed 1 Nov 2023.
Merhi, M. I. (2023). An assessment of the barriers impacting responsible artificial intelligence. Information Systems Frontiers,25(3), 1147–1160.
Motahhari, H., Nejad, A. D., & Ghotbaddini, M. (2023). Effective factors on student loyalty: A relationship marketing approach. Research and Planning in Higher Education, 20(3), 121–140.
Mulligan, Deirdre, K., Kluttz, D., & Kohli, N. (2019). Shaping Our Tools: Contestability as a Means to Promote Responsible Algorithmic Decision Making in the Professions Draft. https://doi.org/10.2139/ssrn.3311894
Organization of Scientific Area Committees for Forensic Science. (2021). OSAC Preferred Terms. https://www.nist.gov/system/files/documents/2021/10/05/STRP%20Final%20Report_2021-S-0011_Standards%20for%20the%20Technical%20Review%20of%20Bloodstain%20Pattern%20Analysis%20Reporting%20_Final.OPEN%20COMMENT%20VERSION.pdf. Accessed 1 Nov 2023.
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., Hall, P. (2022). Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, NIST publication. https://doi.org/10.6028/NIST.SP.1270. Accessed 1 Nov 2023
Shaw, C. (2014). Have traditional student recruitment campaigns lost their bite? The Guardian. https://www.theguardian.com/higher-education-network/blog/2014/mar/10/university-student-recruitment-social-media-marketing. Accessed 1 Nov 2023.
Shrestha, S., & Das, S. (2022). Exploring gender biases in ML and AI academic research through systematic literature review. Frontiers in Artificial Intelligence, 5, 976838.
Smith, G., & Rustagi, I. (2020). Mitigating bias in artificial intelligence: An equity fluent leadership playbook. Berkeley Haas Center for Equity, Gender, and Leadership. https://haas.berkeley.edu/wp-content/uploads/UCB_Playbook_R10_V2_spreads2.pdf
Soundarajan, S., & Clausen, D. L. (2018). Equal Protection Under the Algorithm: A Legal-Inspired Framework for Identifying Discrimination in Machine Learning. In Proceedings of the 35th International Conference on Machine Learning.
Staats, C., Capatosto, K., Tenney, L., & Mamo S. (2017). State of the Science: Implicit Bias Review 2017. Kirwan Institute for the Study of Race and Ethnicity. https://search.issuelab.org/resource/state-of-the-scienceimplicit-bias-review-2017.html
Team, A. P. (2021). Artificial Intelligence Measurement and Evaluation at the National Institute of Standards and Technology. National Institute of Standards and Technology. https://www.nist.gov/system/files/documents/2022/08/25/aime%20workshop%20summary%202.2.pdf. Accessed 1 Nov 2023.
Tilmes, N. (2022). Disability, fairness, and algorithmic bias in AI recruitment. Ethics and Information Technology, 24(2). https://doi.org/10.1007/s10676-022-09633-2
Whittaker, M., Alper, M., Bennett, C., Hendren, S., Kaziunas, L., Mills, M., Morris, M. R., Rankin, J., Rogers, E., Salas, M., & West, S. M. (2019). Disability, power and AI. AI Now Institute. https://ainowinstitute.org/publication/disabilitybiasai-2019. Accessed 1 Nov 2023.
Zheng, Y., Zhou, Z., & Blikstein, P. (2022). Towards an inclusive and socially committed community in artificial intelligence in education: A social network analysis of the evolution of authorship and research topics over 8 years and 2509 papers. In Proceedings of the International Conference on Artificial Intelligence in Education. p. 414–426. Springer. https://doi.org/10.1007/978-3-031-11644-5_34. Accessed 1 Nov 2023
Zweben, S., & Bizo, B. (2022). CS enrollment grows at all degree levels, with increase gender diversity. Computer Research Association 2021 Taulbee Survey, (Vol. 34, No.5). https://cra.org/crn/2022/05/cra-2021-taulbee-survey-cs-enrollment-grows-at-all-degree-levels-with-increased-gender-diversity/. Accessed 1 Nov 2023
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
None.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Roshanaei, M. Towards best practices for mitigating artificial intelligence implicit bias in shaping diversity, inclusion and equity in higher education. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12605-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10639-024-12605-2