Equity and bias in automated educational evaluation systems based on artificial intelligence
DOI:
https://doi.org/10.63688/aprendizaje.v2.i2.16Keywords:
higher education, systematic review, educational assessment, evidence, PRISMA, educational technology.Abstract
Introduction: Equidad y sesgos en sistemas automatizados de evaluación educativa basados en inteligencia artificial is an emerging higher-education issue related to measurement quality, decision-making and learning experience. Objective: To systematically synthesize available evidence and identify patterns of effectiveness, methodological quality and implementation conditions. Method: A PRISMA-oriented systematic review was conducted on a consolidated academic corpus of 135 records; 7 studies met relevance and documentary sufficiency criteria. Results: Evidence was methodologically heterogeneous but converged on the need to align technology, constructs, feedback and educational decisions. Conclusions: Findings support evidence-informed adoption, institutional monitoring and continuous assessment of validity, equity and utility.References
Chen G, Wu AD, Zhang C. Assessing fairness in AI-assisted writing scoring: Developing fairness measures to detect predictive bias in automated essay scoring. Assessing Writing. 2026;69:101066. https://doi.org/10.1016/j.asw.2026.101066
Andersen N, Mang J, Goldhammer F, et al. Algorithmic Fairness in Automatic Short Answer Scoring. International Journal of Artificial Intelligence in Education. 2025;35:3128–3165. https://doi.org/10.1007/s40593-025-00495-5
Huang Y, Palermo C, Wilson J. Accuracy and fairness of generative AI in automated essay scoring: Comparing GPT-4o, feature-based models, and human raters. Assessing Writing. 2026;69:101047. https://doi.org/10.1016/j.asw.2026.101047
Choi I, Johnson MS. Identifying Features Contributing to Differential Prediction Bias of Automated Scoring Systems. Journal of Educational Measurement. 2025;62(4):838–861. https://doi.org/10.1111/jedm.70015
Johnson MS, Liu X, McCaffrey DF. Psychometric Methods to Evaluate Measurement and Algorithmic Bias in Automated Scoring. Journal of Educational Measurement. 2022;59(3):338–361. https://doi.org/10.1111/jedm.12335
Al-Maamari H, Sidhu MS, Hussain SM, Al-Ghaili AM, Sidhu KK. Accessible ICT-Enabled Examination Technologies for Students with Special Educational Needs in Higher Education: A Technical Evaluation of Usability, Reliability, Accommodation Quality, and Assessment Fairness. International Journal of Special Education. 2026;41(17s):635–654.
Huggins-Manley AC, Booth BM, D'Mello SK. Toward Argument-Based Fairness with an Application to AI-Enhanced Educational Assessments. Journal of Educational Measurement. 2022;59(3):362–388. https://doi.org/10.1111/jedm.12334
Baker RS, Hawn A. Algorithmic Bias in Education. International Journal of Artificial Intelligence in Education. 2022;32(4):1052-1092. https://doi.org/10.1007/s40593-021-00285-9
Ferrara S, Qunbar S. Validity Arguments for AI-Based Automated Scores: Essay Scoring as an Illustration. Journal of Educational Measurement. 2022;59(3):288-313. https://doi.org/10.1111/jedm.12333
Ercikan K, McCaffrey DF. Optimizing Implementation of Artificial-Intelligence-Based Automated Scoring: An Evidence Centered Design Approach for Designing Assessments for AI-based Scoring. Journal of Educational Measurement. 2022;59(3):272-287. https://doi.org/10.1111/jedm.12332
Shermis MD. Anchoring Validity Evidence for Automated Essay Scoring. Journal of Educational Measurement. 2022;59(3):314-337. https://doi.org/10.1111/jedm.12336
Ramesh D, Sanampudi SK. An automated essay scoring systems: a systematic literature review. Artificial Intelligence Review. 2022;55(3):2495-2527. https://doi.org/10.1007/s10462-021-10068-2
Susanti MNI, Ramadhan A, Warnars HLHS. Automatic essay exam scoring system: a systematic literature review. Procedia Computer Science. 2023;216:531-538. https://doi.org/10.1016/j.procs.2022.12.166
Khosravi H, Shum SB, Chen G, Conati C, Tsai YS, Kay J. Explainable Artificial Intelligence in education. Computers and Education: Artificial Intelligence. 2022;3:100074. https://doi.org/10.1016/j.caeai.2022.100074
Memarian B, Doleck T. Fairness, Accountability, Transparency, and Ethics (FATE) in Artificial Intelligence (AI) and higher education: A systematic review. Computers and Education: Artificial Intelligence. 2023;5:100152. https://doi.org/10.1016/j.caeai.2023.100152
Zhu H, Sun Y, Yang J. Towards responsible artificial intelligence in education: a systematic review on identifying and mitigating ethical risks. Humanities and Social Sciences Communications. 2025;12:1111. https://doi.org/10.1057/s41599-025-05252-6
Deho OB, Zhan C, Li J, Liu J, Liu L, Le TD. How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics? British Journal of Educational Technology. 2022;53(4):822-843. https://doi.org/10.1111/bjet.13217
Khalil M, Prinsloo P, Slade S. Fairness, Trust, Transparency, Equity, and Responsibility in Learning Analytics. Journal of Learning Analytics. 2023;10(1):1-7. https://doi.org/10.18608/jla.2023.7983
Froehlich L, Weydner-Volkmann S. Adaptive Interventions Reducing Social Identity Threat to Increase Equity in Higher Distance Education: A Use Case and Ethical Considerations on Algorithmic Fairness. Journal of Learning Analytics. 2024. https://doi.org/10.18608/jla.2024.8301
Cohausz L, Kappenberger J, Stuckenschmidt H. What Fairness Metrics Can Really Tell You: A Case Study in the Educational Domain. Proceedings of the 14th Learning Analytics and Knowledge Conference. 2024:792-799. https://doi.org/10.1145/3636555.3636873
Yang K, Raković M, Li Y, Guan Q, Gašević D, Chen G. Unveiling the Tapestry of Automated Essay Scoring: A Comprehensive Investigation of Accuracy, Fairness, and Generalizability. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(20). https://doi.org/10.1609/aaai.v38i20.30254
Jones-Jang SM, Chung M, Choi J, Kim N, Lee S. Fairness perceptions of AI in grading systems: Examining how discontent with the status quo and outcome favorability reduce AI reluctance. Computers and Education: Artificial Intelligence. 2025;8:100419. https://doi.org/10.1016/j.caeai.2025.100419
Villegas-Ch W, Gutierrez R, Garcia-Ortiz J, Govea J. Mitigating algorithmic bias in educational prediction systems using reweighting and adversarial learning. Frontiers in Education. 2026;11:1841724. https://doi.org/10.3389/feduc.2026.1841724
Albuquerque J, Rienties B, Hlosta M, Holmes W. Learning Analytics to Uncover Ethnic Bias in Educational Texts: An Ensemble Learning Approach. Journal of Learning Analytics. 2026;13(1):143-162. https://doi.org/10.18608/jla.2026.8905
Lluch Molins L, Lindín Soriano C. The self-regulatory paradox of learning analytics: student expectations and the conditions for fair algorithmic assessment in higher education. Frontiers in Education. 2026;11:1913278. https://doi.org/10.3389/feduc.2026.1913278
Agarwal B, Urlings C, van Lankveld G, Klemke R. Identifying the ethical values and norms for artificial intelligence in education: A systematic literature review. International Journal of Artificial Intelligence in Education. 2026;36(1-2):100004. https://doi.org/10.1016/j.ijaied.2026.100004
Zhuang M, Long S, Martin F, Castellanos-Reyes D. The affordances of Artificial Intelligence (AI) and ethical considerations across the instruction cycle: A systematic review of AI in online higher education. The Internet and Higher Education. 2025;67:101039. https://doi.org/10.1016/j.iheduc.2025.101039
Holmes L, Morris W, Crossley S, Choi JS. Assessing fairness in finetuned scoring models with demographically restricted training data. Assessing Writing. 2026;68:101032. https://doi.org/10.1016/j.asw.2026.101032
Wu HN, Chu MN, Hsu JL. Comparing GPT and human raters in essay assessment: Variability, bias, and the potential of LLM-based scoring. Computers and Education Open. 2026;10:100341. https://doi.org/10.1016/j.caeo.2026.100341
Sun J, Song T, Weiming P, Song J. A survey of automated essay scoring: Challenges, advances, and future. Neurocomputing. 2025;650:130916. https://doi.org/10.1016/j.neucom.2025.130916
Lin P, Deng Q, Zhou Y. Towards responsible AI in education: A Delphi-AHP-based framework for evaluating educational large language models. Computers and Education: Artificial Intelligence. 2026;10:100534. https://doi.org/10.1016/j.caeai.2025.100534
Published
Issue
Section
License
Copyright (c) 2025 Pamela Navarro Escobar (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
