No Cover Image

E-Thesis 28 views 6 downloads

Bayesian Active Learning for Comparative Judgement: A New Paradigm for Educational Assessment / ANDREW GRAY

Swansea University Author: ANDREW GRAY

  • 2026_Gray_A.final.71898.pdf

    PDF | E-Thesis – open access

    Copyright: the author, Andy Gray, 2026. Distributed under the terms of a Creative Commons Attribution 4.0 License (CC BY 4.0).

    Download (10.73MB)

DOI (Published version): 10.23889/SUThesis.71898

Abstract

Assessment is a cornerstone of education, yet traditional marking methods can be inconsistent, biased, and cognitively demanding for educators. Comparative Judgement (CJ)offers an alternative by ranking student work through pairwise comparisons but faces challenges around transparency, efficiency, and...

Full description

Published: 2026
Institution: Swansea University
Degree level: Doctoral
Degree name: Ph.D
Supervisor: Rahat, A., Crick, T., Pearson, J., and Lindsay, S.
URI: https://cronfa.swan.ac.uk/Record/cronfa71898
Abstract: Assessment is a cornerstone of education, yet traditional marking methods can be inconsistent, biased, and cognitively demanding for educators. Comparative Judgement (CJ)offers an alternative by ranking student work through pairwise comparisons but faces challenges around transparency, efficiency, and biases in pair selection. This thesis pro-poses Bayesian Comparative Judgement (BCJ) as a structured, data-driven alternative to address these limitations.BCJ integrates entropy-driven active learning to select the most informative comparisons, systematically improving ranking accuracy while avoiding the model deterioration seen in traditional CJ. By generating complete predictive rank distributions, BCJ also enables probabilistic grading aligned with assessors’ decisions. Experiments using syn-thetic and real-world datasets, including GCSE essays, demonstrate BCJ’s strong performance and efficiency against existing CJ methods. BCJ also introduces methods to quantify assessor agreement, reinforcing reliability and accountabilityTo enhance performance insights, the thesis extends BCJ to a multi-criteria frame-work (MBCJ), aligning with rubric-based assessment by evaluating individual learning outcomes (LOs) alongside overall performance. This approach preserves CJ’s efficiency while providing granular, criterion-specific insights.The practical implementation of BCJ and MBCJ is evaluated through studies with professional markers in higher education, focusing on transparency, usability, and trust.Findings show that while traditional marking is familiar, BCJ and MBCJ reduce subjectivity, improve alignment with target rankings, and maintain educational integrity, offering a viable alternative for assessment.Overall, this research demonstrates how Bayesian methods can refine CJ to enhance transparency, fairness, and efficiency in assessment. The thesis supports adopting structured CJ methods to improve feedback and reduce educator workload, contributing to fairer and more transparent assessment practices across diverse educational contexts.
Keywords: Artificial Intelligence, Machine Learning, Active Learning, Comparative Judgement, Education, Assessment, Transparency, Bayesian Statistics.
College: Faculty of Science and Engineering
Funders: EPSRC Enhancing Human Interactions and Collaborations with Data and Intelligence-Driven Systems

Similar Items