Editors’ Summary: In this post, Ben Williamson examines the growing quality control and methodological rigor crisis within the field of Artificial Intelligence in Education (AIED) research. By highlighting the recent retraction of a high-profile paper on ChatGPT and analyzing two new critical literature reviews, Williamson demonstrates how the pressure to quickly produce statistical evidence has led to a proliferation of poor-quality “media comparison” studies and flawed meta-analyses. He argues that these studies often rely on incomparable control conditions, display publication bias, and overlook critical ethical questions, ultimately resulting in overinflated and misleading claims about AI’s positive impact on learning. Consequently, he cautions policymakers and educational specialists against relying on the current empirical evidence base to inform educational practice or policy decisions.
Editors’ Choice: The problem with evidence production on AI in education
