# Study finds AI marking misses student misconceptions in computing

> Three AI models marked 3,041 university computer science answers. Their scores were broadly in line with a teacher's, but their feedback rarely pointed out what students had misunderstood.

The Yoink Report (CodeYoink) | Worldwide | AI in learning | News of 7 October 2026 | Published 5 November 2026

Researchers at Columbia University and New York University posted a study on 7 October 2026 showing AI marking rarely spots students' misconceptions.

Xi Zhao, Xinyue Jiao and Zhen Xu wrote [the paper](https://arxiv.org/abs/2610.09460). It uses 3,041 answers to 50 open questions from computer science courses at the Universidad Francisco de Vitoria in Spain in 2025 to 2026. A teacher marked each answer out of 10. So did three large language models (LLMs), the AI models behind chatbots: DeepSeek, Qwen and Gemini.

The models' scores agreed moderately to strongly with the teacher's, [the full text](https://arxiv.org/pdf/2610.09460v1) says. Their written feedback differed more. Teachers flagged a misconception, an error in understanding, in 15.0% of comments and the models in 4.8% to 7.4%. For answers the teacher scored 0 to 2, the figures were 31.5% and 5% to 7%.

For lecturers and teachers who use AI to mark, the authors suggest it may suit procedural topics such as software testing. They say topics that need judgement should still be marked by people. In England, [Department for Education guidance](https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education) already says "Teachers, leaders and staff must use their professional judgement when using these tools."

The paper's header says it appears in the proceedings of the 34th International Conference on Computers in Education. The authors say the data comes from one institution and one software engineering course. They did not test newer models, so it is not known whether those give better feedback.

## Sources

- arXiv: [Beyond Score Accuracy: Examining the Diagnostic Quality of LLM-Generated Structured Assessment in Higher Education](https://arxiv.org/abs/2610.09460)
- arXiv: [Beyond Score Accuracy, full text](https://arxiv.org/pdf/2610.09460v1)
- GOV.UK: [Generative artificial intelligence (AI) in education](https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education)

Picture: Lucia LRG, CC BY-SA 3.0, https://commons.wikimedia.org/wiki/File:Campus_oficial_UFV.JPG

Web page: https://codeyoink.com/news/ai-marking-misses-misconceptions