AI Boosts Homework Scores by 18% but Tanks Exam Performance by 20%: The Cost of False Efficiency

A Counterintuitive Research Finding

Artificial intelligence is infiltrating every corner of education. From completing homework to exam preparation, students increasingly rely on ChatGPT and other large language models to accomplish learning tasks. These Large Language Models (LLMs), built on the Transformer architecture and trained on massive text datasets, can generate coherent, logically sound text outputs. Since ChatGPT's release in late 2022, its penetration into educational settings has far exceeded expectations—a 2024 Stanford University survey revealed that over 60% of college students admitted to using generative AI tools in their academic work. These tools can not only answer questions but also generate essay outlines, problem-solving steps, code snippets, and even simulate expert conversations in specific disciplines, posing unprecedented challenges to traditional homework assessment methods.

However, a study has revealed an alarming phenomenon: while AI can indeed boost homework scores in the short term, it may be quietly eroding students' genuine learning abilities over the long run.

The research showed that students using AI assistance saw their homework scores increase by 18%—seemingly a solid improvement. But when these same students entered closed-book exams where AI was unavailable, their test scores dropped by 20%. The stark contrast between this rise and fall is precisely what sparked widespread discussion around this study.

The "Grade Illusion": Why Homework Scores Are Deceptive

The improvement in homework scores is easily misread as an improvement in learning outcomes. But upon careful analysis of the data, we discover the essential problem: homework scores measure "task completion quality," while exam scores measure "degree of knowledge internalization." There is a critical difference between the two.

When students complete homework with AI assistance, AI takes on most of the cognitive load—it helps understand the questions, organize answers, and check for errors. Students appear to be "doing homework," but in reality they're mostly "reviewing AI's output." This process bypasses the genuine learning stages: active thinking, trial and error, and the difficult leap from confusion to understanding.

How Cognitive Load Transfer Affects Learning Outcomes

There's an important concept in psychology called "Germane Cognitive Load," referring to the mental effort that genuinely promotes long-term memory and skill formation. This concept comes from Cognitive Load Theory, proposed by Australian educational psychologist John Sweller in 1988, and is one of the most influential theoretical frameworks in instructional design. The theory categorizes cognitive load during learning into three types: Intrinsic Cognitive Load, determined by the inherent complexity of the learning material; Extraneous Cognitive Load, produced by poor instructional design and which should be minimized; and Germane Cognitive Load, the mental effort learners invest in integrating information into long-term memory schemas. Germane cognitive load is the core mechanism through which learning actually occurs—it drives the encoding, organization, and storage of knowledge.

Learning is inherently an effortful process—it is precisely this moderate struggle that allows knowledge to take root in the brain. When AI eliminates this sense of effort for students, learning becomes a superficial, passive reception of information. Students obtain correct answers without acquiring the ability to arrive at those answers. When AI performs cognitive processing on behalf of students, what gets eliminated is precisely this germane load that facilitates deep learning. This also explains why students who perform excellently with AI assistance immediately reveal hollow knowledge mastery once this "crutch" is removed.

Real-World Validation of "Desirable Difficulties" Theory

This study actually provides yet another real-world case for the famous "Desirable Difficulties" theory in educational psychology. The theory was systematically proposed in the 1990s by Robert Bjork, a cognitive psychology professor at the University of California, Los Angeles (UCLA), and his wife Elizabeth Bjork. Its core argument is that conditions which introduce moderate difficulty during learning, while lowering immediate performance, can significantly enhance long-term retention and transfer ability.

The theory is grounded in extensive experimental evidence, including classic findings such as the Spacing Effect, Interleaving, Testing Effect, and Generation Effect. For example, spaced review—though it makes each practice session feel harder—yields over 50% better long-term retention compared to massed practice. Bjork summarized this phenomenon into a key distinction: there is a fundamental difference between "Performance" and "Learning"—good immediate performance doesn't mean genuine learning has occurred, and poor immediate performance doesn't mean no learning is taking place. This distinction is crucial for understanding the paradox of AI-assisted learning.

AI assistance does precisely the opposite—it reduces immediate learning difficulty, making homework effortless, but at the cost of sacrificing long-term learning outcomes. In other words, AI makes learning "too easy," thereby disrupting the very process through which learning is supposed to occur.

This is consistent with other familiar learning phenomena:

• Highlighting with a marker feels efficient, but active recall is what truly works
• Repeatedly reading notes creates a sense of "familiarity," but this familiarity is an illusion of memory
• AI-generated answers make homework "look perfect," but the brain hasn't truly engaged

The Right Way to Use AI for Learning

This study isn't meant to wholesale reject AI's value in education, but rather to remind us that we must rethink how AI is used. The problem isn't AI itself, but how we design its intervention in the learning process.

From "Answer Tool" to "Learning Partner"

If AI is only used as a tool for quickly obtaining answers, it will inevitably undermine learning outcomes. But with proper guidance, AI can absolutely become a partner that promotes deep learning. For example:

• Socratic Questioning: Have AI guide students to think for themselves through counter-questions, rather than directly providing answers. The Socratic Method originates from the dialogic practice of the ancient Greek philosopher Socrates. Its core approach is to guide learners through continuous questioning to discover contradictions and gaps in their own cognition, thereby actively constructing knowledge. Modern educational research shows this method effectively activates Metacognition—the ability to monitor and regulate one's own thinking processes. Designing AI as a Socratic tutor requires careful engineering at the system prompt level. For example, Khan Academy's AI assistant Khanmigo employs this strategy—when students ask questions, it first responds with "What methods have you already tried?" rather than immediately providing the solution.
• Process Feedback: AI critiques the student's reasoning rather than replacing it
• Error Analysis: AI helps students understand the conceptual gaps behind their mistakes

The key is that AI should increase "germane cognitive load" rather than eliminate it.

Restructuring Educational Assessment

This study also challenges the educational assessment system. When homework scores are distorted by AI, educators need to place greater emphasis on assessment methods that genuinely reflect learning outcomes—closed-book exams, oral defenses, real-time problem solving, and more. The gap between homework and exam scores is itself a diagnostic signal worth monitoring.

A Deeper Warning: The Trade-off Between Short-Term Efficiency and Long-Term Capability

The numerical contrast of 18% versus 20% carries significance far beyond education. It reveals a universally present trap in human-AI collaboration: short-term efficiency gains may mask long-term capability degradation.

Whether it's students relying on AI for homework, programmers relying on AI to write code, or professionals relying on AI for decision-making, we all face the same question: when AI handles the "effortful" thinking for us, are our own capabilities quietly atrophying? This concern is far from unfounded—cross-domain evidence is already accumulating. In aviation, over-reliance on autopilot systems has been proven to erode pilots' manual control skills, with the Federal Aviation Administration (FAA) issuing multiple warnings requiring increased manual flight training. In medicine, research has found that doctors who over-rely on diagnostic support systems show significantly reduced diagnostic accuracy when those systems are unavailable. In software development, developers who frequently use AI coding assistants have shown diminished ability to catch bugs during code review. These cases collectively point to a phenomenon known as the "Automation Paradox": the more reliable a system becomes, the less motivated humans are to maintain their skills, but when the system fails, degraded human skills make the consequences far more severe.

Perhaps the greatest insight this study offers us is this: AI is a powerful tool, but a tool's value depends on the wisdom of its user. While embracing the convenience AI brings, we must be vigilant against the "false efficiency" that comes at the cost of long-term growth. True progress has always required the effort of thinking.

Key Takeaways