Can LLMs Audit Machine Unlearning? A Benchmark That Exposes Fake Forgetting Code

When Code Claims to "Forget" but Quietly Multiplies by Zero

Machine Unlearning is becoming a critical topic in AI safety. When users request data deletion, or when training data contains malicious patterns, models need to genuinely remove the influence of those samples — not just put on a show. The problem is that many so-called unlearning implementations only look like they're forgetting.

A developer built a benchmark called Unlearning Security Audit on a Kaggle Benchmarking Challenge specifically to test whether large language models can see through this kind of disguise. The experiment starts with a deceptively crafted piece of PyTorch code — it computes the loss on the data to be forgotten, then multiplies that loss by zero. In other words, the unlearning term contributes nothing to the gradient, and the model never actually performs any "reverse unlearning" update.

The core question is simple: Can an LLM tell the difference between code that claims to forget and code that actually executes an unlearning update?

Two Carefully Designed Bug Cases

Each model was fed two flawed cases, corresponding to two common but incorrect unlearning implementations.

Case One: The Masked Unlearning Loss

The code replaces the forget-set labels with zeros, then multiplies the forget loss by 0.0. The result: the forget term produces no gradient, training continues only on the retain data, and there's no update whatsoever aimed at "erasing bad learning." This is a classic "present in form, broken in substance" trap.

Case Two: The Zeroed-Out Target

The code removes the loss mask but still replaces forget-set labels with zeros. This effectively trains the model to predict "class 0" for those samples. But relabeling is not the same as removing a sample's influence — the model may simply learn a new incorrect mapping while the original influence remains intact.

Each model was asked to do three things: explain the bug, rewrite the code using original labels and a gradient ascent baseline, and propose how to test whether a backdoor still exists. The author specified the gradient ascent approach in the prompt, so the test was evaluating auditing and implementation ability — not asking the model to invent an unlearning algorithm from scratch. Each case came with eight automated checks covering Python syntax, loss computation, original label usage, optimizer steps, problem diagnosis, evaluation planning, and discussion of limitations.

These two bug types represent two typical failure modes in real-world unlearning implementations: one where the operation is silently short-circuited (multiplying by 0.0 zeros out the gradient), and one where the target is incorrectly substituted (training on wrong labels instead of truly removing influence). Notably, neither error triggers any runtime exception — the program runs normally, the loss curve appears to converge, and only someone who deeply understands the unlearning objective would notice anything wrong. That's precisely why testing this kind of code audit with LLMs is meaningful: models need semantic understanding of whether a training objective mathematically achieves its intent, not just syntactic code inspection.

Both Gemini Models Caught the Conceptual Errors

The author tested Gemini 2.5 Pro and Gemini 2.5 Flash from the same model family, using identical prompts and checks.

The expected failure mode was a model that cleans up the code aesthetically but misses why it can't actually unlearn anything. That didn't happen. Both models accurately identified the issues and used the original forget-set labels in their fixes. All four selected responses passed the mathematical objective check, with the rewritten core logic looking like this:

retain_loss = F.cross_entropy(model(x_r), y_r)
forget_loss = F.cross_entropy(model(x_f), y_f)
objective = retain_loss - alpha * forget_loss

A standard optimizer minimizes this objective: the positive retain term encourages good performance on retained samples, while the negative forget term encourages increasing loss on samples to be forgotten, with alpha controlling the balance. This is a gradient ascent baseline — but it doesn't guarantee that user influence or backdoors have been thoroughly eliminated.

The Biggest Surprise Came from the Scorer Itself

Initial scores appeared to show a clear gap: Pro passed all checks on both cases, while Flash only passed all checks on one.

Model	Original Full-Pass Cases	After Manual Review	Avg. Response Time
Gemini 2.5 Pro	2/2	2/2	38.34s
Gemini 2.5 Flash	1/2	2/2	22.05s

When the author carefully read Flash's "failing" response, it turned out to correctly explain the masked loss bug — pointing out that "multiplying forget_loss by 0.0 effectively removes any contribution of the forget data gradient to the total loss." But the text check was only looking for words like "zero" or "mask" and missed the explanation phrased with the numeric literal 0.0.

Similarly, Flash noted that the method "cannot mathematically guarantee complete and irreversible erasure," yet the limitations check still marked it as failed because it didn't use the expected words "approximate" or "heuristic."

The author flagged both as false negatives in the scorer, preserving the original labels and documenting manual corrections based on the original answers. After review, both models met the bar on both cases. The apparent performance gap was entirely due to brittle keyword-matching checks. The author also honestly notes that this small-scale test doesn't prove the two models are equivalent in overall capability.

This finding carries an important implication: when we use LLMs to audit code, the tooling used to score LLM outputs also needs to be audited.

Loss Moving in the Right Direction ≠ Successful Unlearning

The author also ran a separate experiment with a synthetic logistic regression classifier (note: this did not involve running any LLM-generated PyTorch fix code). It measured clean accuracy and trigger attack success rate — the frequency with which the model produces the attacker's target label on trigger-embedded test inputs.

Method	Clean Accuracy	Trigger Attack Success Rate
Poisoned checkpoint	100%	100%
Masked unlearning loss	100%	100%
Retain descent + forget ascent (80 steps)	100%	100%
Retrain on retain data only	100%	0%

The results are striking. After 80 steps of gradient ascent-based repair, the forget loss did rise from 0.0545 to 0.1649 — but the trigger still activated on 100% of trigger-embedded test inputs. Only full retraining on the retain data simultaneously eliminated backdoor behavior and preserved clean accuracy in this simple setting.

The core lesson here is unmistakable: loss moving in the expected direction is not sufficient evidence of successful unlearning. Security behavior requires its own dedicated testing. A backdoor can survive an "unlearn me" instruction precisely because we mistake a loss change for proof of forgetting.

A backdoor attack is when an adversary injects poisoned training samples carrying a specific trigger, causing the model to output the attacker's target class whenever that trigger appears — while behaving normally on clean inputs. Trigger Attack Success Rate (ASR) measures the proportion of trigger-embedded test inputs on which the model outputs the target class, and is the key metric for evaluating whether a backdoor has been removed. Clean accuracy measures normal model performance on clean inputs; both must be monitored simultaneously — a successfully unlearned model should maintain high clean accuracy and near-zero ASR. In this experiment, after 80 steps of gradient ascent, clean accuracy stayed at 100% while ASR remained at 100%, demonstrating that gradient ascent only perturbed loss values without actually destroying how the backdoor pattern is encoded in the model weights.

Speed Observations and Future Directions

Across these calls, Flash's two responses took 23.35 and 20.75 seconds, while Pro took 43.83 and 32.84 seconds. Flash was faster in both paired cases and, after manual review, met the same standard. However, the author notes cautiously that with only one selected response per case, this shouldn't be treated as a reliable latency ranking.

For next steps, the author plans to add more flawed scripts, run repeated calls, review explanations anonymously (without displaying model names), and release improved keyword checks as a revised scorer while keeping the original results visible. Looking further ahead, the goal is to actually execute the generated fix code in an isolated environment and measure clean accuracy, trigger attack success rate, and comparison against retraining.

As the author summarizes: he started out asking whether a model could audit an unlearning script, and ended up discovering that the evaluator itself needed auditing. This is both a pragmatic test of LLM code auditing capability and a sober reminder about the methodology of AI safety evaluation.

Background

The core motivation for Machine Unlearning comes from privacy regulations (such as GDPR's "right to be forgotten") and AI safety needs: when a model's training data contains backdoor samples, poisoned data, or personal information a user wants retracted, the ideal outcome is to completely erase those samples' influence from the model's parameters. However, full retraining is extremely costly, so researchers have proposed various approximation methods. Gradient Ascent is the most intuitive — while normal training uses gradient descent to minimize loss, gradient ascent does the opposite, deliberately maximizing loss on the data to be forgotten, thereby pushing the model away from making correct predictions on those samples. But the inherent limitation of approximate methods is that changes in loss values are only an indirect signal — they don't equal genuine changes in the model's internal representations or decision boundaries, and deep patterns like backdoor triggers may persist even as the loss metric improves.