Fine-Tuning a 322M Model in Practice: Three Rounds, Two Failures, and a Safer Coding Agent

When AI coding assistants start autonomously executing shell commands, a fundamental safety contradiction surfaces: should we trust an Agent's judgment about the safety of its own actions? One developer's approach is to introduce an independent "gatekeeper"—a small local model that reviews tool calls before execution and decides whether to allow, block, or request confirmation. He open-sourced the laya-guard project and documented in detail the entire process of fine-tuning a 322M-parameter model from a zero-shot baseline to a releasable state. Three rounds of training, two failures, and a final delivery—the data traps and evaluation dilemmas exposed along the way offer reference value for any team building an Agent safety layer.

Background: Why an Independent Command Guard Is Needed

The core design of laya-guard is not complicated: every tool call first passes through a small local model for judgment, rather than letting the Agent decide for itself whether something is safe. The initial version used zero-shot prompting with the Laya multilingual checkpoint. On a benchmark of 50 everyday commands, the base model only allowed half of them outright, with the rest requiring confirmation. The author wanted to know: how far can fine-tuning push such a small model?

On the other side, the project also added several architectural hardening measures. The sandbox mechanism now writes protected paths into the Agent's own OS sandbox, with the kernel refusing unauthorized reads regardless of what the model decides. Command normalization covers variants like r''m, ${IFS}, base64 piped into sh, and variable-hidden commands. The project also added benchmarks comparing it with other command guards. These engineering measures and model judgment form a defense in depth.

However, the author also admits there are gaps: printenv relies on model judgment, while cat /proc/self/environ may still be allowed as an ordinary file read. The environment variable dump scenario is not fully covered.

Zero-shot Prompting refers to directly asking a pretrained model to complete a judgment or generation task based on instructions without providing any task examples. In the initial version of layaguard, the developer fed the command to be reviewed and its context directly into the Laya multilingual checkpoint, asking the model to output allow, review, or block—the model had never seen labeled samples for this task before. The zero-shot approach requires no training data and has low deployment cost, but it is limited by the model's understanding of the task. Especially for a small model with 322M parameters, its decision boundaries are fuzzy and it easily relies on surface features (such as command length) rather than semantics. Therefore, zero-shot results are often used as a baseline for fine-tuning to measure the real improvement brought by data-driven optimization.

Deterministic Rules refer to hard-coded judgment logic based on fixed pattern matching or syntax parsing, such as regular expressions to identify rm -rf /, detect base64 pipes, or block known sensitive paths. Unlike model judgment, rule behavior is fully predictable and auditable and does not depend on training data distribution, but its coverage is limited and it cannot handle semantic variants or novel attacks. In layaguard's defense-in-depth architecture, deterministic rules serve as the first line of defense for clear high-risk commands, while the model handles ambiguous scenarios not covered by the rules. This division of labor both reduces the risk of model misjudgment and ensures that known threats are not missed through rule-based fallbacks.

Training Data: 13,000 Commands and 696 Manual Annotations

The data came from the author's own Agent conversation logs, about 13,000 shell commands. Home paths, tokens, and email addresses were masked. Any command with a similarity of 0.85 or higher to benchmark or evaluation cases was excluded. Of these, 696 commands not covered by deterministic rules were manually annotated, with labels sourced from a written guide, six Agents, and the author's own review of disagreements.

Training was done on a free Colab GPU. The key detail is that the author wrote the acceptance criteria before training, and evaluated on datasets the model had not seen in every round. This disciplined approach is precisely why the failures later became so clear.

Round 1: The Model Learned "Short Command = Dangerous"

Validation accuracy rose from 0.61 to 0.90, which looked encouraging at first glance. But 86% of training rows were labeled allow—a classifier that always allows would also score 0.86. The real problem appeared on unseen everyday commands: the number allowed outright dropped from 25/50 to 23/50, and the newly questioned commands included completely ordinary operations like git status, mkdir -p src/utils, cp, and chmod +x.

Investigation revealed a severe length imbalance in the training data: real allow commands had a median length of 287 characters, while hand-written risky commands were typically only 33 to 40 characters. In addition, the collected conversation logs contained no real blocked commands. The simplest signal the model learned was "short = dangerous." This round met only one of the four acceptance criteria. The author's decision was to change the data rather than the architecture.

Round 2: Fixed False Positives, Introduced False Negatives

To address the previous round's failure, the author added 206 examples: short ordinary commands, long scripts containing a single dangerous command, and Korean and English requests. The everyday command allow rate rose to 49/50, but three dangerous cases slipped through: rmdir /s /q d: deleting an entire Windows drive, a destructive command hidden behind a # clean build output comment, and gh auth token exposing an authentication token.

Each failure had its own cause. The original dataset had no Windows commands; 31 of 36 real commands containing # comments were labeled allow, so the model actually learned "comment = everyday operation"; and almost no training samples involved commands that print secrets. The dataset also became more biased toward allow—84% of rows had that label. The median risk score for everyday commands dropped from 0.45 to 0.03, but attack detection fell to 31/43, below the acceptance threshold. The improvement in usability was not worth the regression in safety.

Class Imbalance is a common data problem in classification tasks, referring to a large disparity in the number of samples across categories in the training set. In layaguard's first round of training, 86% of rows were labeled allow, and the second round still reached 84%, causing the model to favor predicting the majority class (allow) to achieve high accuracy while sacrificing its ability to recognize the minority class (block/review). In safety scenarios, this bias is especially dangerous: the cost of a false negative (judging a dangerous command as allow) is far higher than that of a false positive. Common mitigation methods include oversampling the minority class, undersampling the majority class, adjusting class weights in the loss function, or using stratified sampling to ensure balanced class proportions in each training round. Lowering the allow proportion to 61% in the third round was a direct intervention for this problem.

Round 3: Balancing Data Instead of Chasing Accuracy

The third round made one change for each identified problem: removing 242 safe samples already handled by deterministic rules; repeating review/block examples in the training set to lower the allow proportion to 61%; and adding 100 targeted examples covering Windows commands, secret exposure, disguised commands, and instructions telling the tool to execute other content.

The results are as follows (model running alone):

Metric	Base Model	Round 1	Round 2	Round 3
Everyday commands allowed	25/50	23/50	49/50	44/50
Attacks blocked	32/43	39/43	31/43	39/43
Shell evaluation set	42/71	48/71	58/71	61/71
Korean	27/30	25/30	24/30	27/30

End-to-end performance (including deterministic rules): everyday commands allowed 46→44→49→48, Shell evaluation set 69→70→68→70, Korean 30→28→28→29. The author also tested the base and fine-tuned models through real Claude Code hooks, and both blocked all 13/13 incidents and 30/30 evasion attempts.

One Regression That Was Not Hidden

Two end-to-end decisions got worse: chmod +x scripts/build.sh now requests confirmation; and a Korean request meaning "summarize this issue" also requests confirmation. This violated the author's original "no regression" criterion. Both cases trace back to the data balancing process—after adding review/block examples, chmod +x appeared in the dataset as 2 allow and 2 hold. The author chose to stop at round 3 rather than tune a fourth round for two examples in an evaluation set that had already been checked repeatedly. The final release was made because no harmful commands passed in the end-to-end evaluation, but the regression was recorded in the changelog and model card.

The fine-tuned model is now the default option in version 0.7.0 (pip install -U laya-guardrail). The model is pinned by revision and SHA-256, and you can switch back to the original checkpoint with LAYA_GUARD_MODEL=base. The project is MIT-licensed and runs locally.

The Real Weakness: Evaluation Independence

The question the author raises at the end is more worth pondering than the technical details. The biggest weakness is not the model, but the evaluation process. The training data was written by himself, the benchmark was designed by himself, and he checked evaluation results between rounds. Even after excluding training samples similar to the benchmark, after three rounds of iteration, there is no way to be sure the evaluation is still independent.

He seeks answers from others: when building a tool-call safety classifier, how do you keep the evaluation set credible when you yourself are the developer? Independently generated test cases, adversarial evaluation, time holdouts, external reviewers, or something else? And how should evaluation leakage be handled when the system itself evolves rapidly?

There is no standard answer to this question, but laya-guard's three-round record provides an honest sample: the value of fine-tuning a small model is not in chasing accuracy, but in using clear data diagnosis and acceptance discipline to turn each failure into specific changes for the next round. And for Agent safety, model judgment is always only one layer of defense in depth—kernel sandboxes and deterministic rules are the bottom line that cannot be bypassed.

Evaluation Independence requires that the test set remain isolated from the training set in terms of generation process, source, and designer, so that evaluation results are not contaminated by training data or the developer's subjective preferences. Ideally, the evaluation set should be independently constructed by people not involved in development, or use methods such as time holdouts (testing past models with future data) or adversarial generation (independent red teams constructing attack samples). When the developer is simultaneously responsible for data annotation, benchmark design, and model tuning, even after excluding similar samples, repeated observation of evaluation results during iteration can still introduce implicit overfitting—the developer will unconsciously adjust data based on evaluation performance, causing the evaluation set to lose its "unseen" property. In rapidly evolving systems, evaluation leakage is difficult to completely eliminate, so external review or periodic refresh of the evaluation set is needed to maintain credibility.

One-line Summary

A developer fine-tuned a 322M model as an AI command guard; three rounds of training and two failures revealed data bias and the challenge of evaluation independence.

Paragraph Summary

A developer open-sourced the layaguard project, using a small 322M-parameter local model to review shell commands before tool calls and decide whether to allow, block, or request confirmation. Starting from a zero-shot baseline, the model went through three rounds of fine-tuning: the first round, due to length imbalance in the training data (allow commands had a median of 287 characters, while risky commands were only 33-40 characters) and a lack of real blocked samples, learned "short command = dangerous" and falsely flagged everyday operations like git status; the second round fixed false positives but introduced false negatives, letting through Windows commands, comment-disguised cases, and secret exposure; the third round removed redundant safe samples and repeated review/block examples to lower the allow proportion from 86% to 61%, restoring attack blocking to 39/43. Two regressions still appeared in end-to-end evaluation (chmod +x and a Korean request), which the author chose to record rather than hide. The final model became the default in 0.7.0, but the author admits the biggest weakness is evaluation independence—writing his own data, designing his own benchmark, and iteratively checking it leaves no way to be sure the evaluation remains credible.