Your Local LLM Isn't Broken — Your Context Has a Bug

A small local model kept re-reading the same file during a coding task and claiming "I read it but I can't see the contents." The developer initially blamed the model for being too weak. After digging deeper, the problem turned out to be in his own Agent framework: tool results survived only a single turn before being discarded. A 4B-parameter model, in the most honest way possible, exposed this flaw.

The Failure Scene: Repeated Reads and "Invisible" Content

The author benchmarked a local LLM as an in-IDE coding Agent in a pure CPU environment (Ryzen 5 4500U, 32GB RAM, Ollama). Task T3 required the model to fix a failing pytest: find the cause, fix it, and confirm all tests pass. He handed it to qwen3.5:4b — a fast, small model.

Under the old framework, the model's performance was nothing short of a disaster: first read test_calc.py, then read calc.py, then read test_calc.py again. The framework's deduplication logic responded with "file unchanged, please use the content you already read." The model replied: "I read it, but the content isn't visible here." It then tried GREP, got zero hits, read again, exhausted six turns, and failed the task.

The author's first reaction: the 4B model isn't smart enough to follow instructions.

Ollama is an open-source tool for running local large language models, supporting loading and inference of various open-source models (such as Llama, Qwen, etc.) on consumer-grade hardware, including pure CPU environments. Through a simple API and command-line interface, it lets developers run models locally without a GPU, commonly used in privacy-sensitive or resource-constrained scenarios. The author of this article ran models like qwen3.5:4b on Ollama for Agent benchmarking. Ollama's lightweight nature makes it a common foundation for local Agent development, but it also means model inference speed is constrained by CPU performance, and context window management requires extra care.

pytest is one of the most popular testing frameworks in the Python ecosystem, used for writing and running unit tests and functional tests. It supports automatic test file discovery, rich assertion rewriting, and plugin extensions. In Agent coding tasks, pytest failures are often used as the starting point for repair tasks: the Agent needs to read the test code and the code under test, locate the cause of failure, modify the source code, and re-run the tests to confirm they pass. This article's T3 task is exactly this complete loop.

Root Cause Tracing: Tool Results Kept for Only One Turn

After tracing the Agent loop's code, the defect surfaced. The history accumulation logic had a severe asymmetry: user messages were retained, assistant replies were retained, but tool results — file contents, command outputs — existed in context for only one turn after execution, then were silently discarded.

This means: read a file in step one, and you can still see the content in step two; from step three onward, the content has vanished from context. Meanwhile, the framework's deduplication suppression mechanism was still telling the model "unchanged, use what you have," pointing to data the model no longer possessed.

The model saying "I can't see the content" was not nonsense — it was an accurate status report. It knew its history contained READ_FILE commands, but the payload had evaporated. This 4B model was actually correctly debugging the framework's context assembly logic.

Context assembly in the Agent loop refers to the process by which the framework, before each reasoning turn, concatenates the system prompt, user messages, assistant history replies, and tool call results in order into a complete prompt. The large model itself is stateless; it can only see the content in the current prompt and cannot access prior conversation memory. Therefore, the framework must be responsible for persisting all key information — especially tool-returned file contents, command outputs, etc.— and re-injecting it in subsequent turns. If the framework discards certain messages, the model will "lose its memory" in later turns. The defect in this article is precisely that tool results exist only for one turn after execution, then are silently removed, preventing the model from referencing file contents it has already read.

Surprising Discovery: Past Evaluations of the 9B Also Need Rewriting

This bug changed the interpretation of earlier test results. The author's 9B model (qwen3.5:9b) had been evaluated as "redundantly re-reading files," with 2 of 7 steps wasted under the old framework.

But the truth is: that wasn't weak memory, it was reasonable self-repair — re-fetching content that had disappeared. The stronger model was working around the author's bug, while the weaker model simply reported it honestly. This contrast reveals an often-overlooked fact: model performance is the product of model capability and framework constraints.

The Fix and a Second Bug Discovered Along the Way

The fix had two parts. First, persist tool results to the run history — each step's tool output stays in context for the entire run, and the "unchanged" message finally lives up to its name. Second, strip outer quotes from GREP patterns — the model repeatedly output GREP "member.*discount", and the literal quotes entered the regex, causing zero matches; seven turns of repeated quoting, zero hits. The parser now strips surrounding quotes, and zero-match GREP replies also include a hint suggesting simplifying the pattern or reading the file directly.

The fix shipped with Teaspoon IDE v1.5.1 under the FSL-1.1-MIT license.

GREP is a command-line tool for text search in Unix/Linux systems, supporting regular expression matching. In coding Agents, GREP is often wrapped as a tool so the model can quickly locate specific strings or structures in code via pattern matching. Special characters in regular expressions (such as ., *, ") have specific meanings; if the model's output pattern is wrapped in outer quotes and not stripped, the quotes enter the regex as literal characters, causing match failures. In this article, the model repeatedly output GREP "member.*discount", and the parser didn't strip the quotes, so the regex actually matched literal strings containing quotes, hence zero hits. After fixing the parser to strip surrounding quotes, GREP works as expected.

Controlled Experiment: Same Model from "Despair" to Shortest Path

The author ran a controlled experiment with the same model, same task, same prompt, changing only the framework's fix status.

Condition	Result
4b · Old framework	Six turns exhausted, repeated read loop, failure
4b · Fixed	Five turns to complete (pytest→read→fix→re-verify, correct)
9b · Old framework	Seven steps, including two repeated reads (self-repair)
9b · Fixed	Three steps, zero repeated reads, shortest path
e4b · Fixed	Four turns, zero repeated reads, clean completion

The same 4B model, merely by fixing the surrounding code, went from "hopeless" to "clean shortest-path completion."

Three Lessons Worth Remembering

"Weak model" and "weak framework" are indistinguishable before inspection. Every article claiming "this model is useless" may actually be evaluating the framework's constraints. If you're building Agent tools, suspect what's in the context before suspecting the model.

Deduplication and context savings must be designed around what the model can actually see. "It read this before" is only valid if that content physically exists in the prompt.

Weak models are the best framework testers. Strong models mask flaws with self-repair; the 4B model has no margin and faithfully visualized the author's bug. When testing your own Agent loop, run an edge-case model.

Data and Environment

All raw logs, task definitions, and evaluation records are public, including complete measurement data and analysis (labo.mond.jp/articles/localllm-harness-bug.en.html), fixtures and prompts and run artifacts (github.com/cuculhart/ai-editor-lab), and the Teaspoon IDE framework (github.com/cuculhart/teaspoon-ide).

The environment was deliberately kept modest: an old six-core laptop CPU, no GPU. If the framework is worse, every model will look broken. Before inspecting your model, inspect your context.

One-line Summary

A 4B local model repeatedly re-reading files exposed an Agent framework bug that discarded tool results; after fixing, the same model completed the shortest path.

Paragraph Summary

The author ran qwen3.5:4b as a coding Agent on Ollama in a pure CPU environment and found the model repeatedly re-reading the same file while claiming "I can't see the contents." Deep investigation confirmed the problem was in the self-built Agent framework: tool results (file contents, command outputs) were kept in context for only one turn, then silently discarded, while deduplication logic still told the model to "use what you've read." After fixing tool result persistence and stripping outer quotes from GREP patterns, the same 4B model went from six-turn failure to five-turn completion. Controlled experiments showed the 9B model used repeated reads for self-repair under the old framework, and completed in three steps with zero repeated reads after the fix. The core conclusion is that model performance equals model capability times framework constraints, and weak models are actually better framework testers.