Running a 7B Model on CPU: Lessons from Building LocalAgentBrowser, an AI Browser Agent

Running a 7B parameter model on a consumer-grade CPU to power a browser agent that autonomously completes research and writing tasks — this sounds like something that requires a cloud GPU cluster. But the open-source project LocalAgentBrowser proves that after 30 iterations of refinement, a single Intel i5-9500 can let Qwen2.5-7B-Instruct handle the complete workflow from searching and reading to writing answers.

The project author documented the entire development process, testing and measuring every change in real runs. The end result: a four-turn conversation completing a research task on Africa's highest peak in about twelve minutes, with only search requests leaving the local machine.

System Architecture: A Single-File Python Agent

The entire agent is one Python file. Each turn, the model receives the goal, information discovered so far, a log of recent actions, and the current page content — then outputs a JSON-formatted action command.

The agent supports five actions: SEARCH, NAVIGATE (go to a link), READ_MORE (read the next portion of a long page), DISCOVER (record a finding), and FINISH. LM Studio constrains the model's output format via JSON schema, while Playwright executes each action in a visible Chromium window.

This basic architecture isn't complex. The real challenge is keeping a 7B model on track — small models easily fall into repetitive searching, forgetting to record findings, and drifting off topic.

LM Studio and JSON Schema output constraints: LM Studio is a local LLM runtime environment built on top of llama.cpp. It supports passing a JSON Schema at inference time, and the server enforces that the model's output conforms to that structure — this is critical in agent design, because 7B models natively produce outputs that often include explanatory text, missing quotes, or misspelled field names, causing parsing failures. Structured output turns the question of "whether the model's output can be consumed by the program" from probabilistic to deterministic, making the action dispatch logic simple and reliable by routing on the action field. Similarly, Playwright is Microsoft's open-source browser automation framework that can drive a real Chromium instance, supporting navigation, clicking, and DOM text extraction — complementing this project's observable design of "executing actions in a visible window."

Core Lesson 1: Rules Belong in Code, Not Prompts

The author's first attempt was to write "don't repeat searches" in the prompt — but the model repeated them anyway. For small models, reading a rule and then still doing the most likely thing is the norm, especially after multiple turns in a long prompt.

The solution was to enforce every checkable rule in code: repeated searches are intercepted, opening a page you're already on is rejected, accessing the same URL more than three times is blocked, and blank findings and duplicate findings (detected via word pairs, so rewording or reordering can't bypass it) are all filtered. Rejected actions are returned to the model with a reason.

Two costly lessons here. First, the checkers themselves need testing: an early version of the duplicate URL check ignored query strings, causing every search after the first to be flagged as a duplicate — 34 out of 35 turns were blocked. Second, when the model keeps making the same mistake, replay the prompt: the model kept sending empty DISCOVER parameters, and replaying revealed it thought DISCOVER was a command that would automatically extract information. Adding one line to the prompt stating "you must write the information itself into the parameter" solved it completely.

Core Lesson 2: Prompt Reading Is the Main Cost on CPU

On this machine, the model reads at roughly 8 tokens per second, and reading the prompt accounts for about 90% of each model call's time. A 4,000-token prompt takes several minutes before the model starts outputting its first token.

But LM Studio (and other llama.cpp-based servers) skips the identical prefix portion of a prompt from the previous call. So instead of rebuilding the prompt each turn, the agent maintains a growing conversation, appending only what changed: a line of results, the next section of a page, a new page. A new prompt is only rebuilt when the conversation approaches the context limit.

Optimization	Effect
Reorder prompts so consecutive turns share a longer identical prefix	~25% faster per turn
Continue the conversation instead of rebuilding	Same-page turns drop from 150-220s to 35-120s
Correct rejected actions in the same conversation ("cheap retry")	25-45s instead of wasting a full 150-200s turn
Shorten old findings in new prompts; rebuild after 75% context use	New prompts drop from 234s to 162s on average

The "cheap retry" mechanism matters more than it appears. Small models make mistakes often, and if every rejection wasted a full turn, the agent would spend most of its time being rejected. If all it takes is appending a short note, the cost of making mistakes becomes extremely low.

Prompt caching and KV Cache reuse: llama.cpp-based inference servers keep the key-value cache (KV Cache) from the previous request in memory. When a new request's prompt prefix is identical to the previous one, those tokens don't need to be recomputed. The agent's "growing conversation" strategy exploits exactly this: each turn only appends new content, the history stays unchanged, so only the few hundred new tokens need to be re-read. This also explains why reordering the prompt speeds things up — putting stable content (system instructions, goal) first and changing content (page text, results) later maximizes the identical prefix length between consecutive turns. Understanding this mechanism is a key prerequisite for designing agents on slow hardware.

Core Lesson 3: Small Models Don't Know When to Stop

This is the real bottleneck. The agent would find good material and then keep browsing, or drift to loosely related pages, or leave a page without recording anything. Three mechanisms taught it to finish for the first time:

• Progress check: after four consecutive turns with no new findings, it's asked to compare findings against the goal, then either finish or state what's missing and go find it
• Goal check every eight turns: executed even when findings keep flowing, because loosely related findings keep resetting the progress check
• Confirm before leaving: when leaving a page without recording anything, a short question is triggered asking it to record or confirm the departure

With these mechanisms, version 1.27 autonomously completed a documentation research task: 13 turns, about 36 minutes, outputting the requested commands and file layout.

But the opposite problem then appeared. Facing the mountain peak question, the agent recorded the complete answer at turn 3, wrote "goal achieved" in its own reasoning, yet kept reading through 33 sections of the article looking for interesting details. The fix was adding a reminder after every new finding: if your findings already cover every part of the goal, finish immediately. Next run: finished at turn 4.

Core Lesson 4: Demand Evidence, Not Yes-or-No Answers

Ask a 7B model "do your findings satisfy the goal?" and it answers "yes" far too easily. In a task about researching how spiritual traditions view cats and forming an opinion, it finished after reading only one blog post that mentioned no specific tradition at all, delivering a summary instead of an opinion.

The more effective approach is demanding it show its work. Before finishing, it must break the goal into parts, write out each part and the finding covering it, or mark it "uncovered." It's also told how many independent sources its findings came from. In the next run, it listed six traditions from two sources and gave an opinion.

The model still finds loopholes. In a comparison task, its own check wrote "available libraries: uncovered. Will write at FINISH," then finished anyway — it had stretched wording meant for its own opinion onto information it hadn't found yet. The current handling: if "uncovered" appears in the finishing reasoning, it's sent back to work on that part, at most twice per run, ensuring content truly not available online doesn't block forever.

Core Lesson 5: Writing Is a Separate Job

Initially, the agent's FINISH text was the final result. Small models perform far better when writing is their only task, so the final result is now a separate model call that sees only the goal and a numbered list of findings, with a short numbered list of rules.

Every rule comes from a real failure. When the writing step continued the browsing conversation (to reuse cache), it used page text the agent had read but never recorded, and cited findings that didn't support its claims — the fix was a fresh prompt containing only findings. In a sourdough tutorial task, it fabricated the first water amount as "45g (about 1/4 cup)," when 45g was actually the daily feeding amount. For a task requesting three Photoshop alternatives, it listed four plus an unrequested opinion — the cause was a rule buried in a long paragraph; switching to a short numbered list produced exactly three with no extra opinions next time.

Context pollution and separation of concerns: When the writing step carries over the browsing conversation, the model can "see" all the page text it read but never recorded, causing it to use unverified content as established evidence. This is cognitively similar to context pollution — irrelevant or unconfirmed information interfering with judgment on the current task. Splitting a small model's single call into single-responsibility steps (collect, write, verify) is essentially separation of concerns: each call carries only the context that step requires, and the model doesn't have to make complex judgments about "should I use this information." Unlike large models that can rely on long context for self-discipline, small models are far weaker at resisting interference from mixed information in context.

Core Lesson 6: Test Different Task Types

Until version 1.29, all tests used the same goal — which is exactly how you overfit a prompt. Six different task types each exposed new problems:

• Documentation research: completed in 13 turns (~36 min), including commands, roles, and file layout
• Multi-source opinion: six traditions from two sources plus an opinion, 18 turns (37 min)
• Factual question: correct answer with citations, 4 turns (~12 min)
• Comparison and recommendation: comparison and recommendation with citations, 16 turns (33 min), one section uncovered
• List task: exactly three requested items, all cited, 9 turns (24 min)
• Step-by-step tutorial: clear plan in 7 turns (~19 min), with one fabricated measurement that led to introducing a number check

Prompt overfitting: When developers repeatedly tune prompts, rules, and checkers against the same task goal, the system gradually adapts to that task's surface features rather than truly learning a general research process. This shows up as performing well on the training goal but exposing new flaws the moment you switch to factual Q&A, list tasks, or step-by-step tutorials. This is homologous to overfitting in machine learning: the optimization signal is too concentrated, and generalization is sacrificed. The solution is to introduce a diverse task set as a "test set" early and proactively, so every change is validated across tasks rather than repeatedly polished against a single goal.

Unsolved Problems

The agent's main current weakness is reading without recording — even with the page's heading outline right in front of it, it sometimes reads through a long page section by section without recording the relevant parts. Also, when facing strict goals like "don't finish before fully verifying," it keeps searching for details that may not exist online. On judgment, results depend heavily on the first few pages search returns, and the 7B model sometimes cites the wrong finding number, so claims need to be manually checked against the findings list.

Advice for Small-Model Agent Developers

The author summarized six lessons: put every checkable rule into code and tell the model why it was rejected; make mistakes cheap by correcting them in the same conversation; design for prompt caching on slow hardware — append rather than rebuild; don't ask "are you done," demand a part-by-part display of goal coverage; separate collection from writing, giving the writer only evidence; and test different task types early while measuring every change.

The project is open-sourced on GitHub (SnowFXXX/LocalAgentBrowser), installable via pip install localagentbrowser, requiring LM Studio to load the model and Python 3.10 or above. The author used Claude as a coding partner starting from version 1.13. These lessons are valuable reference for any developer building AI agents in resource-constrained environments, especially those wanting data and computation to stay fully local.

One-line Summary

Open-source LocalAgentBrowser lets a 7B model autonomously complete research and writing tasks on a GPU-less i5, distilling six lessons for small-model agent development.

Paragraph Summary

LocalAgentBrowser is a single-file Python browser agent that runs Qwen2.5-7B-Instruct on just a consumer-grade Intel i5-9500 CPU, autonomously completing the full workflow from search and reading to writing answers — finishing an Africa's highest peak research task in about twelve minutes across four turns, with all computation and data staying local except for search requests. After 30 iterations, the author distilled core lessons: put checkable rules into code rather than relying on prompts; exploit LM Studio's prompt caching to append conversation rather than rebuild, dropping same-page turns from 150-220s to 35-120s; use external mechanisms like progress checks, goal checks, and confirm-before-leaving to help small models judge when to finish; demand part-by-part evidence coverage instead of yes-or-no answers; make writing a separate model call receiving only the goal and a numbered findings list; and test early with diverse task types to avoid prompt overfitting.