VQ Agent Core's Three-Layer Agent Loop Explained: Making Agents Auditable

Most agent frameworks hand the steering wheel, the gas pedal, and the rearview mirror to a single large model. VQ Agent Core takes a different path: a locked THINK → ACT → DECIDE → THINK loop that splits responsibilities across three layers. According to an analysis of this open-source project from rss feeds, what's worth discussing here isn't the appearance — it's the architectural decisions themselves.

The Core Loop: Three Layers, Each With Its Own Job

agent.py defines the Agent class. Input first goes to the THINK layer (think.py: understand()), which returns an actionable_request that the ACT layer then routes on. ACT is handled by needle_router.py, built on Needle 2's (cactus-needle==2.0.15) grammar-constrained function calling — meaning tool names and arguments are valid at the construction level, rather than passing format validation by luck.

When routing confidence falls below router.confidence_threshold (default 0.7 in config.yaml), or when the request itself is ambiguous, the agent invokes _decide_gate() — the DECIDE layer. THINK's compose() then writes the final response. All events are written to runs/run-<timestamp>.jsonl, one file per run, forming a complete audit trail.

Grammar-Constrained Function Calling: The traditional approach lets the model freely generate text, then uses a JSON parser or regex to validate whether the output matches the tool-call format, retrying or erroring out when it doesn't. Needle 2 takes a different path — it constrains the vocabulary during decoding itself, restricting the output space to valid function names and argument structures, so the generated token sequence satisfies the grammar at the construction level. The direct benefit is the elimination of "format hallucination": the model cannot output nonexistent tool names or arguments of the wrong type. The cost is reduced flexibility and limited support for nested or free-text arguments, which is why such engines are typically used only for routing and structured decisions rather than open-ended generation.

THINK's Honest Degradation Mechanism

This is the most commendable part of think.py. When no local model is available:

• understand() returns the raw request tagged with via="skipped" — recorded, not hidden.
• compose() returns a factual structured summary of what actually happened, never fabricating LLM-style prose.

The media describer inlines text files under 100KB and marks binary media as "described via metadata only," rather than making up content. The code comment says it plainly: "Never fakes content." More agent code should have this property.

The ACT Layer: A Restrained Tool Registry

Tools live under the tools/ directory with a registry (tools/__init__.py, tools.get(name)). The project tree includes local_tools.py (draft_text), system_tools.py (get_time, sysinfo), web_tools.py (web_search, fetch_url), plus notes_tools and decide_tools. Deliberately kept minimal — this is a registry meant for extension, not a grab-bag with everything stuffed in.

Needle 2 runs in-process on desktop via ctypes, or connects over HTTP to the needle CLI binary (serve mode) — that's exactly the Termux/Android path, letting the same engine run on a phone.

The DECIDE Layer: Four Judges, One Interface

deciders.py is a factory selected via the decider: key in config.yaml (jev | local | auto):

1. Jev (TypeSafe cloud) — a dedicated decision model returning choice / score / noul in a single call.
2. Strands Decider 2B (local) — Qwen3.5-2B + pointer head + LoRA, Apache-2.0 licensed, ~4.5GB of weights, producing calibrated typed answers in a single forward pass.
3. Laya (local) — a ~1GB dedicated decision model.
4. Prompt-based local LLM — the last resort, strict JSON prompting, uncalibrated but free and available offline.

LocalDecider._backends = ("strands", "laya", "prompt") tries them in order, with each backend throwing a typed *DeciderUnavailable exception caught by the runner. All four backends expose the same interface: decide(), classify_reply(), score_lead(), should_escalate(). Swapping Jev for an on-device solution requires no changes to calling code — just a config value. auto mode means Jev first, falling back to local on failure. If everything fails, the error propagates up and the agent escalates to a human — a safe default, written into the code.

On specific tasks: classify_reply() turns a prospect email into labels like demo_requested with confidence scores; score_lead() returns lead temperature and fit (1–5); should_escalate() returns a probability and a recommended action. Tools read the same VQ_DECIDER environment variable set by agent.py, so standalone tool calls stay consistent with the running agent.

Calibration and Pointer Heads: Uncalibrated model output probabilities are often disconnected from true accuracy — for example, the model says "0.9 confidence" but is actually only 60% correct, which breaks downstream threshold decisions. Strands Decider 2B emphasizes "calibrated typed answers" because it adds a pointer head on top of Qwen3.5-2B — a lightweight classification head that maps hidden states directly to a distribution over predefined answer options, rather than letting the model freely generate answer text. This approach keeps outputs naturally within a closed label set and, through training, brings the probability distribution closer to true accuracy. For agents that need to "decide whether to escalate to a human based on a score," calibration matters more than raw accuracy, because it determines when thresholds actually work.

should_escalate() and Human-in-the-Loop Escalation: This is a key safety valve in agent reliability design. When all automated decision backends are unavailable, or when should_escalate() returns a high probability, the system hands control to a human rather than forcing an answer. This design acknowledges model capability boundaries and avoids producing low-quality output under uncertainty. In the same spirit as "honest degradation," the escalation mechanism makes failure explicit — better to stop and wait for a human than to pretend success. In customer service, lead scoring, and similar scenarios, this behavior directly affects the cost of misjudgment, so having a reliable escalation path often determines production viability more than the model's own accuracy.

Getting It Running

git clone https://github.com/Shivay00001/vq-agent-core.git
pip install -r requirements.txt
# Point local_model.base_url in config.yaml to an OpenAI-compatible endpoint
# decider: jev | local | auto
python agent.py "score this lead: ..."

fetch_model.sh helps with model downloads; phone-setup.md covers the Termux path (Needle serve backend + prompt-based local decider, since torch can't be installed on Termux). The repo also includes test_agent.py, test_strands_backend.py, and dated engineering notes from real end-to-end runs.

An Honest Assessment

This is a working foundation, not a finished product: no UI, no hosted service, THINK's multimodal capability depends on the model behind whichever endpoint you connect, and local DECIDE backends need a PC-class machine. But the features that really matter — honest degradation, a swappable calibrated judgment layer, a complete run trace — are in the code, not the README. If you're building agents that need to be auditable or run offline, it's worth a read.

Repo: https://github.com/Shivay00001/vq-agent-core

Audit Trail and JSONL Event Streams: Each run produces a runs/run-<timestamp>.jsonl file, the technical foundation of the "auditable" selling point. JSONL (one JSON object per line) is easy to append to, stream-read, and parse line by line — well suited to recording the scattered temporal events of an agent run: THINK's input understanding, ACT's routing results, DECIDE's judge scores, the final compose output. Compared to stuffing state into a single large JSON, a line-by-line event stream naturally supports incremental persistence and crash recovery, and makes it easy to replay the entire decision process along a timeline afterward. For agent scenarios requiring compliance, debugging, or reproducibility, this "run-as-log" design is more reliable than an optional logging switch, because it can't be bypassed.

One-line Summary

VQ Agent Core builds an auditable, offline-capable agent foundation with a three-layer locked loop and a swappable decision layer.

Paragraph Summary

VQ Agent Core uses a locked THINK→ACT→DECIDE→THINK loop to split agent responsibilities into three layers instead of cramming everything into a single large model. The ACT layer relies on Needle 2's grammar-constrained function calling, making tool names and arguments valid at the construction level and eliminating format hallucination; the DECIDE layer unifies four backends — Jev, Strands Decider 2B, Laya, and a prompt-based LLM — behind a factory pattern, swappable via config alone. The THINK layer degrades honestly when no local model is available, returning factual structured summaries instead of fabricated content. All events are written to per-run JSONL files forming an audit trail. The project positions itself as an auditable, offline-capable foundation rather than a finished product with a UI or hosted service, and local decision backends require PC-class hardware.