llm-echo 0.5 Released: More Accurate Simulation of LLM Token Usage

In the ecosystem of the LLM command-line tool developed by Simon Willison, there is an unassuming but highly practical debugging plugin—llm-echo. It does not call any real large language model; instead, like an "echo wall," it returns the received prompt verbatim. This seemingly simple feature can be a great help when debugging LLM applications. Recently, llm-echo released version 0.5, with a focus on improving the simulation of token usage calculations.

Why Do We Need a "Fake Model"?

When developing LLM-based applications, developers often need to test prompt assembly, multi-turn conversation management, tool calling, and other logic. Calling a real model every time not only incurs costs but also slows down debugging due to network latency and the uncertainty of model outputs. The echo model provided by llm-echo is designed to solve this problem: it performs no inference and simply presents the input in a structured way.

For example, running the following command:

uvx --with llm-echo llm -m echo 'hi there' -o thinking 1 --usage

You will get a clear JSON output containing fields such as prompt, system, attachments, and stream, fully presenting what the LLM tool actually sends to the model. This is very intuitive for verifying whether prompt templates are correct and whether attachments are loaded properly.

LLM command-line tool (llm) is an open-source CLI tool developed by Simon Willison that uses a plugin mechanism to uniformly connect to various backends such as OpenAI, Anthropic, and local models. Users initiate requests in the form llm -m <model name> 'prompt', pass model options via -o, and print token usage with --usage. A plugin (such as llm-echo) only needs to register a model ID and implement the corresponding Python interface to be automatically discovered and invoked by the main program—this is exactly the prerequisite for the echo model to seamlessly replace real models.

uvx is a command provided by the Python package manager uv for executing command-line tools in an isolated temporary environment without requiring a prior pip install. uvx --with llm-echo llm ... means temporarily installing the llm-echo plugin and launching the llm command, discarding the environment after debugging to avoid polluting global dependencies—this is also why the article emphasizes "no pre-installation required."

Core Improvement in Version 0.5: More Realistic Token Usage Simulation

According to the official release notes, the biggest change in llm-echo 0.5 is an updated calculation method for simulated token usage, which now incorporates tool calls, system prompts, and reasoning tokens into the statistics.

Previously, the token count output by the echo model may have been based simply on the length of input and output text, failing to reflect the complex token consumption structure in modern LLM interactions. As LLM tools increasingly support function calling, system role settings, and chain-of-thought reasoning, developers need more realistic estimates of cost and behavior. This improvement in version 0.5 allows the echo model to provide token usage references closer to real-world scenarios during debugging.

As can be seen from the example output, when using the -o thinking 1 option, the echo model simulates outputting a piece of reasoning text ("First I consider the prompt, then I decide what to say."), which is reflected in the final token statistics: 2 input tokens, 24 output tokens. Although these numbers are still simulated values, the calculation logic now takes reasoning and system prompts into account, making them more meaningful for debugging token budgets and truncation strategies.

Reasoning tokens refer to the tokens consumed by the internal reasoning content generated by models with chain-of-thought capabilities (such as OpenAI o series, DeepSeek^Ad-R1, etc.) before giving the final answer. This content is usually not directly shown to users but is counted in billing and context window usage, so it is often tracked separately. The llm tool requests the model to output the reasoning process through options like -o thinking 1, and the echo model correspondingly simulates a piece of reasoning text and includes it in the token count.

Tool calls refer to the ability of a model to request external function execution in structured JSON form (i.e., function calling). In an interaction involving tool calls, the tool definitions, the call parameters returned by the model, and the tool execution results all enter the context as messages, generating additional token overhead. In real applications, this part often accounts for a considerable proportion, so echo 0.5 incorporating it into simulated statistics allows developers to observe a usage structure closer to reality during the debugging phase.

What This Means for LLM Developers

This update to llm-echo reflects the maturing of the LLM application development toolchain. Early developers may have only cared about "what the model returned," but now token economics, the overhead of tool calls, and the occupation of system prompts have all become aspects that must be carefully managed. A debugging tool that can independently simulate these details without relying on a real model can significantly reduce the cost of development iterations.

For developers using Simon Willison's LLM command-line tool or related Python libraries, upgrading to llm-echo 0.5 allows them to more accurately test token usage-related logic—such as usage statistics dashboards, cost alerts, and context window management—without consuming API quotas.

The plugin follows an open-source model and can be run directly via uvx without prior installation. The project source code and release notes can be viewed on GitHub.

One-line Summary

llm-echo 0.5 released, improving token usage simulation by incorporating tool calls, system prompts, and reasoning tokens.

Paragraph Summary

llm-echo is a debugging plugin in Simon Willison's LLM command-line tool ecosystem. It does not call a real model but instead echoes the received prompt verbatim as structured JSON, making it easy to verify prompt assembly, attachment loading, and multi-turn conversation logic. The core improvement in version 0.5 is an updated calculation method for simulated token usage that incorporates tool calls, system prompts, and reasoning tokens, making the usage figures from the echo model closer to real interaction scenarios. Developers can therefore test usage statistics, cost alerts, and context window management logic without consuming API quotas or being affected by network latency. The plugin follows an open-source model and can be run directly via uvx without prior installation.