GitHub Hit: The Proxy Solution That Gives Claude Code Unlimited Context

Introduction: When Long Sessions Hit the Context Wall

When running long AI coding sessions, nothing is more frustrating than the context window filling up, causing history to be truncated and agent performance to plummet. A GitHub project called Billion Context is trying to solve this pain point. According to a Bilibili creator, the project has racked up roughly 3,700 stars and 300 forks in under three months. Its core idea is to process billions of tokens through a single context window using monthly sessions—a window that can hold 100,000 tokens at a time, cutting token costs to one-fifth of the original.

More importantly, it doesn't simply truncate. It lets the model decide on its own when to compress and what to compress, while keeping the compressed content reversible. This design keeps models like Claude Code and DeepSeek^Ad from "losing their memory" during long runs, while dramatically reducing costs.

How It Works: Reversible Compression and Cache Protection

Billion Context is essentially a tiny proxy server that sits between your agent and the model API. Point your agent's base URL at this proxy, and every request passes through it. The proxy reads the conversation, compresses the parts that have already been executed, and forwards the result to the real model. It supports both Anthropic- and OpenAI-style APIs, so it works with virtually any agent.

Four Tools That Give the Model Autonomy

The most interesting part is its compression mechanism. Instead of crudely truncating at some length, it gives the model four new tools:

• compress: compress a segment of messages into a detailed summary
• decompress: restore that content when precise details are needed
• search context: perform keyword search across all compressed content
• acp status: show how much context has been used and how much more can be compressed

When to compress and what to compress are entirely up to the model. This differs from an agent's built-in summarizer: compression happens incrementally, handling small scopes rather than generalizing all history at once; and compression is reversible—everything can be decompressed back, preserving the prompt cache. A healthy session is said to achieve a cache hit rate of 95% to 97%, while compression itself costs only 2% or even less.

Prompt cache is the key to understanding Billion Context's economics. Mainstream LLM APIs cache the KV computation results of request prefixes (system prompts, conversation history, etc.) on the server side. When subsequent requests share the same prefix, the cache is reused directly, billed at a price far below regular input tokens (typically around 10% of the original price). Therefore, the "cache hit rate" directly determines the marginal cost of long sessions. Billion Context insists on reversible, incremental compression precisely to avoid altering the conversation prefix structure—once the prefix changes, the cache is invalidated and all history must be recomputed at full price. It appends compression results as new content rather than rewriting history, keeping the cached prefix as stable as possible, which is the precondition for the 95%-97% hit rate.

An agent's built-in summarizer is usually trigger-based: when the context approaches the window limit, earlier messages are summarized into a single block of text and the originals discarded. This approach has two inherent flaws: first, the summary is one-time and irreversible, so details are permanently lost; second, rewriting history breaks the prompt cache prefix. Billion Context's difference is that it turns compression from a "passive fallback" into a "tool the model actively invokes"—the model can compress just one completed tool-call result while leaving the rest untouched, giving finer compression granularity, and the original text is retained in a searchable, decompressible form. In other words, the former is a context management strategy, while the latter turns context management into an action the agent can plan autonomously.

Real-World Data: Cheaper Without Performance Loss

The project author included an article in the repo describing four and a half months of real-world usage: 174,000 model calls, 18.76 billion input tokens, and zero context window overflows on a 200,000-token model.

Third-party benchmarks further validate its value. In the Context Bench test, where a coding agent must explore real repositories, Billion Context took first place with an F1 of 0.31, beating full context's 0.29; cost dropped from $11.05 to $4.61, a 58% savings, with slightly better performance. In the Long Memo Well test, it scored 79.2% versus full context's 83.2%, while the best-performing reg setting was 62.6% but at 71% lower cost.

Interestingly, in the needle-in-a-haystack test—which requires retrieving an exact sentence verbatim—the score dropped from 69% to 27%. Summaries preserve meaning but don't reproduce text word for word, which is usually a reasonable trade-off for coding agents, but you should be aware of it.

Context Bench is a benchmark for evaluating a coding agent's ability to locate, modify, and verify within real code repositories, commonly using F1 to measure retrieval and answering accuracy; Long Memo Well focuses on maintaining and recalling long-range memory; needle in a haystack requires retrieving a specified sentence verbatim from a large amount of irrelevant text. The three have different focuses: the first two care about "whether the meaning is preserved," while the latter cares about "whether the original text is preserved verbatim." This explains why Billion Context approaches or even surpasses full context on the first two while dropping noticeably on verbatim recall—summaries preserve semantics but reword phrasing, which is an inevitable cost of compression rather than an implementation flaw.

Configuration and Ecosystem: Seamless Integration with Claude Code

Installation is done with npm install -g billion-context, linking it to your local folder, which gives you a command called billion. It has three usage modes:

1. Run billion plugin install with your client name, and it becomes a native plugin for clients like Claude Code and OpenCode.
2. Run billion directly after the client name, e.g. billion claude or billion codex, which runs the client through the proxy without changing your configuration.
3. Run billion directly, then put the proxy address at the front of your base URL.

You can use models like DeepSeek V4 Flash, Claude 3.1 Pro, GLM-5.3, Qwen 3.8 Max, and Minimax M3.1 Flash, with quota 2-5x higher than usual—just create an API key in the client console.

The Billion Context client repo lists it as a verified client based on the OpenAI-compatible interface. You just use OpenAI's base URL for this purpose, which should also work in the client's desktop version. The desktop version is the standalone app version of the client agent, well suited for long-running tasks—you can run multiple sessions in parallel, schedule tasks, and import conversations from Claude Code or Codex to continue. In the desktop version's provider list, besides the client path, there's also an OpenAI-compatible option using the same base URL, same key, and same model.

Why does this combination make sense? Claude Code gives you a flat price, but it has a 5-hour window plus weekly and monthly limits, and long sessions burn through these limits quickly because every step resends the full history. Billion Context shortens that process, so the same plan lasts much longer.

Once everything is running, open localhost 8787 in your browser and you'll see a control panel: session list, live logs, and a config editor. In supported clients, type /acp cache and you'll get a report with the cache hit rate, how many tokens were saved, and—if a cache flush occurred—the reason why.

Base URL proxying is a zero-intrusion way to integrate with clients: most clients compatible with OpenAI or Anthropic APIs allow custom API endpoints, so pointing the address at a local proxy (e.g. localhost:8787) routes requests through the proxy before reaching the real service, with no need to modify client code or configuration logic. This lets Billion Context act as a "middleman" applicable to virtually any agent, at the cost of requiring the client to support custom endpoints, and some tabs built into desktop apps may hardcode endpoints and bypass the proxy—which is exactly why the article mentions that the code tab in the Claude desktop app can't benefit from compression.

Limitations and Caveats

Billion Context is not a silver bullet. First, the project is evolving very quickly and is still at version 0.121, auto-updating from the master branch and potentially breaking at any time—better to install the stable release from npm. Second, if you run it with the host option on your own network, there is no authentication at all, so only do this on a network you trust. Third, the code tab in the Claude desktop app bypasses the proxy, so there's no compression there. Fourth, reproduction accuracy degrades—if you need to restore a precise line of code from the past, the model has to decompress it again. The documentation is extensive and highly technical, and setup is somewhat involved, requiring time to learn.

Working Alongside Orchestration Tools

If you need an AI agent for orchestration, Pentacode has an architect that tracks all tasks and distributes work to multiple performers, with Claude as one of the tools. The architect session lives the longest and grows the most, so this is where compression is needed most. Since agents can work on PyOpenCode or Pimicode, and these tools all have the billion plugin built in, you can keep the whole team lean.

In short, Billion Context is a proxy mechanism that lets a model use tools to compress its own history into small, recoverable chunks while keeping the cache intact. Tests on DeepSeek V4 Flash show it's cheaper than full context and even slightly better on coding tasks. Hook it up to Claude Code or Claude Desktop, and you can keep sessions going with cheaper open-source models. If your long agent sessions keep getting interrupted or cost too much, this approach is worth a try.

One-line Summary

Billion Context uses a reversible compression proxy to stop long sessions from overflowing—cheaper without performance loss.

Paragraph Summary

Billion Context is a small proxy server deployed between an agent and the model API. Through four tools—compress, decompress, search context, and acp status—it hands the decision of "when to compress and what to compress" to the model itself. It uses incremental, reversible compression that avoids breaking the prompt cache, achieving a 95%-97% cache hit rate at a compression cost of only about 2%. The author's four-and-a-half-month real-world testing showed 174,000 calls and 18.76 billion input tokens with zero context overflows; on Context Bench it scored an F1 of 0.31, slightly above full context's 0.29, at 58% lower cost. The trade-off is reduced verbatim reproduction, with needle-in-a-haystack dropping from 69% to 27%. The project supports Anthropic- and OpenAI-style interfaces and can be used as a plugin for clients like Claude Code and OpenCode.