CodeCook: A VS Code Extension That Slashes AI Coding Token Costs with Context Compression

The "Old Tab" You Keep Paying For in Long Sessions

Anyone who uses AI for coding has probably run into this frustrating reality: during a long coding session, every new message sends the entire conversation history — all previous messages and tool outputs — back to the model. In other words, you're paying for the same historical context over and over again. Your token bill keeps ballooning, but the actual work being done doesn't grow at the same rate.

This is exactly the pain point that the open-source project CodeCook aims to solve. The author's goal is straightforward: let developers comfortably use larger models without constantly watching their token consumption.

What Is CodeCook

CodeCook is a free, MIT-licensed VS Code extension — essentially a coding agent embedded in the sidebar. Its core positioning is BYOK (Bring Your Own Key): you run it with your own API keys, pay the model providers directly, with no markup in between and no platform lock-in.

This model is quite developer-friendly for heavy users. Compared to some subscription-based or per-seat AI coding tools, BYOK hands cost transparency and control back to the developer — you can clearly see where your money goes, how much you're spending, and you can switch providers at any time.

Core Mechanism: Compressing Old Conversation Rounds

CodeCook's approach to cost reduction focuses on how historical context is handled.

Automatic Summarization of Older Rounds

Earlier conversation turns and bulky tool outputs aren't resent verbatim with every request — they're summarized and compressed. This way, you primarily pay for newly generated work, not for repeatedly transmitting stale history.

The author presents a striking data point: in one long session, token consumption for a single round dropped from 800,787 to 14,857 — a reduction of roughly 98% — while the corresponding plan and work output remained intact. It's worth emphasizing that this is an extreme single-round case from a long session, not an average figure. Actual savings will vary with session length. The author honestly acknowledges this in the original post.

Summarization-based context compression is a classic "long-term memory management" strategy in agent frameworks. The basic idea is to use an LLM to generate concise semantic summaries of early conversations, preserving core information with fewer tokens, then substitute those summaries for the original content in subsequent requests. This has some analogy to how human memory works — we compress distant experiences into key points rather than recalling them word for word. However, an inherent limitation of this approach is that summarization is lossy compression. When an LLM generates a summary, it makes selective choices about what to keep, and technical details, specific values, and edge cases — precisely the things that matter most in coding tasks — have a higher probability of being dropped. The design decision to keep the plan persisting independently of summaries is a targeted countermeasure against this weakness.

Plans Survive Summarization

The biggest risk of context compression is that summaries may lose critical information, causing the agent to effectively "forget" things. CodeCook's response is to let the "plan" persist independently of summaries. You can build a plan, then reopen it from the command palette later, and the agent will continue following that plan throughout a long session.

Cost Visibility and Spending Caps

The tool has a built-in token optimizer, savings reports, and cost estimates. It also supports setting daily, weekly, or monthly spending caps — meaning a long session won't suddenly hit you with a surprise bill.

Understanding this mechanism requires knowing how large language model context windows work. Every API call submits the full conversation history as input tokens. This means in a 50-turn session, the cost of turn 50 is roughly the sum of all 49 prior turns plus the new content. As sessions grow longer, token consumption increases near-linearly. In coding agent scenarios that use tools (like code execution, file reads and writes), the full output of every tool call also gets added to the history, accelerating the inflation further. This is a structural problem faced by all AI applications based on conversation history — not a challenge unique to CodeCook.

Designed for Real Development Work

CodeCook isn't just a cost-saving gimmick — it includes thoughtful engineering details:

• Checkpoints at every turn: You can review native diffs and choose to keep or revert all changes, or even roll back to an earlier turn and restore file state. This reversibility is critical for reviewing AI-generated code.
• Flexible context addition: Highlight a snippet in the editor or select text in the terminal, then choose Add to Chat to include it in the conversation directly.

These features make it feel more like a productivity tool that fits into everyday workflows, rather than a purely experimental demo.

Supported Model Providers

CodeCook covers the major model channels, including Google Gemini (Vertex AI or API key), OpenAI, Anthropic Claude, Azure OpenAI, and AWS Bedrock. You only need to connect one of them to get started. This multi-provider compatibility further reinforces the "no lock-in" philosophy.

Vertex AI and using a Gemini API key directly are two different access paths: the former calls models through Google Cloud's enterprise infrastructure, supporting VPC network isolation, audit logging, and IAM access controls — suitable for teams with compliance and data residency requirements. The latter calls directly via a key issued from Google AI Studio, with lighter configuration, making it better for individual developers getting started quickly. AWS Bedrock works similarly — it's Amazon's managed model inference service that lets you call models from multiple vendors, including Claude, within AWS private networks. User request data is not used to train models, which offers additional data security value for enterprise users.

Getting Started

According to the author's steps, the setup process is quite straightforward:

1. Install the CodeCook extension from the VS Code Marketplace
2. Open CodeCook from the activity bar and sign in with Google or GitHub — this creates an account stored only locally in VS Code; CodeCook does not gain access to your code repositories
3. Connect a model provider
4. Have the agent execute tasks in your workspace and review the generated diffs

The local account design is worth noting for privacy: the sign-in process doesn't involve uploading your code to a third party, which is a plus for teams that prioritize code security.

A Few Thoughts

Context compression itself isn't a new concept — many agent frameworks are exploring similar "memory management" strategies. But CodeCook packages it together with BYOK, cost visualization, and spending caps into a ready-to-use VS Code product, lowering the barrier for everyday developers.

That said, the "98%" figure deserves some perspective — it's a single-round extreme case and shouldn't be taken as a general expectation. Summarization compression always carries the potential cost of information loss alongside its token savings. Whether an agent might make flawed judgments in complex tasks due to compressed context still needs to be validated in real long sessions. The author is actively soliciting user feedback, with particular interest in actual savings in real sessions, missing providers or features, and rough edges in the experience.

For developers frustrated by AI coding token bills, this is a free, open-source option worth trying.