Stop Feeding Whole Notebooks! Code Diff Approach Saves 95% of Tokens

Why Is Your Token Bill on Fire?

If you're in the habit of writing code in Jupyter Notebooks and then throwing the entire .ipynb file at Claude Code or Codex CLI for review, you've probably already noticed the absurdly high cost. A Kaggle developer measured it and found that a single notebook can easily consume around 30k tokens — ask a few times back and forth, and the bill spirals out of control.

The root of the problem is that .ipynb is fundamentally JSON, packed with cell outputs (logs, images) and metadata, while what the LLM actually needs is often just the code itself. This developer shared a combination of techniques on Qiita that cuts the token consumption per review down to 1k–1.6k — a reduction of over 95%.

The Core Technique: Send Only Code Cell Diffs

Strip Outputs and Metadata

The author wrote a script that automatically discards the outputs and metadata in the notebook, extracting only the code cells. This step alone eliminates the vast majority of redundant volume — after all, when the LLM reviews code, it doesn't need to know what logs your last run printed.

An .ipynb file is essentially a JSON document following the nbformat specification, whose top-level structure contains a cells array, a metadata object, and an nbformat version number. Each cell is further divided into cell_type (code/markdown/raw), source (a list of source lines), outputs (code cells only), and metadata. The outputs may nest text/plain logs, base64-encoded image/png (a single screenshot can be hundreds of thousands of characters), application/vnd.plotly and other rich media — all of which is pure noise for inline AI review. In addition, per-cell metadata like execution_count and collapsed also quietly inflates the size. Understanding this structure makes it clear: the only field that actually carries logic is the code cell's source, and stripping the rest is an operation that loses nothing in review quality while massively compressing the context.

Output Only the Diff From the Previous Version

A cleverer approach: the script outputs only the diff relative to the previous version. In iterative development, each change is actually small, and sending the full code is entirely unnecessary. The diff precisely presents "what changed this time" to the model, making the context cleaner and the review more focused.

diff (difference comparison) is a foundational concept in version control, usually presented in unified format: lines starting with - are deletions, + are additions, and @@ marks the location and line numbers of the changed hunk. For an LLM, the value of a diff isn't just saving tokens — it constrains the model's attention to "what changed this time," preventing the vast amount of unchanged logic in the full code from diluting the review focus. This aligns with how humans do code review: reviewers look at the pull request patch, not the entire repository. In the notebook scenario, since cells can be freely added, deleted, reordered, and modified, the diff granularity typically falls at the cell level, then does line-level diffing within each cell's source — balancing accuracy and compression ratio.

Replace Overlong Lines With Length + Hash

Notebooks commonly contain very long strings (such as embedded prompt templates), which consume a lot of tokens and hurt readability when sent directly. The script replaces lines exceeding 300 characters with "length + hash" — the LLM only needs to know whether the line changed, not see the full text.

Separate Reading and Writing Views

The author also proposes an engineering habit: split the lengthy prompt strings in the notebook into standalone .txt files, forming a "reading view." The notebook that actually runs stays as a single file, while the reading view is what gets sent to the LLM for review. Execution and reading are two separate roles, and both sides stay clean.

Calling AI Coding Assistants From the Command Line

Claude Code: Disable Tools, Feed Only the Summary

With claude -p you can do a single question-and-answer without entering the interactive interface:

claude -p --max-turns 1 --disallowedTools "Bash,Read,..."

The key point is to disable tools like file reading and pass only the summary you've prepared yourself. Otherwise the model will actively go read files in the repository, and token consumption immediately rebounds. Only by reclaiming the initiative is the cost controllable.

The -p in claude -p stands for --print, which is Claude Code's non-interactive mode: the prompt is passed as an argument or via standard input, and the model exits directly after outputting the result without entering the multi-turn REPL interface — making it ideal for scripted invocation. --max-turns 1 limits the model to at most one round of reasoning and tool-calling loop, preventing it from retrying even after tools are disabled. --disallowedTools is a blacklist where you can list the names of tools to forbid (such as Bash, Read, Grep, Glob); disabled tools won't appear in the model's available tool list. Claude Code by default has the ability to autonomously read/write files and execute commands — that's its selling point as a coding agent, but in the scenario of "only review the summary I send you" it instead becomes a cost leak: the model will actively dig through the repository to complete the context. Only by narrowing permissions with the blacklist can you keep the context boundary firmly in the caller's hands.

Codex CLI: Pass the Question via Standard Input

Codex CLI is installed via npm: npm i -g @openai/codex. When browser login fails, the author found that device-code authentication can bypass it:

codex login --device-auth

Questions are then passed via standard input:

echo "Your question here" | codex exec --skip-git-repo-check -

Windows PowerShell Pitfalls

Japanese Garbled Text: Set UTF-8 in All Three Encoding Places

When piping a Japanese prompt to the CLI in PowerShell, garbled text appears. The solution is to set the input, output, and $OutputEncoding all to UTF-8:

[Console]::InputEncoding  = [System.Text.Encoding]::UTF8
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$OutputEncoding           = [System.Text.Encoding]::UTF8

This issue applies equally to all non-ASCII characters (including Chinese), so Windows users would be wise to configure it in advance.

The pipe encoding issue in Windows PowerShell stems from its historical baggage: the Windows console defaults to the system code page (such as GBK/CP936 in Chinese environments, CP932 in Japanese environments), while modern CLI tools generally read and write standard input/output as UTF-8. When text passes through a pipe, PowerShell encodes the string into a byte stream using the encoding specified by $OutputEncoding and feeds it to the child process; if that value is still the legacy code page, non-ASCII characters get corrupted during the encoding conversion and appear as garbled text. The three settings correspond to different stages of the pipeline: [Console]::InputEncoding controls the encoding used when the console reads input, [Console]::OutputEncoding controls the encoding used when the console displays output from external programs, and $OutputEncoding controls the encoding used when PowerShell writes to the pipe of an external program. Only by unifying all three to UTF-8 can you ensure non-ASCII characters aren't incorrectly transcoded from input to output.

You Can't Chain Commands With &

PowerShell doesn't support cmd-style & command chaining and will throw an AmpersandNotAllowed error. People migrating from cmd easily trip up here.

The Boundaries of This Approach

It should be noted that this solution is built on an "iterative review" workflow — you write the code first, then let the model look at the incremental changes. If it's the first time letting the model understand the entire project structure, the diff strategy doesn't apply, and you still need to provide a certain amount of full context. Also, while disabling tools saves money, it means the model can't verify the actual dependencies of the code in the repository on its own, which reduces the depth of the review.

For notebook developers who frequently go back and forth with LLMs, this is still a very high-ROI optimization: a script of a few dozen lines buys you over 95% in token savings.

Key Takeaways

• Send code cell diffs instead of the whole notebook — tokens drop from ~30k to 1k–1.6k
• Replace overlong lines with length + hash — saves tokens while preserving "whether it changed" information
• Use claude -p with --disallowedTools to disable tools and prevent the model from reading files on its own
• When Codex CLI login fails, switch to codex login --device-auth
• When handling non-ASCII text in PowerShell, unify all three encoding settings to UTF-8

One-line Summary

Strip notebook redundancy and send only code diffs, combined with command-line tool invocation, to cut tokens by over 95%.

Paragraph Summary

The article addresses the problem of token costs spiraling out of control when handing entire Jupyter Notebooks directly to an LLM for review, and presents an engineered compression and invocation solution. The core idea: .ipynb is essentially JSON stuffed with cell outputs and metadata, and only the code is truly useful. Through a script that strips outputs and metadata, sends only the diff relative to the previous version, replaces overlong lines with length plus hash, and splits prompts into standalone txt files to form a reading view, the author cuts per-review tokens from about 30k down to 1k–1.6k. At the invocation level, Claude Code uses claude -p with --disallowedTools to disable file reading and prevent the model from digging through the repository itself, while Codex CLI passes questions via standard input and uses device-code login to bypass browser authentication failures. The article also flags the pitfalls of unifying the three PowerShell encoding settings to UTF-8 and the lack of & command chaining, and notes that this approach suits iterative review while still having limitations for first-time project understanding or when repository dependencies need verification.