TokenPrune
An open-source, zero-dependency Python package that reduces redundant tokens in Agent tool output through deterministic pruning and table compression. It does not require calling a second model and is installed with pip install tokenprune.
Timeline (last 90 days)
TokenPrune is an open-source, zero-dependency Python package that reduces redundant tokens in tool output through deterministic pruning and table compression in local memory, without calling any external model.
TokenPrune can be installed via pip install tokenprune.
The core usage of TokenPrune is to wrap tool functions with the @prune_tool decorator, which automatically analyzes the structure of the returned dictionary, strips null values, compresses repetitive key-value patterns, and converts list-type data into a compact table form.
According to benchmark tests published by the author, TokenPrune achieves 70%–90% token reduction on enterprise-grade API payloads.
TokenPrune's runtime overhead is under 0.5 milliseconds, and the number of external API calls is 0.
TokenPrune is implemented based on AST tree-shaking using only the pure Python standard library.
The TokenPrune project is open-sourced on GitHub (GeekyMK/tokenprune) and published to PyPI.
TokenPrune's pruning logic consists of deterministic heuristic rules, so the same input always yields the same pruning result.
When the object structures in a list are inconsistent, TokenPrune's table compression may fail or produce ambiguity.
TokenPrune's default pruning prioritizes stripping metadata such as HATEOAS link fields, and if these links are indeed necessary for traceability or subsequent calls, developers need to explicitly preserve them.
1 more timeline events
All Facts (11)
TokenPrune has not yet been validated in large-scale production environments; the 70%–90% reduction figures come from the author's own tests, and actual results will vary depending on the type of API workload.
50%UnverifiedTokenPrune can be installed via pip install tokenprune.
50%UnverifiedThe core usage of TokenPrune is to wrap tool functions with the @prune_tool decorator, which automatically analyzes the structure of the returned dictionary, strips null values, compresses repetitive key-value patterns, and converts list-type data into a compact table form.
50%UnverifiedAccording to benchmark tests published by the author, TokenPrune achieves 70%–90% token reduction on enterprise-grade API payloads.
50%UnverifiedTokenPrune's runtime overhead is under 0.5 milliseconds, and the number of external API calls is 0.
50%UnverifiedTokenPrune is implemented based on AST tree-shaking using only the pure Python standard library.
50%UnverifiedTokenPrune is an open-source, zero-dependency Python package that reduces redundant tokens in tool output through deterministic pruning and table compression in local memory, without calling any external model.
50%UnverifiedTokenPrune's pruning logic consists of deterministic heuristic rules, so the same input always yields the same pruning result.
50%UnverifiedWhen the object structures in a list are inconsistent, TokenPrune's table compression may fail or produce ambiguity.
50%UnverifiedTokenPrune's default pruning prioritizes stripping metadata such as HATEOAS link fields, and if these links are indeed necessary for traceability or subsequent calls, developers need to explicitly preserve them.
50%UnverifiedThe TokenPrune project is open-sourced on GitHub (GeekyMK/tokenprune) and published to PyPI.
50%