TreeSpark
An adaptive draft-tree construction method for speculative decoding that optimizes tree-structure expansion for semi-autoregressive drafters through path survival rate calibration and load-adaptive serving strategies, improving inference speed while preserving lossless decoding.
Timeline (last 90 days)
TreeSpark is a method optimized for draft trees, proposing a calibrated, load-adaptive draft tree construction approach designed specifically for semi-autoregressive drafters.
TreeSpark reads the parent-conditioned distribution from the drafter's existing Markov head and calibrates it into edge acceptance estimates, used to assess the likelihood of each edge being accepted by the target model.
TreeSpark lets path survival probability govern three decisions: best-first expansion, per-round stopping, and load-adaptive serving strategy.
TreeSpark samples sibling nodes without replacement and matches residuals in recursive rejection sampling, thereby keeping decoding results consistent with the target model distribution at any temperature.
Under a matched fixed budget, TreeSpark's adaptive tree outperforms fixed trees at every temperature.
Relative to a tuned chain-based scheme on the same drafter, TreeSpark accepts 15–25% more draft tokens per round.
In terms of single-request wall-clock time, TreeSpark improves decoding speed by 8–14%.
As load increases, TreeSpark can shrink the tree back to a chain structure to avoid resource waste under high load.
The speedup numbers reported by TreeSpark are measured under specific drafters and comparison baselines; actual gains will vary with model size, task type, and hardware configuration.
TreeSpark has open-sourced its code and experimental artifacts on GitHub (PopSoda2002/TreeSpark).
All Facts (10)
TreeSpark is a method optimized for draft trees, proposing a calibrated, load-adaptive draft tree construction approach designed specifically for semi-autoregressive drafters.
50%UnverifiedTreeSpark reads the parent-conditioned distribution from the drafter's existing Markov head and calibrates it into edge acceptance estimates, used to assess the likelihood of each edge being accepted by the target model.
50%UnverifiedTreeSpark lets path survival probability govern three decisions: best-first expansion, per-round stopping, and load-adaptive serving strategy.
50%UnverifiedTreeSpark samples sibling nodes without replacement and matches residuals in recursive rejection sampling, thereby keeping decoding results consistent with the target model distribution at any temperature.
50%UnverifiedUnder a matched fixed budget, TreeSpark's adaptive tree outperforms fixed trees at every temperature.
50%UnverifiedRelative to a tuned chain-based scheme on the same drafter, TreeSpark accepts 15–25% more draft tokens per round.
50%UnverifiedIn terms of single-request wall-clock time, TreeSpark improves decoding speed by 8–14%.
50%UnverifiedAs load increases, TreeSpark can shrink the tree back to a chain structure to avoid resource waste under high load.
50%UnverifiedThe speedup numbers reported by TreeSpark are measured under specific drafters and comparison baselines; actual gains will vary with model size, task type, and hardware configuration.
50%UnverifiedTreeSpark has open-sourced its code and experimental artifacts on GitHub (PopSoda2002/TreeSpark).
50%