[KongchangAI]
· 2 min read· 1,277 words

TextSeal2: A Distortion-Free LLM Watermark with Significantly Improved Detection

TextSeal2: A Distortion-Free LLM Watermark with Significantly Improved Detection

TextSeal2 achieves a distortion-free, speculative-decoding-resistant LLM watermark via virtual keys and exponential racing, significantly boosting detection.

TextSeal2 tackles the trade-off between detection capability and output quality in LLM watermarking with a distortion-free scheme. It improves on TextSeal's multi-key routing by merging keys into a virtual key coupled with fresh noise via a Gaussian copula, avoiding detection signal dilution. Under speculative decoding, it uses the memorylessness of the exponential distribution to preserve the watermark via a keyed exponential race on rejected drafts without changing acceptance rates or output distributions. Experiments show 8-22 percentage point higher detection than TextSeal for 100-token answers at equal diversity, and 18 and 22 points higher than TextSeal and textGrain for 200-token answers under speculative decoding, with no measurable latency or logit post-processing, though cross-model generalization and adversarial robustness remain unverified.

LLM watermarking has long faced a dilemma: embedding a detectable signal in text often comes at the cost of output quality and diversity. TextSeal2 attempts to break this trade-off—it claims to significantly improve detection capability while preserving distortion-free properties, and remains robust when combined with speculative decoding.

TextSeal2 paper source screenshot

From TextSeal to TextSeal2: Solving the Detection Signal Dilution Problem

TextSeal is the predecessor of TextSeal2. It routes each token to one of two keys via a coin flip, thereby restoring output diversity in the Gumbel-max sampling scheme. The problem with this design lies on the detection side: the detector needs to compute scores for both keys simultaneously and average them, yet has no way of knowing which key was actually used during generation, causing the detection signal to be diluted.

TextSeal2's approach is to merge rather than split. It fuses two or more keys into a single "virtual key" and couples it with fresh noise through AutoMark's Gaussian copula. Crucially, any number of keys—including publicly shared keys—can be merged this way without weakening detection power. This means watermark key management can be more flexible, while detection sensitivity does not decline as the number of keys increases.

Gumbel-max sampling is a technique for sampling from discrete distributions, commonly used in language model decoding. It works by adding independent and identically distributed Gumbel noise to the logit of each candidate token, then taking the token corresponding to the maximum value. The beauty of this trick is that taking the argmax after adding noise is equivalent to sampling from the original softmax distribution, and the randomness of the noise provides a natural vehicle for watermark embedding: by controlling how the noise is generated (e.g., using key-derived random numbers), a detectable signal can be embedded without changing the output distribution. The TextSeal series exploits exactly this, coupling keys with Gumbel noise to achieve distortion-free watermarking.

A Gaussian copula is a statistical tool that uses the Gaussian distribution to characterize the dependence structure among multiple random variables. Given the marginal distributions of each variable, a copula can model their joint distribution in Gaussian form, thereby flexibly controlling the correlations among variables without changing their respective marginals. In TextSeal2, the Gaussian copula is used to couple the random signal of the virtual key with fresh noise, so that merging multiple keys does not break the distortion-free nature of sampling while preserving the distinguishability of the detection signal.

Watermark Preservation Under Speculative Decoding: Memorylessness and Exponential Racing

Speculative decoding rapidly generates candidate tokens via a draft model and then verifies them in parallel with the target model; it is a mainstream means of accelerating LLM inference today. However, this process poses a special challenge for watermarking: rejected draft tokens need to be replaced—can the replacement tokens still preserve the watermark signal?

TextSeal2's answer is yes. The paper points out that under speculative decoding, replacement tokens for rejected drafts still maintain the watermark using the same key, and the acceptance rate remains unchanged. The mechanism is to run another keyed exponential race on the residual of the draft races. Thanks to the memorylessness of the exponential distribution, this race can reproduce the behavior of block verification, so that the output strictly follows the target model's distribution. In other words, watermark embedding does not change the model's generation results—this is the core guarantee of being "distortion-free."

The paper also emphasizes that TextSeal2 adds no measurable inference latency and preserves output diversity. These two points are critical for practical deployment—if a watermark introduces noticeable performance overhead or degradation in text quality, it is hard to deploy in production environments.

The memorylessness of the exponential distribution means: if a random variable T follows an exponential distribution, then for any s, t ≥ 0, P(T > s + t | T > s) = P(T > t). Intuitively, after having waited for time s, the distribution of the remaining waiting time is exactly the same as starting from zero. This property enables, in speculative decoding, that when a draft token is rejected, the race among the remaining candidate tokens can "restart" a keyed exponential race without introducing bias. Because the residual lifetime of an exponential distribution is distributed identically to the original, the new race can precisely reproduce the sampling result of block verification, thereby guaranteeing that the final output strictly follows the target model distribution—this is the mathematical foundation for TextSeal2 remaining distortion-free under speculative decoding.

Speculative decoding is a technique for accelerating large model inference: first, a small, fast draft model autoregressively generates several candidate tokens, and then the large target model verifies in parallel whether these tokens are acceptable. If a draft token is rejected, resampling occurs from that position and subsequent drafts are discarded. This method can significantly improve inference speed without changing the output distribution, but the watermark signal is easily lost or shifted during the reject-replace process, so specialized designs are needed to preserve the watermark. TextSeal2 solves this problem using exponential racing and memorylessness.

Experimental Performance: Detection Rate Far Ahead at Equal Diversity

The paper evaluates TextSeal2 on three LLMs. At a false positive rate of 10^-3, using the same output token diversity as the baseline, TextSeal2's detection rate for 100-token answers is 8 to 22 percentage points higher than TextSeal, and comparable to textGrain, but without the need for post-processing optimization of logits.

In the speculative decoding setting, the gap widens further: at the same acceptance rate, TextSeal2's detection rate for 200-token answers is 18 and 22 percentage points higher than TextSeal and textGrain, respectively. This result shows that TextSeal2's design has a clear advantage on the real deployment path of accelerated inference.

Note that the above data all come from the experimental setup self-reported in the paper; actual performance may vary depending on model, task, and data distribution. But for the watermarking field, achieving such detection improvements while maintaining distortion-free properties and zero latency overhead is a direction worth attention.

Technical Insights and Limitations

The core contribution of TextSeal2 lies in combining copula coupling with memoryless racing to solve the detection dilution problem of multi-key watermarking and to make watermarking compatible with speculative decoding. For LLM applications requiring content provenance, copyright protection, or abuse detection, this type of distortion-free watermark offers a viable path without sacrificing generation quality.

However, the paper does not yet disclose details such as cross-model generalization, robustness under adversarial attacks, and detection stability on long texts. The practical value of watermarking technology ultimately depends on its performance in real attack scenarios, and these aspects still require validation by future research.

One-line Summary

TextSeal2 achieves a distortion-free, speculative-decoding-resistant LLM watermark through virtual keys and exponential racing, with significantly improved detection rates.

Paragraph Summary

TextSeal2 addresses the trade-off between detection capability and output quality in LLM watermarking by proposing a distortion-free watermarking scheme. It improves upon the multi-key routing design of its predecessor TextSeal by merging multiple keys into a virtual key and coupling it with fresh noise via a Gaussian copula, avoiding dilution of the detection signal through averaging. In the speculative decoding setting, TextSeal2 exploits the memorylessness of the exponential distribution to preserve the watermark through a keyed exponential race when a draft is rejected, without changing the acceptance rate or output distribution. Experiments show that at equal output diversity, TextSeal2's detection rate for 100-token answers is 8 to 22 percentage points higher than TextSeal, and under speculative decoding its detection rate for 200-token answers is 18 and 22 percentage points higher than TextSeal and textGrain, respectively. The scheme adds no measurable latency and requires no logit post-processing, but cross-model generalization and adversarial robustness remain to be verified.

#LLM watermarking#TextSeal2#distortion-free watermark#speculative decoding#Gaussian copula#exponential racing
Share:

Related articles