Can Raising the Temperature Wash Away the SynthID Watermark? A 100-Text Test

Title

Can Raising the Temperature Wash Away the SynthID Watermark? A 100-Text Test

Meta Description

A 100-text test of whether raising rewriter temperature can wash away the SynthID watermark faster: temperature 1.3 with default top-p removes about half the watermark signal for free, but once you break past nucleus sampling the text collapses first, and a quarter of rewrites lose facts.

Article Content

Can raising the temperature of a rewriting model wash away the SynthID watermark faster? In a previous article, a developer rewrote 100 watermarked texts with three local models and found that the detector could still catch traces in most of them. A reader followed up: would a "hotter" rewriter break the 5-token window that the watermark relies on more quickly? So he ran a full sweep of sampling parameters, and the answer turned out to be more layered than expected.

Why the Sampling Knobs Should Matter

The SynthID watermark is embedded in an exact token sequence: it fine-tunes each token based on the previous 4 tokens, so rewritten text only carries the watermark when it reproduces the original's 5-token window. Two parameters determine how much the rewriter reuses the original wording.

Temperature sets how strongly the rewriter prefers high-probability tokens. Low temperature produces the most natural wording, and the most natural wording is often exactly what the original model wrote. Top-p (nucleus sampling) determines which tokens can be sampled. At top-p 0.9, no matter how high the temperature, the rewriter only picks from a plausible set; at top-p 1.0 the entire long tail opens up, including tokens the model considers unlikely.

The previous article found that the survival rate of original 5-grams after rewriting had a correlation of 0.77 with the watermark z-score. So the prediction is straightforward: anything that reduces 5-gram overlap should reduce signal strength. The question is what it costs.

SynthID is a text watermarking scheme from Google DeepMind. Its core mechanism is that when a large language model generates each token, based on the hash of the preceding several tokens (usually 4), it applies a reproducible pseudo-random bias to the probability distribution of candidate tokens, so that the selected token is statistically skewed in a hidden way that only someone with the key can verify. Therefore, the watermark is not written into the plaintext but hidden in a continuous pattern of token transitions—only when the rewritten text reproduces the same 5-token window as the original does the detector accumulate this bias. This explains why the 5-gram overlap rate can serve as a proxy for watermark strength: the less overlap, the fewer windows the detector has to accumulate evidence, and the z-score drops accordingly.

The z-score is the output statistic of the SynthID detector, measuring how far the bias of watermarked tokens in the tested text deviates from unbiased random text. The detector calculates at each position whether the actual token falls in the watermark-biased high-scoring region, then aggregates the hits across the sequence into a standard score. The higher the z, the less likely it is by chance; in the experiment, the "flagged rate" is the proportion of texts whose z exceeds a false-positive threshold (such as the cutoff corresponding to 1% false positive). AUC measures the detector's overall ability to distinguish watermarked text from clean text: 0.5 means no discrimination at all, and 1.0 means perfect discrimination. Together these two metrics characterize watermark detectability without relying on a binary judgment for any single text.

Experimental Setup

The rewriter chosen was gemma3:12b—the model with the strongest watermark-washing effect in the previous article. Same prompt, same seed, same context length, changing only the sampling options.

Five settings: temperature 0 (greedy decoding), 0.7 (original baseline), 1.0, 1.3, and temperature 1.3 with top-p 1.0. The texts were the same batch of 100 watermarked originals (190 to 900 words, covering 10 genres), and each setting also rewrote 100 clean originals as a control. The detector and cutoff were not recalibrated.

For meaning evaluation, 500 original-rewrite pairs were shuffled into 25 batches of 20 pairs each, and a Claude judge scored meaning (1-5) and fluency (1-5) without knowing the setting.

Results: The Knob Works, but the Cliff Is at Top-p

Raising the temperature does reduce the watermark signal, and it is almost free as long as you do not touch nucleus sampling.

Temperature 1.3 + default top-p 0.9: the free half. The flagged rate dropped from 25% to 14%, with a paired z change of -0.48 and a confidence interval far from zero. The judge could not tell the difference: meaning 4.11 vs. 4.19, fluency 4.93 vs. 4.95, both within noise. This means an attacker can wash away about half the watermark signal without harming text quality.

Greedy decoding actually preserves more watermark. At temperature 0, the rewriter retained 12.7% of the original 5-grams (baseline 10.4%), and the flagged rate rose from 25% to 31%. This again confirms the "second-pass effect": the highest-probability wording is the original wording, and the watermark comes along with it.

Open the long tail, and the text collapses before the watermark does. At the same temperature, raising top-p to 1.0 reduced the flagged rate only from 14% to 11%, statistically indistinguishable. But it clearly damaged rewrite quality: average meaning score dropped by 0.37 and fluency by 0.63, and a quarter of texts scored 3 or below on meaning (baseline 7%). Specific cases included: a scientific explainer lost the inverse fourth-power law and invented "two to ten times"; a story got the mother's date of death and dialogue attribution wrong; an accident report changed "voltage dropped below threshold" to "voltage reading was below the allowed level," completely reversing the meaning.

The null hypothesis on clean text did not move. Across the five settings, the average z for clean rewrites was 0.07, -0.01, -0.04, 0.08, and 0.20, and the 5-gram loss rate was the same as for watermarked text. The signal decline is real signal loss, not a shift in how the detector perceives clean text.

No setting drives it to zero. Even the hottest setting still flagged 11% of rewrites at the 1% false-positive cutoff, and AUC against the rewrite null hypothesis was still 0.67. The conclusion from the previous article still holds: too weak to accuse any single short text, but significant enough once accumulated across a whole portfolio.

Nucleus sampling (top-p) and temperature are two orthogonal sampling controls. Temperature divides the logits by a coefficient before softmax, changing the steepness of the entire distribution; top-p instead accumulates probabilities from high to low, truncates once the cumulative probability reaches p, removes low-probability tokens outside the long tail from the candidate set, then renormalizes and samples within the remaining set. So even if the temperature is very high and the distribution has been flattened, as long as top-p stays at 0.9, those extremely unlikely words still cannot be selected. Conversely, when top-p is set to 1.0, the long tail is fully open, and at high temperature the rewriter may pick semantically unreasonable but extremely low-probability words—this is the source of the factual distortions and fluency drops in the experiment.

Greedy decoding means directly choosing the highest-probability token at each step without any random sampling, so given the model, prompt, and context the output is completely deterministic, and the same input always yields the same output. This is also why the article says "temperature 0 is deterministic by construction." Its difference from low-temperature sampling is that temperature 0.1 still randomly draws from a distribution near the top, whereas greedy decoding has none of that randomness. It is precisely this "always choose the most natural wording" property that makes greedy rewriting more likely to reproduce the original wording, thereby preserving more 5-gram windows and watermark signal, and in the experiment the flagged rate actually rose to 31%.

What These Findings Mean

For people trying to hide AI use: The rewriter's temperature is a real and effective lever, and it is free until you touch nucleus sampling. A hot rewrite with default top-p can eliminate about half the traces left by the default setting, with no measurable loss in text quality. Beyond nucleus sampling there is almost no further gain, and the text starts losing facts. Deterministic rewriting is the worst choice.

For people trying to detect AI use: Attackers have a knob you cannot see, and its first half costs them nothing. The watermark traces that survive under the best attacker setting are about one in ten (at a 1% false-positive rate). Your aggregate statistics still work; single-document judgments become weaker.

For the 5-gram proxy metric: It holds up again. Every setting that reduced overlap reduced the signal in order. If you cannot run the detector, counting surviving 5-grams still tells most of the story.

One other number worth noting: across all 500 scored rewrites, the correlation between 5-gram retention and meaning score was only 0.15. Preserving the original wording buys almost no faithfulness. Faithfulness comes from the model's understanding of the text, and copying is just a habit that happens to carry the watermark.

Limitations and Caveats

The experiment used only one rewriter, one prompt, and one seed per setting. Temperature 0 is deterministic by construction. Different rewriters may have different nucleus sampling breaking points. There was only one judge per item, with no second rater or agreement check; the 1-5 scale is coarse, so pay attention to paired intervals rather than raw means. The baseline is not flawless either: the judge found that the 0.7 run changed the tea price "one crown per pound" to "one pound per crown," and two other settings turned a 17-year-old restaurant into a 17-year-old owner.

With 100 texts per setting, the confidence intervals for flagged rates are wide. The AUC column and paired z differences are stronger evidence. Top-k and the generator's own temperature were not changed. The Ollama model is 4-bit quantized.

The whole pipeline added four short scripts on top of the first article's code: one to run each setting's rewrites through the Ollama HTTP API, one to score with the same detector and count surviving 5-grams, one to do paired analysis, and one to build blind evaluation batches and collect scores. About three hours of GPU time, almost all of it for rewriting.

The reader who asked the question was right to ask it. The answer is not the one I would have guessed: the knob does work, but the cliff is at top-p, not temperature.

One-line Summary

Raising the rewriter temperature can wash away about half the SynthID watermark for free, but the gains stop at the nucleus sampling boundary, and beyond top-p the text collapses first.

Paragraph Summary

This article tests whether raising the rewriter temperature can wash away the SynthID watermark faster through a sampling parameter sweep. The author rewrote 100 watermarked texts with gemma3:12b under five sampling settings and evaluated watermark residue and text quality using the original detector and a Claude judge. The results show that temperature 1.3 with default top-p 0.9 reduced the flagged rate from 25% to 14% with no statistically significant loss in meaning or fluency; greedy decoding actually preserved more watermark, raising the flagged rate to 31%; and setting top-p to 1.0 reduced the watermark by only 3 more percentage points while causing a quarter of texts to lose or distort facts. The null hypothesis on clean text remained stable across all settings, indicating that the signal decline is real loss rather than detector drift. The correlation between 5-gram retention and meaning score was only 0.15, showing that copying the original wording buys almost no faithfulness. The conclusion: temperature is an effective lever, but the cliff is at top-p rather than temperature, and no setting can drive the watermark to zero.