A Watermark in the Word Choice

A Signature No One Sees

Since May 2024, text written in the Gemini app and on the web has carried a watermark. It is not in the letters and there are no hidden symbols: it reads as an ordinary answer. The watermark is in the word choice, in which of several words that all make sense gets picked.

The method is called SynthID-⁠Text; it was published in Nature in October 2024 and released openly the same day. It answers one question: was this text written by a model that watermarked it with this key? Whoever holds the key works out a score; above a threshold, the text counts as watermarked.

On the desk is this page’s metaphor machine: on the left a type case holding the candidate words by the model’s odds; in the middle a knockout board scoring round by round; on its right cams, set by the key and the last four words, that decide the scores; the winning word runs down a chute onto the composing rail in front; on the far right a gauge reads it back with the same key. No real model has these parts.

SynthID-⁠Text · the Gemini app and web, since May 2024 · Nature, October 2024

Every Word Has Odds

A language model writes one token at a time (a word or a piece of one). At each step it gives every possible token a probability, then draws one by those odds. The paper’s example: after “… my favourite tropical fruit is”, mango has a 50% chance of coming next, lychee 30%, papaya 15%, durian 5%.

It draws rather than always taking the likeliest, so the same question gets different answers. The paper’s runs draw from the 100 likeliest tokens (top-⁠k = 100) at temperatures of 0.5, 0.7 or 1.0. For where tokens and their odds come from, see “Tokens and Scaling Laws”.

In the type case on the left of the desk, each tier holds one candidate, with tiles by its odds: 5 for mango, 3 for lychee, 2 for papaya, 1 for durian. The watermark leaves these odds alone; it changes how a word is drawn from them.

… my favourite tropical fruit is → mango 50% · lychee 30% · papaya 15% · durian 5% (the paper’s Fig. 2)

A Knockout for Each Word

With the watermark on, the model draws more than once. It first draws 2ᵐ candidates by its own odds, repeats allowed; they meet in pairs, the higher score in each pair goes on, and a tie is settled by a coin. The first round is scored by g₁, the second by g₂, for m rounds; the one left is the word written.

The paper’s Fig. 2 has m = 3: eight candidates, durian, mango, lychee, mango, papaya, lychee, mango, mango, and after three rounds mango wins. The knockout board on the desk follows that example: the flip dot above each tile shows its score for the round, red for 1, ivory for 0, and the losing tiles fall into the tray in front.

The paper’s runs use m = 30 rounds. Played match by match, that would take 2³⁰ ≈ 1.07 billion candidates a word; instead the paper works out the chance each word has of winning and draws once from that, which gives the same distribution.

2ᵐ candidates · pairs meet, the higher score wins, a coin settles ties · on the desk m = 3: 8 → 4 → 2 → 1 · the paper’s runs: m = 30

The Key and the Last Four Words

The scores are not arbitrary. At each step the last four tokens (H = 4) are hashed together with a secret key into a random seed; g₁, g₂, g₃ and the rest are worked out from the seed and the candidate, each 0 or 1, half and half.

So the same key and the same four words always give a candidate the same scores; another key, or other words before it, and the scores are as if the coins were tossed again. Without the key there is no telling which words scored 1. In Fig. 2, mango scores 1, 0, 1 over the three rounds; lychee 0, 1, 0; papaya 0, 0, 1; durian 1, 0, 0.

When the same four words come up again in an answer, that step is left unwatermarked (repeated-⁠context masking), so one bias does not repeat until the text goes round in loops. On the desk, the key turns the cams; a brass frame on the rail over the last four words is wired to the cam column.

Seed = hash(key, last 4 tokens) · g-⁠values: 0 or 1, half and half · repeated contexts left unwatermarked

The Odds Stay the Model’s

With two candidates a match there is a useful property: averaged over every possible key, the chance of writing each word is exactly the model’s own. What a word gains in a round when it scores 1 it loses when it scores 0, and on average the two cancel.

For one fixed key and one fixed context the odds do tilt. At the step of Fig. 2, by the paper’s formula, mango goes from 50% to about 82% and lychee from 30% to about 8%. The tilt points another way at every step and still looks random; only the key shows which way.

The paper proves that with two candidates a match each word’s distribution is unchanged on average, and with repeated-⁠context masking a whole answer’s too. More than two a match makes a stronger watermark but changes the distribution; so did the earlier “green list” watermark (2023), which raised the odds of part of the vocabulary. The cost: ask the same question many times and the answers differ a little less from each other.

Over all keys: 50% · 30% · 15% · 5% · this key, this context: about 82% · 8% · 8% · 2%

Reading It Back

The check needs no model: only the text, the key and the way the seeds are made. Work out every token’s g-⁠value in every round and average them: ordinary text averages 0.5; watermarked text averages higher, because words that scored 1 were likelier to win.

The longer the text, the more evidence. The threshold is set so that ordinary text passes only 1% of the time, and it falls with length. In the paper’s test, with the chat version of Gemma 7B at temperature 0.7, human-⁠written answers as the comparison and false alarms held at 1%, it caught 69.64% of texts at 200 tokens and 86.64% at 400; at temperature 1.0, 87.39% and 97.52%. That test uses a Bayesian score learned from data, stronger than a plain average.

On the gauge to the right, the black needle is the mean score and the red one the threshold. The reader runs along the rail word by word; the black needle follows, the red one falls with length. It reads a line written by this page’s toy model (3 rounds a word), not Gemini.

Ordinary text: 0.5 on average · threshold: 1% false alarms · Gemma 7B, temperature 0.7: 69.64% at 200 tokens, 86.64% at 400

Twenty Million Answers

Does the watermark make answers worse? The researchers ran a live experiment in Gemini: a random share of requests went to a watermarked model and as many to one without, and they looked at users’ thumbs up and down. About 20 million answers were analysed.

The thumbs-⁠up rates differed by 0.01% (the watermarked one higher), the thumbs-⁠down rates by 0.02% (the watermarked one lower); neither difference was significant. In another test, raters compared Gemma 7B’s two kinds of answer to 3,⁠000 questions side by side and found no significant difference in grammar and coherence, relevance, correctness, helpfulness or overall quality.

Nor is it much slower: on four TPU v5e chips, Gemma 7B takes 15.527 ms a token, and 15.615 ms with a 30-round knockout, 0.57% more. It also works with speculative decoding, at about the same acceptance rate, though the watermark comes out weaker; for that trick, see “Speculative Decoding”.

About 20 million answers · thumbs up: 0.01% apart · thumbs down: 0.02% apart · neither significant · 3,⁠000 questions rated side by side: no significant difference · latency +0.57%

When It Cannot Tell

When there is one right answer, there is no word to pick. Ask “What is the capital of France?”, or for a well-⁠known poem, and nearly every step has one choice; the candidates in the knockout are all the same word and no tilt can come through. On the desk the text is now a times table: the case holds almost a single kind of tile, and the gauge’s black needle stays near 0.5.

Short text holds too little evidence. Rewriting weakens it too: with a few words cut or changed, or a light paraphrase, it can still be found; after a thorough rewrite or a translation, the detector’s confidence falls a long way. In the paper, after Gemini Ultra paraphrased texts sentence by sentence, far longer texts were needed to find it.

It recognises only text from a model that watermarked it with this key. Finding no watermark does not mean a person wrote it: text from other models, or with the watermark off, reads as ordinary too. Finding one only means a score above a threshold, with some rate of false alarms. The key must stay secret or others can copy the watermark, and it was not built to stop determined attackers. Open models run by others cannot easily be made to watermark.

One right answer: no word to pick · short text · thorough rewrites, translation · this key only · no watermark does not mean a person wrote it

Taken Apart

On the desk: on the left a stepped walnut type case, a row of ivory tiles on each tier by its odds, a brass slot at its right end that hands out tiles; in the middle the knockout board, with brass ledges, runs, flip dots and gates; on its right the cam column, three notched brass cams and a key; in front the composing rail, with its reader and the frame over the last four words; on the far right the gauge; before the board a felt-⁠lined tray for the tiles that lose. The machine is a metaphor: a real model has no tiles and no knockout you could watch.

Type case · knockout board · cams and key · composing rail · gauge · tray