Can Human Contribution in AI Writing Be Measured? Likelihood Scoring Under Scrutiny

When AI writing becomes the norm, what question should we really be asking?

The core question surrounding AI text detection has long been "was this article generated by a machine?" But as AI becomes deeply embedded in everyday writing workflows, this binary judgment is losing its meaning. A more practical question is emerging: how much did humans actually contribute to a given piece of text?

According to a recent study, some mainstream approaches in academia attempt to estimate the proportion of human contribution by measuring "how well user prompts predict the final text." It sounds reasonable — the more precisely a prompt points to the output, the more critical the human input seems to be. But this study demonstrates through experiments that this logic has a fundamental flaw.

A simple trick can make AI text "look more human-written"

The research team proposed a manipulation technique called SPECIFICITY PADDING. The operation is extremely simple: just have the model copy a few random ID strings from the prompt, and the human contribution score changes significantly.

How dramatic is the effect? A request that provides only a title, after this treatment, actually scores as "more human-written" than a request that provides the article's core arguments. This means the scoring system is not measuring the depth of human intellectual investment, but rather certain string features in the text that can be "copied."

Context-swap audit: the same article can score half as much

To verify that this finding was not coincidental, researchers designed a context-swap audit. The method is straightforward: keep the document content completely unchanged, and only show or hide the random ID strings in the prompt to the evaluator.

The results were shocking — the same unmodified text fluctuated by one-third to one-half on the human contribution scale. Across all test documents, scores increased. The test samples covered news reports and scientific abstracts, and the models invoked ranged from 3.8 billion to 753 billion parameters.

More intuitively: the same batch of articles, with only the prompt records changed, could receive an average human contribution score of 85% or 15%. Even without providing any prompt at all, scoring methods based solely on output text variants would still reward the random strings themselves.

The credit ledger: why scoring mechanisms are naturally biased toward "rare strings"

The study provides a precise explanatory model called the credit ledger.

The core mechanism is this: such scoring is essentially a surprisal-weighted average of the text. That is, rare strings that become copyable due to their context automatically collect higher credit scores, regardless of their actual meaning or whether they carry human thought.

This explains why copying random IDs is "more effective" than providing core arguments — the former is statistically rarer, and therefore assigned higher weight by the system. The scoring system rewards statistical features, not cognitive contributions.

A deeper theoretical boundary

The study also proposes a two-point bound, revealing a more fundamental limitation:

If the prompt itself may be machine-generated, then all methods relying solely on text records have, in the worst case, an error of at least half the contribution involved.

This conclusion points to a key judgment — likelihood scoring alone cannot reliably confirm human contribution. To truly measure human investment in AI-assisted writing, two things must be understood simultaneously: what content the human specifically added, and how the text was produced.

Without this production-process evidence, any scoring based on prompt-output predictive relationships may be measuring statistical artifacts rather than human intelligence.

Likelihood in machine learning refers to the probability a model assigns to observed data, typically used to measure how much a piece of text "looks like" samples from the training distribution under a given model. In AI text detection or human contribution assessment, a common approach uses "the likelihood of generating the text given the prompt" as a proxy metric: lower likelihood is considered "more human-written." However, likelihood is fundamentally a statistical quantity — it only reflects relative rarity under a probability distribution and cannot distinguish "unique expressions written after deep human deliberation" from "meaningless strings randomly copied by a machine." Therefore, when the paper states that "likelihood does not equal authorship," it emphasizes precisely the gap between statistical correlation and causal/cognitive attribution. Establishing genuine authorship judgment requires process evidence, not reliance solely on the likelihood score of the final text.

Implications for AI writing tools and evaluation systems

The impact of this research extends beyond academic discussion. For teams building AI writing assistance products and platforms attempting to assess content originality through automated means, the following questions deserve reconsideration:

• Can scoring metrics based on prompt predictive power serve as proxy variables for human contribution?
• When scores can be easily manipulated by a few random strings, how much credibility do such metrics retain in products?
• Does assessing human contribution require process evidence (such as edit history, version iteration records) rather than relying solely on static comparison between final text and prompts?

The study does not offer an alternative, but it clearly draws a boundary: likelihood does not equal authorship. In the context of increasingly common AI-human collaborative writing, how to more honestly measure human contribution remains a far-from-resolved open question.

Background supplement

Surprisal is a foundational concept in information theory, referring to the "unexpectedness" of a word or string appearing in a given context, commonly measured by its negative log probability: the lower the probability, the higher the surprisal. In natural language processing, language models output conditional probabilities for each token, allowing word-by-word computation of the surprisal distribution across an entire text. When a piece of text is highly constrained by the prompt, the model's prediction probability for that text increases, and surprisal decreases accordingly; conversely, strings like random IDs with almost no contextual cues have extremely low generation probability and high surprisal. The so-called "surprisal-weighted average" uses these per-word surprisal values as weights to compute a weighted sum of the "contributions" of each part of the text. Because rare strings carry extremely high weights, they disproportionately inflate the overall text score, even if these strings carry no human thought whatsoever. This mechanism is the mathematical root of why the "credit ledger" can be easily manipulated by a few random IDs.

One-line Summary

AI text human contribution scores can be easily manipulated by random strings — likelihood does not equal authorship.

Paragraph Summary

In AI text detection, academia commonly uses "how well prompts predict the final text" to estimate human contribution ratios, but a recent study proves this logic has a fundamental flaw. Researchers propose "specificity padding": simply having the model copy a few random ID strings can significantly alter human contribution scores, even making a title-only request "more human-written" than one providing core arguments. Through a "context-swap audit," the same unmodified text's score can fluctuate between 85% and 15%. The root cause is that the scoring mechanism is a "surprisal-weighted average" of text — the system rewards statistical rarity rather than cognitive contribution. The study further proposes a "two-point bound" theorem: if prompts may be machine-generated, the worst-case error of pure-record methods is at least half the contribution. The conclusion is that likelihood scoring alone cannot reliably confirm human contribution, and production-process evidence must be introduced.