A Title Promised a Share the Data Never Measured: A Content Review Retrospective

A Title Promised a Share the Data Never Measured: A Content Review Retrospective

An author who regularly publishes LLM telemetry notes received a batch of new title suggestions during a reader improvement initiative. One of them read more smoothly, yet the author rejected it — because a single word in the title had no supporting data anywhere in the article body. The episode ultimately gave rise to a content contract rule.

A Review Across Multiple Surfaces

The improvement initiative proposed new titles and summaries for three published notes. Before adopting any of them, the author compared every surface on which each note was presented to readers: the HTML title, the h1, the Open Graph title, the meta description, the summary on list cards, the summary box at the top of the page, and the body text.

For each surface, the question was the same: does it promise something the body does not deliver?

Of the three, one note's surfaces already matched the body; the other two became the occasion for this retrospective.

Open Graph (OG) is a metadata protocol proposed by Facebook and later widely adopted, using tags like og:title, og:description, and og:image inserted into a page's <head> to control how links appear when shared on social platforms, messaging apps, and aggregators. For content authors, the key point is that the OG title is an independent field from the HTML <title> and the page h1, and each can differ. This means the same article may have four or five different "titles" simultaneously across browser tabs, search results, social share cards, and in-site listings. The wording on any one surface is not automatically constrained by the others, so whether a promise is kept must be verified on each surface individually rather than checking only the title at the top of the body.

The meta description is often misunderstood as "a summary that search engines will display verbatim." In reality it is only a suggestion to the crawler; the search engine decides whether to use it based on the query and context, or extracts a snippet from the body to generate a summary itself. But in in-site list cards, RSS readers, and some social previews, it is often reused directly, so its wording still needs to stay consistent with the body's promises. It is precisely this uncertainty — "may be displayed verbatim, may be replaced" — that makes moving semantic review forward to the moment of writing more reliable than depending on how any single surface happens to render afterward.

"Most" Is a Measurement Result

The first rejected title targeted the note about telemetry. That article discussed a weekly report whose health checks continued to pass after the author's unattended work migrated to a tool whose data ingestion never reads them.

The body was especially careful in one place: it said the routing change moved a portion of the work entirely out of the table, and that "no column in the table could represent how large that portion was."

The candidate title "It No Longer Saw Most of the Work" filled in exactly that column. It asserted a majority — a share exceeding half of some total — yet the article had never measured that share. This was not a stylistic edit but the addition of a quantitative claim onto the surface of a text whose body explicitly declined to make that claim.

As a detail, the improvement initiative itself was not careless. Its instructions stated that "most" should only be used after verifying the time period and denominator in the original observation. The review did so, and the denominator did not exist.

One tempting move is to reconstruct a denominator from numbers actually printed in the article: the body had weekly per-tool transcript file counts, one column of which dwindled to nearly zero. But the article explicitly stated those were files, not sessions, and the two columns were not comparable as measures of work. One column dropping only shows that work left somewhere; it does not show what proportion of all work it represented. Treating either column as a denominator would put onto the title a number the body specifically warned against.

The denominator is the precondition for a quantitative claim, yet it is often omitted in transmission. Saying "most of the work has migrated" implies a ratio: migrated amount divided by total work before migration. The problem is that "total work" must first be defined as an additive quantity — sessions, requests, tokens, or hours? If the two values in a statistic come from different units or different sets, they cannot be divided to yield a meaningful ratio.

This connects to the requirement of "comparability" in statistics: a comparison holds only when two sets of data share the same measurement scale and the same denominator population. File counts and session counts in the article are incomparable precisely because they measure different entities; one column dropping only indicates that work flowed out of that column's coverage, not what share it held in the whole. Therefore, any multiplier, percentage, or "most/all/never"-style quantifier needs a sentence in the body that explicitly gives the same denominator and the same value as the title; without that sentence, the claim is merely rhetoric.

A Title That Mistook the First Step for the Answer

The second rejection was subtler. For the article "Sub-Agent Metrics and Main-Thread Metrics Are Not Comparable," the plan offered "Keep Main and Sub-Agent Metrics Separate."

It sounds like a faithful summary, but it omits what comes after "separate." The body showed that once each model's combination of roles is taken into account, the model's aggregate ranking may reverse; the article then said directly that stratification is only a first step, not the answer — metrics may mean different things in each stratum, and role differences are not a constant that can be removed by adjustment.

A title telling readers to "keep metrics separate" promises that "separating" solves a problem the body says it has only just begun to touch.

That article contained striking numbers, including role differences as high as 135x and main-thread shares across models ranging from 1.5% to 71.4%. They could yield a more eye-catching title. But none of them support "separating them" being the answer, nor do they say how much work moved where.

Simpson's paradox refers to a phenomenon in which, when data is split by some grouping variable, each group shows a certain trend while the combined whole shows the opposite trend, or the direction of ranking reverses. Its cause is that the grouping variable itself is correlated with the outcome, so the pooled statistic mixes between-group composition differences into the comparison.

The "role difference" mentioned in the article is precisely such a grouping variable: different models take on different proportions of main-thread and sub-agent work, and if rankings are aggregated directly, the influence of role composition may be misread as a difference in model capability. The reason stratification is only a first step rather than the answer is that even after stratifying, the meaning of the metric within each stratum may still differ — the same metric in a main-thread versus sub-agent context does not necessarily measure the same thing — and role proportions are not a constant that can be eliminated by normalization. So when discussing such metrics, a title claiming that "separating the metrics" solves the problem skips the consistency-of-definition question that still needs answering after stratification.

The Newly Added Rule

The pipeline that turns drafts into published notes has a written content contract, followed by both the writing step and the verification step. The author added a rule to it.

Any quantifier on a title, description, Open Graph title, or cross-platform title (words like most, all, never; multipliers and percentages) requires a sentence in the body that supports the same value with the same denominator. Promises of fixes, time savings, or ratios require the body to contain the actual fix process or measurement. The verifier must cite the supporting body sentence, or state that the claim is unsupported.

This is a semantic review of the body by a writer or verifier, not a machine check that parses numbers, and the author does not claim it catches every overreach. It moves the problem forward to the moment a surface is written and forces the answer to appear in the form of a cited sentence rather than a feeling.

One adjacent detail matters for the same reason: the description in the draft front matter is copied verbatim onto list cards, with no truncation step. So the description must be a complete sentence that keeps the same promise as the title, because that is exactly what readers see on the card.

What Changed, What Didn't

No existing title, card, or published article was changed. This rule applies only to notes written from now on.

The author has not measured whether stronger or weaker titles change click-through or readership, and this article does not claim they do. If that were to be measured later, it would not be by changing the title, summary, and social card on the same day and then attributing whatever change occurred to one of them — changing multiple surfaces at once cannot isolate the effect of a single element.

This note's own title says that a candidate title promised a share the data never measured. What carries it is the cited sentence from the telemetry note: its table had no column to represent the size of the migrated share, yet the candidate title asserted that share was a majority.

The value of this case is not whether a title was written well or poorly, but a more general content principle: when the body explicitly declines to make a quantitative claim, the title level should not quietly add it back.

One-line Summary

A candidate title claimed a share the body never measured, prompting a content rule that quantifiers require a supporting body sentence.

Paragraph Summary

The article recounts how the author rejected two title suggestions during a reader improvement initiative. The first proposed "It No Longer Saw Most of the Work" for the telemetry note, but the body explicitly stated no column could represent the migrated work's share, so the title added an unmeasured quantitative claim. The second, "Keep Main and Sub-Agent Metrics Separate," seemed faithful but treated the body's "stratification is only a first step" as the answer, ignoring that role differences cannot be eliminated by tuning. The review covered multiple surfaces: HTML title, h1, OG title, meta description, card summary, page-top summary, and body. The author therefore added a rule to the content contract: any quantifier or multiplier in a title or description must have a body sentence supporting the same value with the same denominator, and the verifier must cite that sentence or flag it as unsupported. The rule governs only new notes; existing content is unchanged, and the author makes no causal claim about click-through.