YAML vs JSON: Real Token Benchmarks for LLM Workloads

In large model applications, the choice of data serialization format is often brushed off as a matter of style, but it directly determines how many tokens you pay for on every request and how much content you can fit into a context window. The DevToolHub author ran controlled tests on JSON and YAML using a real tokenizer, rather than repeating the vague "saves about 20%" claims floating around online. The results are further apart than most people expect, and they also expose a parsing bug that can silently pollute LLM output.

Benchmarks: How Many Tokens Does YAML Actually Save?

The author built two test cases—one nested config object and one tidy table of 10 rows with 4 fields each—and encoded both with OpenAI's cl100k_base tokenizer. Note that switching to Claude's or Gemini's own tokenizers will shift the absolute numbers slightly, but the relative ranking between the two formats stays the same.

In the nested config object case (a single object with embedded sub-objects, not a list), JSON encoded to 99 tokens and 322 characters, while YAML came in at 66 tokens and 223 characters. In other words, simply dropping braces, commas, and quotes around keys lets YAML cut the token count of the same config by 33.3%.

On the tidy 10-row, 4-field table data, the picture changed. JSON was 323 tokens and 963 characters; YAML was 221 tokens and 590 characters—still ahead by 31.6%. But encoding the same data in TOON, a format designed specifically for compression, produced just 97 tokens and 235 characters—70% fewer than JSON and 56.1% fewer than YAML.

The gap comes from structure, not surface decoration. YAML's - id: 1\n name: Alice style repeats every field name on every row; TOON declares the field list once at the top (users[10]{id,name,role,active}:) and then streams bare values. For a one-off config file, that repetition barely matters. But for tool return values that enter an agent's context window and get resent every turn, the cost of repetition compounds fast.

cl100k_base is the BPE (Byte Pair Encoding) tokenizer OpenAI provides for the GPT-3.5/GPT-4 series, and one of the default encodings in the tiktoken library. Its vocabulary is roughly 100K tokens, trained mostly on English and code, so it has particular splitting habits for JSON-like text full of punctuation: curly braces, square brackets, commas, and quotes often each occupy their own token, and even the ": " combination after a key gets split into multiple tokens. This means JSON's "symbol overhead" is significantly amplified at the token level, while YAML sidesteps these high-frequency, low-information symbols by using indentation instead of brackets and omitting redundant quotes. Different vendors' tokenizers merge symbols differently, so the absolute savings will vary, but the underlying logic—more symbols, more expensive tokens—is universal.

Where Each of the Three Formats Fits

Which format to pick depends fundamentally on the shape of the data, not personal preference.

Nested, irregular data (application config, a single object with sub-objects): YAML and TOON are nearly identical in size. TOON's spec itself borrows YAML's indentation syntax for non-tabular structures, so switching formats here offers no real benefit. Go with YAML—its tooling ecosystem is larger.

Regular arrays of objects (user lists, log lines, query results, tool outputs): this is TOON's home turf. It's exactly the data shape it was designed to compress, and its advantage is clear.

Anything destined for an API, database, or existing software: keep using JSON. It's universal, strict, and supported by every library by default. Don't swap in YAML or TOON at these layers—conversion should only happen at the boundary where you interact with the model.

A practical engineering pattern: keep JSON as the actual data flowing through your system, convert only the text handed to the model into YAML or TOON, and require the model to return anything that needs to go back into code as JSON.

The YAML Parsing Pitfall Nobody Warns You About

YAML's compactness comes from looser syntax than JSON, and looseness has specific failure modes. The most famous is the Norway problem. In YAML 1.1—which many libraries still implement, including older versions of PyYAML—bare no, yes, on, and off are parsed as booleans rather than strings.

A seemingly normal list of country codes:

geoblock_regions:
  - dk
  - fi
  - is
  - no
  - se

gets silently parsed as ["dk", "fi", "is", false, "se"], turning Norway (no) into False. Version-number strings have the same problem: 10.23 gets parsed as a float rather than the string "10.23", breaking any logic that compares versions as text.

This isn't a theoretical risk for developers. The top-level key in a GitHub Actions workflow file is on: (e.g., on: push). According to a real bug report against the Go YAML library, a generic YAML 1.1 parser reads that key as the boolean true rather than the string "on", so any tool parsing workflow YAML with a non-GitHub-specific library can silently drop or mishandle trigger configuration. GitHub's own runner handles on: correctly because it's YAML 1.2-aware—but if you're writing your own workflow linter, generator, or an LLM tool that reads .github/workflows/*.yml, this is exactly the kind of bug that passes code review and breaks in production.

There are two fixes: quote the key ("on": push), or switch to a YAML 1.2-compliant parser. YAML 1.2.2 is the current spec, revised in October 2021, and it removed bare on/off/yes/no boolean shorthands precisely to fix bugs like these. If you feed YAML to an LLM and expect it to reason about or regenerate that YAML reliably, this trap doubles in impact: models are also likely trained with YAML 1.1-style parsing assumptions.

The key difference between YAML 1.1 and 1.2 lies in type inference rules. YAML 1.1 (released 2005) carried over many implicit type conventions from Perl and RFCs, treating bare y/n, yes/no, on/off, true/false all as booleans, while also supporting octal like 010, underscore thousands separators like 1_000, and sexagesimal times like 12:30. YAML 1.2 (2009) dramatically simplified these rules, keeping only true/false as boolean literals and returning on/off/yes/no to plain strings. The problem is that a large number of mainstream libraries were born in the 1.1 era or retain 1.1 behavior for backward compatibility: Python's PyYAML defaulted to 1.1 for a long time, Go's go-yaml v2 is 1.1, and only v3 switched to 1.2 semantics. That's why the same YAML can yield completely different types under parsers from different languages and versions—it's not a question of "is the syntax right" but "which spec's semantics are you reading it under."

Why Token Counts Matter Far More for Agents Than for Single Conversations

In a single chat message, YAML saving 60 tokens over JSON is nearly irrelevant. Where it genuinely compounds is very specific: AI agents that resend accumulating context every turn—conversation history, tool outputs, retrieved documents, all of it. If an agent calls a tool 20 times in a session and each tool returns an array of JSON rows, you pay JSON's extra overhead 20 times, not once.

This matches the core economics of prompt compression: what's worth optimizing is the lever that gets pulled on every request, not one pulled only once. For agent architectures, serialization format is exactly such a lever.

The agent context resend mechanism is key to understanding this section's economics. Unlike single-turn Q&A, an agent typically runs a "cumulative context" loop: each turn it reassembles the system prompt, conversation history, all past tool calls and their return values, and retrieved documents into one request sent to the model. The context window is a finite resource (e.g., 128K tokens), and you pay again for everything already present each turn. If a tool is called N times in a session, then on the k-th call, the content returned by the previous k-1 calls is still in context, so the serialization overhead gets counted repeatedly N, N-1, N-2... times. Total overhead accumulates on the order of O(N²) rather than linearly at N times—which is why format choice escalates from "style question" to "cost lever" in agent scenarios.

FAQ

Is YAML actually faster to parse for LLMs, or is it just fewer tokens? This article only measured token counts—the levers for cost and context window. Parsing speed is a separate question, depends on the runtime environment, and has nothing to do with the model itself.

Will TOON replace YAML? No. The two only overlap on tidy table data. For nested, irregular structures (most real config files), TOON's own spec uses YAML-style indentation, so switching formats offers no efficiency gain.

Why does "no" become false in YAML? YAML 1.1 treats a set of bare words (y, n, yes, no, on, off, true, false) as boolean literals when unquoted. YAML 1.2.2 removed this behavior from the spec, but many parsers (older PyYAML, Go's go-yaml v2) still implement 1.1 rules for compatibility.

Should I quote every YAML value? Only quote values that could be misread as another type—country codes, version-number strings, and anything that looks like yes/no/on/off. For keys like GitHub Actions' on:, quote the key itself: "on": push.

Key Takeaways

Measured with a real tokenizer, YAML saves roughly 30%–33% of the tokens versus JSON on both nested config and tabular data. On tidy table data, TOON goes further—70% fewer than JSON and 56% fewer than YAML. For nested, irregular data, YAML and TOON are comparable in size, and YAML wins on compatibility. The Norway problem (no → false) is real and has already broken actual tools, with GitHub Actions' on: key being a classic victim under non-GitHub-specific parsers. JSON remains irreplaceable at API and database boundaries—format conversion should only happen at the boundary where you interact with the LLM.

When weighing YAML versus JSON in your own pipeline, the honest answer is "it depends on the shape of the data," not "YAML always wins." Before committing, measure your own real payloads with a tokenizer, and watch out for unquoted values that look like booleans.