The Ignored finish_reason: How LLM Truncation Silently Poisons Your Data Pipeline

A weekly job summarizes customer support tickets into a report — one chapter per product. The report reads smoothly and confidently, arriving every Monday without fail. Nobody noticed that the last two products had been missing for an entire month.

Ticket volume was growing, and the report grew with it. But the max_tokens limit set when the report was half its current size had never been touched. This real-world scenario reveals an engineering trap that's almost universally overlooked: when a large language model's output gets truncated, it won't tell you — at least not in the text.

What Happens When a Model Hits Its Limit

When a model reaches its max_tokens limit, it doesn't wrap up, apologize, or add "that's everything." It just stops — possibly mid-list, possibly mid-word — and hands back whatever it managed to generate.

Here's the thing: the response actually does explain this. Every serious API returns a reason for why generation ended. It could be a natural stop, hitting the length limit, triggering a stop sequence, or being blocked by a content filter. This field is usually called finish_reason or something similar, and it's right there in the response.

The problem is that the code only ever reads the text — it never reads the reason.

The finish_reason field behaves slightly differently across major APIs, but the logic is consistent. The OpenAI API returns a finish_reason with common values including: stop (the model naturally generated an end token), length (the max_tokens limit was reached), content_filter (content was blocked by safety policy), and tool_calls (the model requested a tool call). Anthropic Claude uses a corresponding field called stop_reason, with values like end_turn, max_tokens, and stop_sequence. Google Gemini uses finishReason, with values including STOP, MAX_TOKENS, and SAFETY. Regardless of naming, the meaning is consistent: a value like length or MAX_TOKENS means the model was forcibly cut off, not that it finished on its own. This field typically lives at choices[0].finish_reason (OpenAI-style) or in a top-level stop_reason field — one line of code to read it, yet the vast majority of integration code never touches it.

Prose Is Best at Hiding Truncation

Truncation is dangerous precisely because it manifests so differently across output formats.

Natural language prose hides the problem most effectively. A report that should have eight sections, stopping abruptly after the sixth, looks exactly like a report that was always meant to have six sections. No syntax errors, no parse failures, no anomalous signals — it just "doesn't have the last two products," and the missing parts don't raise their hands.

Structured output fails differently. Truncated JSON can't be parsed, which causes an explicit error. That sounds like a good thing — at least the failure is visible. But this is often where the real danger begins.

Don't Treat "Fixing" as a Solution

When truncated JSON fails to parse, someone will inevitably add a "repair" step: automatically append missing brackets, close uncompleted structures. Suddenly, a half-finished answer passes validation.

This is the worst possible approach. The warning here is blunt: don't repair broken output into valid output. If it can't be parsed, it's incomplete — and that incompleteness should be loud. It should throw errors, halt the pipeline, not be quietly papered over and passed downstream.

A half-answer that passes validation is far more dangerous than a request that explicitly fails, because the former masquerades as a correct result and flows into downstream systems undetected.

Read finish_reason Like an HTTP Status Code

A core piece of advice worth every LLM integration engineer memorizing: read the stop reason after every call, the same way you read an HTTP status code.

• Finished naturally means success.
• Hit the limit is not an answer. It's an error, or a signal to continue generating — but it should never be passed to downstream systems as a result.

The analogy is apt. Nobody pretends a request succeeded when they get an HTTP 500 — yet in LLM integrations, countless systems are doing exactly that with an equivalent failure state.

Concrete Defenses You Can Actually Implement

Here are a few specific actions to build protection:

1. Leave Real Headroom in Your Limits, and Alert When You're Getting Close

When setting max_tokens, build in genuine headroom, and trigger alerts when calls approach the limit. Slow, gradual growth is the classic path to hitting the wall — just like that report that quietly grew week by week, with no single dramatic change, until the day content started getting cut off.

2. If You Know the Structure, Validate the Structure

When the shape of the output is known, check the shape. Eight products go in, eight sections should come out. This kind of business-semantic "count assertion" often catches truncation far better than pure JSON format validation — because a truncated output may be structurally valid but come up short on count.

3. Log Stop Reasons Alongside Costs

Record finish_reason together with call costs and plot them on the same monitoring dashboard. That way, the week summaries start truncating early will show up in the chart — not surface in a meeting when someone finally notices.

This is exactly what observability is for: exposing problems in your data, not in your incidents.

---

For scenarios that require generating longer content, there's also an architecture-level strategy: streaming output paired with incremental validation. With streaming enabled, clients receive tokens in real time during generation. The moment finish_reason: length is detected, alerting or retry logic can fire immediately — no need to wait for the full response before discovering truncation. For very long tasks, segmented generation is another approach: split the large task into multiple independent smaller requests, validate completeness for each segment, then merge results at the application layer. This reduces the probability of any single call hitting the limit, and makes truncation issues in each segment easier to isolate. Segment boundaries should follow business semantics (e.g., one call per product) rather than hard character counts — otherwise structural consistency after merging becomes very hard to guarantee.

An Underestimated Engineering Blind Spot

This is a short piece, but it pinpoints a deeply representative blind spot in LLM engineering. As models become more capable and their outputs smoother, it's easy for developers to default to assuming "if the model stopped, it must be done." But output completeness and output quality are two entirely separate concerns.

A pipeline that runs smoothly, delivers on schedule every week, and reads without a single flaw may have been silently losing a quarter of its data — and the only thing that could have told you the truth is the field you never bothered to read.

"A model that hit its output limit won't tell you in the text. It'll tell you in the field you weren't reading."

For any team integrating LLMs into production workflows, this deserves to be written into the engineering standards.