When an LLM Explains Correctly but Decides Wrong: A Business Evidence Judgment Benchmark

A large language model can accurately explain that a policy "only applies to Europe," yet return a label in its final output claiming it applies to North America as well. This "correct explanation, wrong decision" phenomenon is exactly the core problem exposed by this Kaggle benchmark challenge. For systems that rely on model output labels to drive workflows, even a flawless explanation won't save a downstream process from going off the rails due to a single incorrect decision field.

Test Design: Can Models Judge Whether Evidence Supports a Claim?

The benchmark, called Business Evidence Judgment v4, tests whether models can determine if provided business records support, contradict, or leave undetermined a given "claim." The test spans six business domains: vendor onboarding, customer delivery, system migration, interface compatibility, configuration release, and data pipelines. All records are original, AI-assisted fictional content — not real production data.

Each business family contains six scenario–claim probes, for a total of 36 claims. These 36 claims are repeated across three different source conditions, yielding 108 judgments per model (18 requests):

• S0: Raw source material of approximately 2,500 words
• S1: Administrative context added, expanded to approximately 9,800–9,900 words
• S2: Outdated or similarly scoped distractor records added, same word count as S1

Because content length and evidence position vary simultaneously, score differences cannot be attributed solely to additional token count. Models may only answer supported, contradicted, or unknown — the key distinction being that "missing approval confirmation" does not equal "approval denied." Models must differentiate between evidence that "fails to establish" versus evidence that "establishes the opposite."

The Rigor of Scoring Protocol v4

The v4 protocol scores each of the 108 entries independently. Missing, duplicate, invalid, or incorrect answers only deduct points for that specific entry — other valid answers retain their scores. The parser only accepts a canonical decisions object or root array, and will not repair IDs, labels, or infer positional matches. If an entire batch fails JSON parsing, all six entries in that batch become unscorable.

The protocol's evolution is worth noting. The earlier exploratory v3 version would zero out an entire batch upon schema validation failure. An audit revealed that rejected batches actually contained 48 identifiable correct labels hidden within them. The author redesigned the IDs, containers, and scoring rules accordingly, while preserving the original business records, claims, labels, and order. The author explicitly states: this was a post-hoc reconstruction, not a controlled ID-length experiment, and earlier scores are not merged with current results.

The three conditions S0/S1/S2 are fundamentally a context pressure-testing framework. S0 provides lean core material to assess baseline judgment when information is sufficient but undistracted. S1 expands the text nearly fourfold by injecting administrative context (organizational structures, process descriptions, etc.), testing whether models can locate key evidence amid a large volume of irrelevant information. S2 goes further by introducing "distractor records" — records that are topically or temporally similar to the target but are either expired or out of scope, specifically designed to trigger confusion or misattribution. This design simulates common information redundancy in real enterprise environments, where systems often contain both current and historical versions of documents simultaneously. The three-condition comparison reveals a model's "signal-to-noise resistance" rather than merely measuring comprehension accuracy on clean inputs.

Participating Models: A Mix of Hosted and Open-Weight

The six models that completed the test are Claude Haiku 4.5, gpt-oss-20b, Gemini 2.5 Flash, GPT-5.4 mini, Gemma 4 26B, and Qwen Next. This lineup deliberately mixes hosted models with open-weight models, targeting cost-sensitive deployment scenarios.

The author cautions against being misled by parameter names: these sparse MoE models have vastly different "total parameters" versus "active parameters." Official model cards show Gemma 4 26B A4B at 25.2B total / 3.8B active, Qwen3-Next-80B-A3B at 80B / 3B, and gpt-oss-20b at 20.9B / 3.6B. Names like "mini" and "Haiku" disclose no parameter scale at all.

Tests used frozen requests: system message plus user message, 8,192 token output cap, streaming disabled. Temperature, seed, and reasoning intensity were not set, using provider defaults. The author emphasizes that identical prompts and output caps do not equate to identical reasoning budgets or compute.

MoE (Mixture of Experts) architecture is key to understanding the difference between total and active parameters. Traditional dense models activate all parameters during inference; MoE models partition parameters into multiple "expert" sub-networks, routing and activating only a small fraction during each forward pass. Take Qwen3-Next-80B-A3B as an example: of its 80B total parameters, only about 3B are activated per inference — meaning its computational cost is comparable to a 3B dense model while theoretically retaining larger knowledge capacity. This architecture achieves a unique balance between inference cost and expressive power, but introduces ambiguity in parameter-scale labeling. Comparing "parameter counts" without distinguishing activation ratios severely misleads assessments of model capability and cost. Part of gpt-oss-20b's strong cost-efficiency in this test stems precisely from its low active parameter count (3.6B) translating to low inference cost.

Results: A Divergence in Cost and Reliability

Below are the official run results completed on the Kaggle Task platform version 2, using the v4 scoring protocol:

Model	Correct /108	Score	S0/S1/S2 (each /36)	Run Cost USD
Claude Haiku 4.5	98	90.74%	32 / 32 / 34	$0.363214
gpt-oss-20b	94	87.04%	31 / 32 / 31	$0.022485
Gemini 2.5 Flash	92	85.19%	33 / 26 / 33	$0.236623
GPT-5.4 mini	91	84.26%	33 / 29 / 29	$0.225153
Gemma 4 26B	80	74.07%	22 / 28 / 30	$0.056895
Qwen Next	6	5.56%	0 / 0 / 6	$0.062775

One standout is gpt-oss-20b: it scored 94/108 at the lowest run cost in the test (roughly 2.2 cents). The author carefully notes this is useful evidence for this particular workload, not a universal cost or capability advantage.

As for the one-item gap between Gemini and GPT, the author considers it insufficient to establish a stable ranking — this small-scale task has limited coverage and cannot reliably differentiate models with similar scores. What would be truly valuable is "paired errors on the same claim" and more independent case families, rather than treating these percentages as a robust leaderboard.

Qwen Next's Format Collapse

Qwen Next's dismal score stems from format reliability, not comprehension. It produced 17 invalid_json responses, leaving only 6/108 entries scorable. Local inspection found it output six comma-separated objects with no outer array brackets. Two representative responses reported COMPLETE/stop status with 632/742 completion tokens, well within the output length limit. The author did not repair them or extrapolate a hypothetical corrected score. This format bottleneck entirely blocked a clean evaluation of its evidence comprehension ability.

On batch compliance: GPT scored 18/18; Haiku, gpt-oss, and Gemini each scored 17/18; Gemma scored 15/18; Qwen scored only 1/18. But the author repeatedly emphasizes — none of these metrics measure the truthfulness of explanations.

Key Finding: GPT-5.4 mini's "Correct Explanation, Wrong Decision"

GPT-5.4 mini exhibited a different failure mode — the one directly behind this article's title. Independently re-scoring 18 exported outputs reproduced 91/108, with complete prediction coverage, explanation field compliance, and batch compliance. Its 17 errors were all label disagreements, spanning ten distinct claims.

A concrete example comes from the system migration domain, S2 condition, Q2 entry:

Claim: "CAB-618 also moves the North America ledger window to November 13."

Record ELM-22 states:

"This record does not change the North America migration window."

The reference label is contradicted. GPT returned supported, but its explanation read:

"CAB-618 is the record named in ELM-22 and it moves only the European ledger production window; the North America window is explicitly left unchanged."

The explanation fully preserves the regional restriction, demonstrating the model did read the source correctly. Yet the decision field is still wrong. A system using that label to update a migration plan could take action directly contrary to the explanation sitting right next to it.

Another failure involved a meaningful business identifier. P1-S0/Q2 asked whether all 480 M17 actuators met assembly eligibility. The record established complete sign-off, signed release, and required relabeling. GPT returned unknown, explaining that M18 lacked sign-off and quality disposition evidence — it answered about a different batch and repeated this misattribution error across all three conditions.

Both cases point to the same conclusion: correct formatting and seemingly coherent text are insufficient as a quality check for structured decisions. The author recommends that the next measurement step should independently verify consistency among claims, business entities, evidence, explanations, and final labels — evaluated separately from JSON validity.

This "explanation–decision decoupling" failure mode is known in LLM system design as an internal consistency failure. The underlying mechanism of large language model text generation is next-token prediction — explanation fields and decision fields have no enforced logical binding during generation. Models do not "reason first, then write conclusions" the way humans do; they generate all text sequentially. This means that even if an explanation paragraph "accidentally" produces a correct reasoning chain through autoregressive generation, the model may be in a different attention distribution state when generating the decision label, or may output an inconsistent label due to prompt template effects or training biases. This phenomenon also explains why relying solely on Chain-of-Thought (CoT) prompting cannot guarantee correct final answers — CoT can improve reasoning quality, but cannot architecturally enforce logical consistency between explanation and decision. Common engineering mitigations include: extracting decision labels separately and cross-validating them against explanations, using structured output constraints (such as JSON Schema to enforce field interdependencies), or designing a verification step where the model reviews its own decision through reverse explanation auditing.

An Honest Statement of Limitations

Much of this benchmark's credibility comes from the author's candid acknowledgment of its limitations. Six synthetic families is a small, purposive sample. The 108 judgments repeat 36 claims — they are not independent cases. Label distribution is also unbalanced: 45 supported, 45 contradicted, 18 unknown — a strategy of always answering supported with valid IDs would score 41.67%.

Furthermore, each model has only one selected run, making it impossible to establish sampling variability. Semantic review covered only selected examples and was AI-assisted, not validated by human experts. The author explicitly states this benchmark "does not establish broad honesty or general business capability."

For engineering teams, the most practical takeaway from this test is: when embedding an LLM in an automated decision pipeline, you cannot only verify that the output JSON is valid and the explanation reads coherently. A correct explanation and a correct decision are two independent things. There must be mechanisms to cross-validate labels against evidence, entities, and explanations for internal consistency — otherwise downstream systems may quietly go wrong inside outputs that look perfectly professional.