Can You Trust AI-Written Test Reports? IntDEx Proposes Evidence Grading for Behavioral Verification

When an AI model writes the code, the tests, and the test report all at once, a simple "all tests passed" may just be a self-confirming hallucination. The IntDEx framework proposes the principles of "evidence grading" and "treat self-reports as unverified," transforming behavioral verification in the AI SDLC from trusting summaries into auditing an evidence ledger.

The Problem: When the Same Model Is Both Author and Judge

In an AI-assisted software development lifecycle (AI SDLC), the same model can draft the intent, the prompts, the implementation, the tests, and the final test report. The IntDEx framework directly identifies this risk: a single misunderstanding can be reproduced consistently across every one of these artifacts, so that "traceability looks complete, coverage looks sufficient, verification looks good, and the entire chain is wrong."

This is not a theoretical concern. NIST's Generative AI Profile (NIST AI 600-1) defines "confabulation" as content that is confidently stated but wrong or false; research published at ICLR 2024 (Huang et al., 2024) found that large language models struggle to correct their own reasoning without external feedback, and that performance sometimes even degrades after self-correction. A testing phase that accepts the authoring model's own verdict inherits both weaknesses at once.

In other words, "traceability" between artifacts written by a single author may be nothing more than internal consistency, not genuine verification.

NIST AI 600-1 is the Generative Artificial Intelligence Profile published by the U.S. National Institute of Standards and Technology (NIST) in 2024. It supplements the NIST AI Risk Management Framework (AI RMF 1.0) and provides actionable management guidance specifically for the unique risks of generative AI (such as confabulation, data privacy, intellectual property, and information integrity). The document defines "confabulation" as the phenomenon in which a model confidently generates erroneous or false content, and emphasizes that organizations should establish independent verification mechanisms rather than relying on model self-reports. In the AI SDLC context, NIST AI 600-1 signals that trust in model outputs must be grounded in external evidence and governance controls—providing standards-level support for IntDEx's "evidence grading" principle.

What IntDEx Changes in Phase Four: Behavioral Verification

IntDEx (AI-Native Delivery Experience) defines a seven-phase AI-native delivery process. Phase four, "behavioral verification," ensures that the delivery engine's outputs are correct, safe, ethical, and aligned with stakeholder expectations, with user testing focused on behavior rather than code. Testers assess outputs for correctness, bias, hallucination, constraint violations, and drift.

Compared with traditional QA, UAT, test automation, and evaluation, the framework makes four specific changes:

Evidence independence. A correctness claim is acceptable only when the evidence comes from a source other than the authoring model. Evidence sources are ranked: execution evidence (exit codes, test results, build logs, HTTP responses, database state) outranks human evidence, human evidence outranks cross-model evidence, and cross-model evidence outranks comparison with prior artifacts.

Self-reports treated as unverified. In the IntDEx framework's own words, the authoring model's claim that a check passed "carries no evidentiary weight and must be recorded as unverified rather than passed." The engine's own report is not evidence; humans adjudicate.

Paired behavioral verification. Acceptance verification is performed in pairs—for example, an end user paired with a tester, or a user paired with an analyst; security, UI/UX, and compliance testing are automated before release and checked by human testers.

Continuous verification. Automated continuous behavioral verification (ACBV) re-runs behavioral evaluations whenever the engine configuration changes—whether prompts, constraints, context, or model switches—and uses pass rates as a change gate, so that behavioral quality cannot silently degrade.

Paired behavioral verification is a mechanism that distributes verification responsibility across different roles or model families, with the core aim of preventing a single entity from acting as both "author" and "judge." In IntDEx, pairing can occur between an end user and a tester, between a user and an analyst, or between models from different provider families. The principle resembles the "four-eyes principle" or separation of duties in software engineering: the verifier maintains a degree of independence from the creator in terms of information, objectives, or model architecture, thereby reducing the risk of sharing the same blind spots. Paired verification does not require the verifier to be fully independent of the system, but it does require that the verifier's evidence sources or judgment perspective differ from the creator's, and that disagreements be explicitly recorded and referred to humans for adjudication.

Research-Style Verification: From Design Science to Reconstruction Trials

Question. Can behavioral verification in the AI SDLC produce assurance when the authoring model writes both the tests and the report?

Method. IntDEx was developed using a design science research methodology, treating the framework as an artifact refined through practical application in a public health data experimentation platform. The evaluation covers reconstructing the solution from the artifact alone, as well as how evidence independence and human oversight controls operate.

How verification was applied. The IntDEx framework describes applying these controls during delivery of that platform and when attempting to reconstruct the platform from IntDEx artifacts alone. Mandatory human checkpoints were used to pause work when the engine's own evidence was judged insufficient, and to correct the severity level the engine assigned to a change. Models from different provider families reviewed prompt execution, and their disagreements were passed to humans, who decided whether a change was needed. Reconstruction was assessed across five dimensions—behavior, security, data structures, architecture, and configuration—and the framework notes that security controls implicit in the original system rather than written down as governed intent or security artifacts may not be reliably reproduced.

Illustrative example. Take a typical request: a website contact form that sends inquiry emails to a business owner. Because it involves personal data, IntDEx treats it as a Level 2 (elevated) change, setting a synchronous blocking checkpoint. A verification record might look like this:

Acceptance Criterion	Evidence	Grade	Status
Valid inquiries are accepted and the owner is notified	Integration test exit code and captured outbound messages	Execution	Acceptable
Abusive duplicate submissions are rate-limited	Engine claims "rate limiting implemented"	Self-report	Unverified
Same criterion, after flood-style negative testing	Recorded HTTP 429 response	Execution	Acceptable
Form wording matches intent	Recorded paired user–tester review	Human	Acceptable

This table clearly illustrates IntDEx's core logic: the engine saying "done" doesn't count—execution-level evidence must be produced, or a human must vouch for it.

Proposed evaluation criteria (not results). Teams adopting this practice can track metrics from the framework's own checklist: the percentage of production claims backed by independent evidence, behavioral regressions, first-pass acceptance rate, escaped defects, and negative control test pass rate. Because evaluations of non-deterministic systems vary across runs, IntDEx recommends first establishing expected run-to-run variance and gating only changes that exceed that variance.

Design Science Research (DSR) is a research methodology in the information systems and software engineering fields that emphasizes generating design knowledge by building and evaluating artifacts. Its typical process includes: identifying the problem, defining requirements, designing and developing the artifact, demonstrating the artifact, evaluating the artifact, and summarizing design principles. In the IntDEx context, the "artifact" is the seven-phase AI-native delivery framework itself, evaluated through practical application (a public health data experimentation platform) and a "reconstruct the solution from the artifact alone" trial. DSR does not pursue statistically universal proof, but rather establishes credibility through the artifact's utility and reproducibility in real-world settings—consistent with IntDEx's position of presenting itself as "preliminary feasibility evidence."

Limitations and Boundaries: Preliminary Feasibility, Not Universal Proof

The IntDEx framework openly acknowledges that its application to date covers only a single project and a single primary evaluator, and treats this as preliminary evidence of feasibility rather than universal proof; independent cross-case replication is still needed.

Cross-model review is advisory: it reduces risk but does not eliminate it. Human reviewers themselves are also a failure mode. IntDEx names automation bias—which Article 14 of the EU AI Act likewise requires high-risk systems to address—and treats a zero-rejection rate at checkpoints as a finding that needs investigation rather than a mark of quality. This point is worth pondering: if humans have never once rejected an AI's output, it may not be because the AI is so good, but because the humans have stopped genuinely reviewing.

Automation bias refers to the tendency of humans, when faced with the output of automated systems, to over-trust or adopt the system's suggestions while reducing their own independent judgment. In AI-assisted development, when a model generates test reports or change recommendations, reviewers may accept the results directly—even when the evidence is insufficient—due to time pressure, cognitive load, or trust in the AI's capabilities. Article 14 of the EU Artificial Intelligence Act requires high-risk AI systems to have effective human oversight, and specifically points out the need to guard against automation bias. IntDEx treats a "zero-rejection rate at checkpoints" as an anomalous signal requiring investigation precisely because sustained zero rejections may mean human oversight has become a formality, rather than the system's quality truly being flawless.

Conclusion: Grade the Evidence First, Then Trust the Results

In IntDEx, behavioral verification is not "run the tests and then trust the summary." It is an evidence ledger in which every claim is labeled with its source, self-assessments are marked as unverified, and named humans accept or reject the evidence. For teams transitioning to the AI SDLC, this is a pragmatic first step: before trusting the results, grade the evidence.

The framework's deeper insight is that the greatest risk of AI-assisted development is not that AI can't write code, but that we lose the ability and the will to independently judge whether the code is correct. An evidence grading system is, at its core, a rebuilding of the checks and balances of quality assurance in the age of AI.