Human Approval in AI Agents Is in the Wrong Place: Do You Need Two Checkpoints?

A Design Mistake Nobody Talks About

When building AI Agent workflows, many developers instinctively add a "human review" step at the end of the pipeline — having a reviewer check the Agent's final output. This feels cautious, but it may be placing the checkpoint in exactly the wrong spot from the very beginning.

A Reddit user shared this realization while building workflows with FastGPT. He had originally positioned the reviewer near the end of the process, tasked with checking the final output. He quickly realized: "Is the evidence the Agent used reliable?" and "Should this email be sent, this refund processed, this content deleted or published — right now?" are two fundamentally different kinds of risk.

This seemingly small observation touches on a core question in AI Agent safety design: where you place human oversight determines what risks it can actually catch.

Two Fundamentally Different Types of Review

Treating "human review" as a single, monolithic concept is a hidden flaw in many workflow designs. The original poster astutely broke it into two distinct categories:

Decision / Evidence Review

This type of review focuses on the quality of the reasoning process: Did the Agent reach its conclusion based on sound evidence? Is the information it cited reliable? Are there obvious gaps in the reasoning chain? This is a check on "quality of thinking," and it happens at the decision-formation stage.

Pre-Action Review

This type of review focuses on irreversible external actions: Should this email go out? Should this refund be executed? Should this content be published? Once these actions are taken, they often cannot be undone — a fundamentally different risk profile from the first type.

The author's core insight is this: a reviewer sitting at the end of a pipeline, looking only at the final answer, likely hasn't truly evaluated the quality of the evidence or specifically guarded against high-risk external actions. The "risk" being reviewed doesn't map to the risks that actually need reviewing.

This distinction aligns closely with the "control point" theory in information security. Traditional access control design has long differentiated between authentication and authorization — the former asks "who are you?", the latter asks "what are you allowed to do?" The two types of review in Agent workflows are structurally identical: decision/evidence review validates the legitimacy of reasoning; pre-action review acts as secondary authorization for high-privilege operations. Ignoring this distinction means compressing two different levels of control responsibility into a single review action — and that will systematically cause one category of risk to be overlooked.

The Two-Checkpoint Design — and Its Hidden Danger

Based on this analysis, the author proposes a more sensible architecture: two pause points. One centered on evidence and decision quality; another placed immediately before any external action is executed.

This dual-checkpoint design logically mirrors the actual distribution of risk. The decision checkpoint ensures the Agent "thinks correctly"; the action checkpoint ensures the system "acts correctly." Each has a clear, distinct responsibility.

But the author raises an important counter-concern: the second checkpoint risks degenerating into pure muscle memory. When reviewers face a stream of "confirm execution" dialogs, it's easy to slip into mechanical clicking over time. The review becomes performative. At that point, this seemingly safer checkpoint may actually be worse than having a single, carefully considered review — because it creates a false sense of security that "we already checked."

This is a remarkably sharp observation. The value of human review lies not in whether the step exists, but in whether the reviewer is in a cognitive state where they can actually make a judgment. A checkpoint that fires too often and loses the reviewer's attention is equivalent to no checkpoint at all.

This cognitive degradation is known in Human Factors Engineering as Automation Bias — when humans are placed in a supervisory role over automated systems for extended periods, they gradually tend to over-trust system outputs and reduce their independent judgment. The classic example comes from aviation: pilots in highly autopilot-dependent environments show declining manual control skills and situational awareness over time. In Agent approval workflows, the more frequently the "confirm execution" dialog fires and the more consistently the Agent performs, the harder it becomes for reviewers to maintain genuine vigilance. This means checkpoint design must consider not just where to place it, but how to balance trigger frequency against cognitive load.

Design Principles for Agent Workflows

From this discussion, a few practical design principles emerge:

Place review points based on risk type, not pipeline position. Don't default to stacking reviews at the end. Instead, think carefully about what specific risk each checkpoint is meant to intercept, and place it where it can actually do its job.

Distinguish between reversible and irreversible actions. Irreversible operations — sending, refunding, deleting, publishing — inherently require a separate, more deliberate approval mechanism. They shouldn't be lumped together with general quality checks.

Watch for approval fatigue. If a checkpoint fires too frequently, ask whether it's truly effective. One approach: reserve human approval for high-risk scenarios while automatically passing low-risk actions through — preserving the reviewer's attention and judgment for when it matters most.

The question the author poses at the end of his post — "Has anyone actually run both checkpoints simultaneously?" — remains open. This reflects the fact that balancing security and usability in practice still lacks mature consensus. But the way he frames the question already provides a valuable mental model for any developer building AI Agent workflows.

At the engineering implementation level, some Agent frameworks have begun offering native support for marking "irreversible tool calls" — allowing developers to declare at tool-definition time whether a function has side effects, and automatically inserting approval nodes accordingly. The "interrupt" mechanism introduced in frameworks like LangGraph is designed precisely to pause execution at specific edges in the graph and wait for human confirmation. This kind of explicit architectural distinction is far more reliable than repeatedly instructing the model to "be careful" in prompts — it moves the risk boundary from the model reasoning layer down to the execution engine layer, fundamentally reducing the probability of security failures caused by prompt drift.

Conclusion

Human approval is not a safety patch you can insert anywhere — it's a design decision that must be precisely aligned with the structure of risk. A misplaced checkpoint may waste human effort while failing to actually reduce risk. Mechanically stacking checkpoints may create a false sense of security. The real challenge is ensuring that every moment of human intervention happens exactly when and where it should.