Governed Execution vs. Runtime Planning: How Enterprise Analytics AI Achieves 100% Reproducibility
440 experiments show governed AI execution beats runtime planning 110-to-0 on strict answer-and-evidence contracts.
An arXiv paper conducts 440 controlled experiments comparing two AI architectures for enterprise data analytics: runtime planning (8B LLMs autonomously generating SQL and workflows) vs. governed execution (LLMs interpret intent only; a deterministic policy engine runs pre-approved programs). Using a strict answer-and-evidence contract, runtime planning scored 0/330 while governed execution achieved 110/110. The authors note this is configuration-specific and doesn't invalidate runtime agents broadly, but the findings offer enterprises a practical path to auditable, reproducible AI analytics.
When Enterprise Analytics Meets the LLM "Uncertainty Problem"
Large language models are increasingly being deployed in enterprise data analytics — but a core tension persists: the same flexibility that makes these models powerful also makes them unpredictable. When an AI agent autonomously plans its analysis path, generates SQL, and selects tools at runtime, it might return the right answer — or it might quietly go wrong. Worse, ask the same question twice and you may get two different answers.
A recent paper published on arXiv (arXiv:2609.03209) addresses this problem with a "governed execution" paradigm. The core idea is counterintuitive: instead of letting the model freely improvise, the language model is responsible only for understanding the question, while a deterministic policy engine selects and runs pre-approved analysis programs. This design trades some flexibility for what enterprise applications value most — reproducible, auditable results.
Governed Execution vs. Runtime Planning: A Controlled Experiment
Two Fundamentally Different Architectures
The study compares two technical approaches. The first is the popular "runtime planning" model: 8B-scale models autonomously generate SQL, select tools, and determine analysis workflows at runtime. Three different 8B models were used to fill this role across experiments.
The second is the paper's proposed "governed" approach: a Qwen3-8B model is responsible solely for interpreting user intent, while actual analysis program execution is handed off entirely to a deterministic policy engine. The policy engine selects the appropriate program from a set of pre-approved analysis routines, runs it, and returns results along with a full evidence chain.
Expressive Power Is Not Sacrificed
Interestingly, this "constraint" doesn't mean functional regression. The research shows that within a well-defined analytics category, the governed approach still maintains sufficient expressive power — covering relational operations, aggregation, comparison, windows, ranking, and similarity, among other common analytical operations.
In other words, the vast majority of analytics tasks that enterprises need day-to-day can be completed within this governed framework, without relying on the model to improvise.
The Numbers: 110 vs. 0
The experiment ran 440 total runs. The evaluation standard was a strict "answer-and-evidence contract" — results had to be not only correct, but accompanied by complete, verifiable evidence.
The contrast was stark:
- Runtime planning: Across all 330 runtime planning runs, not a single one fully satisfied the answer-and-evidence contract across all test datasets.
- Governed execution: The policy engine's analyzers achieved a 110-of-110 hit rate across all 110 tests.
This dramatic gap reveals a key insight: once the evaluation bar is raised from "roughly correct answer" to "correct answer with complete, reproducible evidence," the free-planning approach exposes serious reliability deficiencies.
The Source of Reproducibility
The governed approach achieves its 100% contract fulfillment rate because of what might be called a "four-fixed" design:
- Fixed semantic meaning
- Fixed policy
- Fixed data
- Fixed execution rules
When all four elements are locked down, any analysis run can be fully replayed — identical inputs always produce identical outputs. For heavily regulated industries like finance, healthcare, and compliance, this is essentially a non-negotiable hard requirement.
How to Interpret These Findings Rationally
The Boundaries of This Conclusion
The researchers are careful to emphasize the limits of their findings: this is a "configuration-specific result" that does not prove runtime agents cannot succeed under other designs.
This caveat matters. The paper is not trying to wholesale dismiss the value of runtime agent planning. Rather, it argues that under this specific evaluation contract, model scale (8B), and task setup, governed execution demonstrates an overwhelming reliability advantage. Larger models, more sophisticated agent designs, or more lenient evaluation criteria could all shift this conclusion.
Three Takeaways for Enterprise AI Deployment
This research has direct practical implications for enterprises exploring AI-powered analytics tools:
First, consider governed architectures when reliability is paramount. For enterprise analytics that require auditing, compliance, and reproducibility, limiting the model's role to "understanding intent" and delegating execution to a deterministic engine may be a more pragmatic choice than fully autonomous agents.
Second, model scale is not a silver bullet. The experiments used 8B-scale models throughout. At this scale, autonomous runtime planning falls clearly short on reliability — a reminder not to place blind trust in small models' autonomous decision-making capabilities.
Third, your evaluation criteria should drive your architecture choices. If you only check whether the answer is right, the gap between the two approaches may seem modest. But once you introduce strict contracts requiring complete evidence and reproducibility, the importance of architectural decisions becomes impossible to ignore.
Finding the Balance Between Flexibility and Control
The value of this research lies not in declaring one technology the "winner," but in using 440 experimental runs of empirical data to clearly map out a critical trade-off in enterprise analytics AI design: the tension between flexibility and controllability.
For exploratory, innovation-driven use cases, runtime planning agents still have plenty of room to shine. But for enterprise-grade, heavily regulated, and accountable analytics tasks, the governed paradigm — "let the language model understand the question; let the deterministic policy run the analysis" — offers a realistic path to trustworthy AI. When AI is truly entering the core decision-making processes of an enterprise, "right every single time" is worth far more than "occasionally brilliant."
Related articles
DeepSeek V4 Pro Burning Through Credits Too Fast? The Hidden Logic Behind AI Model Pricing
Why does DeepSeek V4 Pro drain credits so fast while Flash barely moves? A deep dive into AI token billing, Pro vs. Flash pricing differences, and cost optimization tips.
RealPDE Competition Breakdown: The Frontier Challenge of AI-Powered Real-World Fluid Dynamics PDE Solving
A deep dive into the NeurIPS 2026 RealPDE Competition, covering the Sim2Real and LTTTA tracks, and how neural operators tackle real-world PIV and CFD fluid PDE challenges.
Building a Production-Grade 3DGS Training Library from Scratch: A Deep Dive into Full-GPU Residency and the Vulkan Stack
A veteran graphics engineer builds a production-grade 3DGS training library from scratch using C++23, CUDA, and Vulkan, achieving 60fps with 5M splats. Deep dive into its architecture and design.