Text2Dashboard: A Controlled Agent Architecture for Generating Enterprise Dashboards from Natural Language

From Natural Language to Auditable Dashboards

Enterprise data analytics has long faced a fundamental tension: business users want to ask questions in natural language, while data engineers need to ensure queries are controllable, auditable, and secure. A new paper on arXiv introduces Text2Dashboard as an attempt to address exactly this pain point — a prototype system built for the DataBrain platform that transforms natural language analysis requests into inspectable dashboards.

Unlike the common approach of having a large model output SQL directly, Text2Dashboard's core design philosophy is: the model is only responsible for proposing actions, while deterministic software controls execution and records state transitions. This separation of responsibilities allows the system to leverage the flexibility of LLMs while retaining the controllability of traditional software engineering.

Key Components of the Controlled Agent Architecture

Installable Plugin and Standalone Runtime

Text2Dashboard comes in two forms: an installable Codex plugin and a standalone Agent Runtime. Both combine schema-constrained model decisions with typed tools and persistent state. This means every output from the model must conform to a pre-defined structure, rather than being free-form text.

Schema-constrained model decisions refer to a system that pre-defines a structured output specification (JSON Schema or similar format), requiring each step of the LLM's output to conform to that specification rather than generating arbitrary natural language. This approach draws on ideas from the field of "Structured Generation" — constraining the model's token sampling space at inference time so that outputs are naturally parseable and verifiable. Typed tools are a class of interface abstractions in Agent frameworks: each tool carries explicit input/output type signatures, and when the model calls a tool, it must pass parameters that meet the type requirements. The framework performs type validation before execution, intercepting runtime errors at the execution boundary. Together, these two mechanisms confine the model's "creativity" to the space of valid actions — the system doesn't rely on the model's self-restraint, but instead relies on the static type system at the software layer to provide guarantees.

Deterministic Hooks Mechanism

Much of the system's reliability comes from its "deterministic Hooks" design, which covers approval, audit, checkpointing, recovery, and failure handling. In other words, even if the model makes a mistake, the execution layer has a hard set of processes as a safety net, ensuring no unapproved external side effects can occur.

Checkpointing and Recovery are core reliability mechanisms for long-running Agent systems, conceptually similar to a database's WAL (Write-Ahead Logging) or checkpointing in distributed computing. The system serializes and stores the current state before and after each critical action. If a step fails or times out, execution can be resumed from the most recent checkpoint without re-running the entire pipeline from scratch. This is especially important for Agents involving multiple LLM calls — each re-run not only incurs time costs, but also produces inconsistent results due to the model's stochasticity. Approval workflows force a human or rule-based confirmation step before the Agent executes any action with external side effects (such as writing to a database or calling an external API), ensuring all irreversible operations are explicitly authorized. These two mechanisms together form the core engineering difference between a "controlled Agent" and a typical ReAct-style Agent.

End-to-End Processing Pipeline

The complete processing chain includes: entity parsing, metadata discovery, enforced read-only SQL, dashboard composition, followed by static checks, dynamic preflight checks, and browser inspection. The enforced read-only SQL constraint is particularly critical — it architecturally eliminates the risk of natural language instructions accidentally triggering write operations, which is precisely the security boundary that enterprise data governance cares most about.

Experimental Results: Cautious and Honest Evaluation

The paper evaluates the system on two types of scenarios: "frozen real DataBrain tasks" and "controlled Hook failures," with notably restrained results.

On metadata and SQL tasks, the strict success rate was 6/8: metadata selection passed 4/4; all four SQL tasks met semantic criteria, but only 2/4 met the exact output-column contract. This indicates the model can "understand intent," but still falls short when it comes to precisely aligning output format.

The final release version passed 4/4 single-panel dashboard tasks, one dual-panel task, and one dashboard refinement task; one parameterized task exceeded the step limit and failed to complete.

On the fault tolerance side, all ten failure scenarios achieved expected outcomes with no unapproved external side effects — the most compelling validation of its Hooks architecture.

A noteworthy engineering data point: across every reported run, model inference accounted for over 97% of observed runtime. This means the system's performance bottleneck lies almost entirely in the LLM itself, not in the surrounding deterministic software logic.

output-column contract is a stricter evaluation metric in text-to-SQL than semantic correctness: it requires that the generated SQL not only returns logically correct data, but also exactly matches the expected baseline in terms of column names, column order, and column count. In real systems, this constraint directly determines whether downstream visualization components can render correctly — dashboard charts often rely on fixed column names for field binding. All 4 SQL tasks passed semantic criteria but only 2/4 met the column contract, indicating that the model is capable of understanding query intent but remains unstable when it comes to precisely aligning with interface specifications. This is one of the main reasons why additional post-processing or validation layers are needed when using current LLMs for production-grade SQL generation.

Honest Boundary Claims Without Exaggeration

The paper's authors show rare restraint in their conclusion, explicitly stating that these small-scale, DataBrain-specific results do not demonstrate production-level usability, general text-to-SQL accuracy, or efficiency advantages over manually built dashboards.

This honesty actually highlights the research's true value: Text2Dashboard's contribution lies not in pushing some accuracy metric higher, but in proposing a governed Agent architecture paradigm. In enterprise settings, a system that is "roughly correct but uncontrollable" is far less practical than one that is "limited in capability but fully auditable and reversible at every step."

Implications for Enterprise AI Deployment

Text2Dashboard's design philosophy offers a valuable engineering reference for the current wave of Agent application deployments:

• Separation of responsibilities: Let the LLM make probabilistic proposals; let deterministic code make critical execution decisions.
• Constraints first: Use schema constraints, typed tools, read-only SQL, and similar mechanisms to push security boundaries to the architectural level — not as an afterthought.
• Auditability: Approval, audit, checkpointing, and recovery Hooks make system behavior transparent and traceable for operations and compliance teams.

For teams looking to bring natural language analytics capabilities into internal data platforms, this "controlled Agent" approach may be more practically meaningful than chasing higher end-to-end accuracy. And as LLM inference costs continue to decline, the performance bottleneck that currently consumes 97% of runtime should gradually ease.