The Right Approach to Agent-Native Apps: Shared Action Layers Over UI Puppeteering

From UI Puppeteering to Shared Action Layers

As AI agents become deeply involved in application interactions, a long-overlooked architectural question has come to the surface: how should an agent actually "operate" an application? Many past approaches took the route of UI Puppeteering — having agents simulate human behavior by clicking buttons, filling out forms, and reading screens. While this seems intuitive, it's inherently fragile: a minor UI change can break scripts instantly, and every interaction must pass through a rendering layer, making the whole process inefficient and hard to debug.

BuilderIO's Agent-Native framework was designed specifically to address this pain point. Its core thesis can be summed up in one sentence: Agent-Native applications need Shared Actions, not UI Puppeteering. This idea is gaining traction in the developer community because it decouples "what an agent can do" from the UI layer, abstracting it into a unified, callable action contract.

One Action Layer, Six Types of Callers

The key design principle of the Agent-Native framework is to define a single Action Layer shared across multiple entry points. According to BuilderIO, these entry points include:

• User Interface (UI): Actions triggered by human users through the interface;
• AI Agent: Actions invoked after an agent makes an autonomous decision;
• HTTP: Traditional API requests;
• MCP (Model Context Protocol): A protocol providing context and tool-calling capabilities for large language models;
• A2A (Agent-to-Agent): Collaborative communication between agents;
• CLI: Command-line invocations.

This means that whether a request originates from a user clicking a button, an agent executing autonomously, an external HTTP call, or a command-line script, it all converges on the same action definition. Developers only need to maintain a single set of action logic, allowing human-to-machine, machine-to-machine, and scripted interactions to share consistent behavior and permission boundaries.

Why This Is Better Than UI Puppeteering

UI Puppeteering essentially makes agents "pretend to be human" — driving the interface through visual recognition and coordinate-based clicking. The fundamental problem is that it treats the UI as an API, when the UI was never designed for programmatic invocation. This approach carries three significant costs: fragility (a UI redesign renders it broken), unreliability (asynchronous rendering and element positioning drift cause failures), and poor auditability (it's impossible to clearly log "what the agent actually did").

The Shared Action Layer takes the opposite approach: it explicitly declares executable operations as first-class citizens. Agents no longer need to "read the screen and guess which button to click" — they call named actions directly. The UI itself becomes just one consumer of the action layer; when a button is clicked, it triggers the exact same underlying action. This ensures consistency between human interaction and agent behavior, while making every invocation naturally traceable and authorizable.

---

MCP (Model Context Protocol) is an open protocol led by Anthropic, designed to standardize how large language models interact with external tools and data sources. Its core idea is to provide models with a unified "tool-calling" interface, enabling them to request specific operations or retrieve contextual information in a structured way — without requiring custom integrations for each individual tool. A2A (Agent-to-Agent), proposed by Google, focuses on collaborative communication between different AI agents, defining standard formats for how agents discover each other's capabilities, delegate tasks, and pass results. The emergence of both protocols signals that AI applications are evolving from "a single model calling tools" to "multi-agent collaborative networks" — and it's precisely in this context that the value of a shared action layer becomes clear. The same set of action definitions can be exposed to large models via MCP and accept delegated calls from other agents via A2A, without requiring separate business logic implementations for each protocol.

UI Puppeteering is exemplified by automation frameworks like Selenium and Playwright, as well as the more recently emerged Computer Use capabilities designed specifically for AI agents (such as Anthropic Claude's screen control feature). These approaches drive applications by taking screenshots to identify UI elements and simulating mouse and keyboard events. Their advantage is that they require no modifications to the target application and can theoretically control any graphical interface. However, in production-grade agent scenarios, their shortcomings are equally apparent: even minor changes to DOM structure or visual layout can cause element targeting to fail; asynchronous rendering, overlapping popups, and loading delays introduce timing races; and most critically, this type of control bypasses the application's business logic layer, making permission enforcement and operation auditing extremely difficult. A shared action layer fundamentally elevates "programmatically callable capabilities" from the visual presentation layer to the semantic layer, ensuring every agent invocation has a clear function signature, parameter constraints, and permission declarations.

Implications for Agent Application Development

This approach carries direct engineering guidance for teams building Agent-Native applications.

Design actions first, then design the interface. Traditional development often starts with the UI, with action logic scattered across event handlers. In the Agent-Native paradigm, the right approach is to first abstract the application's core capabilities into a set of well-defined actions, with UI, agents, and APIs serving merely as different trigger channels for those actions.

Consolidate permissions and validation under a unified contract. When all entry points share the same action layer, parameter validation, access control, and audit logging can all be implemented centrally at the action definition level — avoiding duplicated and inconsistent logic spread across UI, API, and agent code paths.

Prepare for the multi-protocol era. The rise of protocols like MCP and A2A signals that future applications won't only serve human users — they'll also serve large language models and other agents. Building a shared action layer upfront is equivalent to reserving standard interfaces for these new types of callers, rather than having to rewrite business logic every time a new protocol needs to be integrated.

An Architectural Direction Worth Watching

It's worth noting that the source material for this article is limited in scope, primarily presenting BuilderIO's core framework philosophy rather than full technical implementation details. That said, the thesis itself — "shared actions over UI puppeteering" — already touches on a critical fork in the road for agent application architecture.

As more and more applications seek to be "natively" driven by agents, the choice facing developers will become increasingly clear: continue patching over fragile UI puppeteering, or fundamentally rebuild around a human-and-machine-compatible action contract. BuilderIO's Agent-Native framework represents a pragmatic answer of the latter kind — and it's well worth tracking for developers following the agent ecosystem.