OpenAI's New Agent Stack: Computer Use, Decisions API, and Ultra-Fast Workflows

Title

OpenAI's New Agent Stack: Computer Use, Decisions API, and Ultra-Fast Workflows

Meta Description

OpenAI Dev Day unveiled a new agent stack: Computer Use operates faster than humans, Decisions API delivers ultra-low-latency classification based on Luna, Agents API opens up the same Computer Use capabilities, and GPT-6.1 Sol costs only one-seventh of Astra. This article covers the core updates, benchmark data, and key takeaways for developers.

Article Content

At its recent Dev Day, OpenAI released a series of major updates centered on agents—from Computer Use, which can directly operate a computer, to the low-latency-focused Decisions API, to the brand-new Agents API and GPT-6.1 Sol model. This is not just a stack of new features; it is a complete technology stack that lets AI truly "get hands-on and do the work." Based on publicly available podcast interviews, this article outlines the core logic of this new stack, its real-world performance, and key takeaways for developers.

From "Can Start" to "Can Finish": The Qualitative Leap of Computer Use

Computer Use is not a new concept, but for a long time agents could start tasks yet often got stuck on pop-ups, loading delays, or changes in page structure. Ari, who leads product and engineering for OpenAI's computer-operating agents, admitted in the interview that early models "could reliably start a task, but then would run into problems." The biggest change now is debugging capability: the model has learned to retry, self-check, and figure out "what went wrong."

On the technical side, Computer Use no longer relies on a single action loop. It writes JavaScript code directly and executes multi-step operations in bulk within the browser environment; at the same time, it fuses multiple modalities such as screenshots, the accessibility tree, and Playwright. Expanding tool calls in Codex reveals that the model often generates and executes a script in one go, rather than mechanically "click, screenshot, click again." This "write code to operate the computer" approach dramatically reduces round-trip latency.

Another easily overlooked update is App Shots. When a user double-taps the Command key, it looks like just a screenshot, but it actually pulls the entire app's metadata, clickable elements, and full text representation. Ari emphasized that information such as the destination URL of a link in a screenshot or an event title truncated in a calendar is complete in an App Shot. For language models, this is an upgrade from "looking at an image" to "reading structure," and token efficiency has been specifically optimized.

Accessibility Tree is the interface semantic structure that an operating system or browser builds for assistive technologies (such as screen readers). It exposes elements on a page—buttons, input fields, links, etc.—as hierarchical nodes with attributes such as role, name, and state. Compared with pixel screenshots, the accessibility tree directly provides structured information about "what this element is, whether it can be clicked, and whether it is currently available." For agents, reading the accessibility tree is more token-efficient and less ambiguous than parsing images, and it can still reliably locate elements when they are obscured or dynamically loaded. However, the accessibility tree is not a panacea: some custom controls, Canvas-rendered content, or cross-iframe scenarios may lack information, so in practice it is often used in combination with screenshots, DOM queries, and other methods.

Decisions API: Not a New Model, but Luna Squeezed to the Limit

The Decisions API is one of the most talked-about products of this release. It focuses on ultra-low-latency classification and decision reasoning, and is seen by outsiders as a response to emerging inference services such as Jeff. But Nikunj, who leads product for the OpenAI API, clarified a key fact: the Decisions API did not train a new model. Instead, it builds on the existing Luna weights and, through structured output, parallel inference, and inference-stack optimization, pushes time-to-first-token (TTFT) to the limit.

"We basically did this zero-shot on Luna, shipped it first, and then iterated based on feedback," Nikunj said. This means developers can get an endpoint deeply optimized for classification tasks at a price close to Luna. It supports visual input and can process multiple questions in parallel, making it suitable for scenarios such as customer support ticket classification, real-time routing, and low-latency evaluation. However, it does not have long-horizon reasoning capability; complex tasks still need to be handed to Astra or GPT-6.1 Sol.

Interestingly, OpenAI internally drew clear inspiration from Jeff's launch. Nikunj described the team as having been "nerd sniped"; engineers on the infrastructure and inference teams built a prototype within a week and then began ramping up latency optimization. This rapid, hacker-culture-driven follow-up also explains why the Decisions API could come to market in such a short time.

Time to First Token (TTFT/TTFD) refers to the time from when a request is sent to when the first output token is received from the model. It is one of the core metrics for measuring the responsiveness of interactive AI. For tasks such as classification, routing, and real-time decision-making, users or downstream systems often only need a short result, so time to first token almost determines the overall experience rather than total generation time. Reducing this metric usually depends on several paths: reducing input preprocessing overhead, reusing KV caches, executing candidate inference in parallel, optimizing scheduling and batching strategies, and using smaller model weights. What the Decisions API claims—"pushing latency to the limit"—is essentially a systematic trade-off across these engineering steps rather than a leap in model capability itself.

GPT-6.1 Sol: A Cost-Effective Model Built for Computer Operation

GPT-6.1 Sol is positioned as a model "particularly well suited for computer operation." The official data given in the keynote is that its cost is about one-fifth of Astra, and in computer-operation scenarios only one-seventh of Astra. This cost advantage comes from continuous optimization of inference efficiency, not simply from a shrunken model.

Ari shared a concrete efficiency comparison: in a task involving subscribing to a customized meal service, manual operation took two hours, while Computer Use on GPT-6.1 Sol completed it in 15 minutes—about eight times faster than a human. Tasks like this—highly repetitive, highly customized, and with little API support—are exactly where Computer Use can deliver the most value.

Model acceleration and toolchain improvements go hand in hand. Internally, OpenAI uses multiple test frameworks and configurations to measure computer-operation capability, including production-grade configurations with safety constraints. Although the measurement approach is complex, the conclusion is consistent: the gains come both from the model itself and from optimization of the harness (execution framework).

Agents API: Opening the Same Computer Use to Developers

For developers, built-in Computer Use in the Agents API may be the most practically significant update. In the past, building a reliable computer-operating agent required developing your own harness, handling screenshot parsing, retry logic, and safety boundaries—an extremely high barrier. Now developers can directly call the same Computer Use implementation used by Codex and ChatGPT.

Ari gave a pragmatic reason: OpenAI's models are trained on this harness, so using the official implementation may offer advantages in speed, cost, and accuracy. Of course, if developers have special requirements, building their own harness is still feasible, but they need to bear the corresponding engineering complexity.

On the safety side, Ari recommends that developers follow a few principles: grant only the website and app access necessary for the task; request user confirmation before performing critical operations such as payments; and gradually build user trust through reliability by design. He used his own experience of "having an agent configure DNS and pay bills" to illustrate that trust accumulates gradually as the product matures.

Harness (execution framework) refers to the engineering layer wrapped around the model that drives the agent to complete tasks: it determines how prompts are organized, how tools are called, how screenshots or structured outputs are parsed, how failures are retried, when to terminate, and how safety constraints are applied. The same model can perform very differently under different harnesses, because the rhythm of tool calls, context management, and error-recovery strategies all affect the final success rate. OpenAI emphasizes that its models are trained and evaluated on its own harness, which means the official implementation closely matches the model's behavior distribution. Although building your own harness is flexible, it requires reproducing these implicit conventions yourself and bearing the corresponding debugging and reliability costs.

Developer Workflows Are Being Rewritten

The impact of Computer Use on development processes may be more direct than imagined. One of Ari's favorite use cases is having an agent test the software he has just written. In the traditional workflow, after Codex generates code, the developer has to manually act as QA; now the agent can build and test on its own and hand a runnable artifact to humans. This completes the final link in the software development lifecycle.

Another interviewee, Chase, who is also a content creator, mentioned that many features on platforms such as YouTube do not expose APIs, making Computer Use a practical solution for automating community posts, A/B testing, and similar operations. He also has the agent call sub-agents to research alternatives while waiting for customer support replies, forming a hybrid workflow in which "humans set strategy and agents execute in parallel."

Infrastructure on the API side is also evolving in parallel. Nikunj introduced asynchronous tool calls, mid-turn steering, and bidirectional communication based on WebSocket brought by GPT-6. Asynchronous tool calls allow the model to continue reasoning while a tool is executing, without blocking and waiting; mid-turn steering lets developers inject new instructions while the model is thinking. Combined with UltraFast inference mode, these capabilities provide underlying support for agent applications with high real-time requirements.

Caching, Compression, and Performance: The Invisible Battle at the Platform Layer

For building long-running personal agents, performance optimization of the Responses API is equally critical. Nikunj revealed that the team is rewriting the Responses API to reduce time to first token and is investing more in caching: it currently guarantees cache hits within 30 minutes, and a 12-hour cache window preview has been rolled out to some users. In addition, a pre-warming feature allows developers to write to the cache in advance and prepare context for subsequent requests.

For context compression, OpenAI offers both automatic server-side compression and manual /compact. Codex's harness has also experimented with a file-based compression system. For agent products that need to maintain session state over long periods, these mechanisms directly affect cost and response speed.

Context Compression addresses the tension between a limited token budget and continuously growing information in long sessions. Common approaches include summarizing historical conversations, discarding low-value turns, externalizing structured state to files or databases, and automatically triggering compression on the server side. Compression strategy directly affects the quality of an agent's "memory"—compressing too aggressively loses key constraints, while compressing too loosely raises cost and latency. The server-side automatic compression, manual /compact, and file-based compression experiments mentioned in the article represent several routes ranging from fully automatic to semi-controllable. Developers need to choose the appropriate granularity based on how much a task depends on history.

Final Thoughts: How Far Are We from Agents' "Superhuman Operation"?

Ari offered a clear judgment in the interview: Computer Use is already faster than an average person on most tasks, and the next frontier is surpassing the operation speed of expert computer users. When agents operate software at or above the speed of skilled humans, real-time products will open up new possibilities, and people will gradually default to delegating more manual operations to agents.

Bottlenecks remain: model capability, inference speed, harness design, interface representation, and even the loading time of the target website itself. External factors such as how fast a DoorDash page loads will all become part of end-to-end latency. But the direction is clear—let agents use all software designed for humans just like people do, while being faster, more stable, and more delegable than humans. For developers, now is exactly the time to start building with the Agents API and Decisions API.

One-line Summary

OpenAI Dev Day released an agent technology stack centered on Computer Use, covering models, APIs, and platform optimization.

Paragraph Summary

Based on podcast interviews, this article outlines OpenAI Dev Day's series of agent-focused updates. Computer Use has achieved a qualitative leap: the model has learned to debug and retry and improves efficiency by writing JavaScript and fusing screenshots with the accessibility tree; App Shots let the model read structured app information rather than just looking at images. The Decisions API is not a new model but an inference-stack optimization based on Luna weights, focused on ultra-low time to first token and suited to classification and real-time decision-making. GPT-6.1 Sol is optimized for computer operation and costs about one-seventh of Astra. The Agents API opens the same Computer Use implementation to developers and offers safety recommendations such as least privilege. At the platform layer, OpenAI continues to invest in caching, context compression, and rewriting the Responses API.