Codex Screen Recording to Skills: Demo Once, Automate Forever

OpenAI Codex recently launched a new feature called Record and Replay. The core concept is straightforward: you perform a workflow on your Mac, Codex "watches and learns" alongside you, then automatically generates a reusable skill file that you can invoke whenever needed. No prompt writing, no step-by-step descriptions—just show the AI what to do. This might be the lowest-barrier AI automation solution available today.

Why Do We Need Record and Replay?

In daily work, many of us are tormented by repetitive tasks. Here are three of the most common pain points:

First, repetitive tasks are exhausting. Publishing videos every week, filing expense reports, creating tickets—the same steps repeated dozens of times, and one slip-up means filling in the wrong field.

Second, prompts keep getting longer. To get AI to help, you have to spell out every single step—which button to click, which field to fill, where to hit save—writing it all out is more tiring than just doing it yourself.

Third, personal preferences are hard to articulate. Naming conventions, field ordering, review processes—you know your habits by heart, but putting them into words is a headache, and AI keeps misunderstanding.

At the end of the day, what you can explain verbally isn't always easy to write down, and showing is ten times faster than describing. Record and Replay was designed based on exactly this insight.

How Codex Record and Replay Works

Record and Replay is a new feature in OpenAI Codex. The entire process breaks down into three core steps:

Step 1: Record

Open the Codex plugin panel, select Record Skill, approve screen recording permissions, then perform your workflow as you normally would. You don't need to do anything extra—just work as usual.

Step 2: AI Learning and Skill Generation (Learn)

Codex observes every action you take and every window's content. After recording ends, it automatically analyzes the entire workflow and generates a SKILL.md skill file. This file contains four key pieces of information:

• When to use this workflow
• What input parameters are needed
• What the specific execution steps are
• How to verify the results

The technical core here is the Computer Use capability—a key technology that OpenAI has been gradually rolling out between 2024 and 2025. It allows AI models to operate a computer just like a human: moving the mouse, clicking buttons, typing text, and reading screen content. The underlying technology relies on the visual understanding capabilities of multimodal large models: the model captures screenshots to obtain current interface information, identifies the position and function of UI elements, then generates corresponding operation commands. Anthropic's Claude also launched a similar Computer Use feature during the same period. The core challenge of this type of technology lies in enabling AI to accurately understand complex graphical interfaces and maintain operational stability across different applications and resolutions.

Step 3: Automatic Replay

Start a new conversation, have Codex invoke the skill, pass in the parameters that vary each time (such as this episode's video title, data file path, etc.), and it automatically completes the entire workflow.

Summed up in one phrase: seeing replaces writing. Instead of writing out every step, just show the AI directly.

Hands-On Demo: Complete Recording Workflow Walkthrough

Using YouTube video publishing as an example, here's the complete workflow:

1. Open Codex, switch to the Plugins panel, and confirm that both Computer Use and Record & Replay are ready
2. Click Record Skill to start recording
3. Before recording begins, Codex asks you to briefly describe what you're about to demonstrate (e.g., "I'm going to show how to publish a YouTube video—copying metadata from a spreadsheet, uploading a thumbnail, adding subtitles, then saving as a private upload")
4. Start your normal operations; Codex tracks every step in real time
5. When the workflow is complete, click the Stop Recording button to end the recording

During recording, Codex displays progress in real time, such as "Step 1: Open the publishing spreadsheet—done" and "Step 2: Copy title and description—done," so you know exactly which operations the AI is tracking.

SKILL.md Skill Files: Readable, Editable, Customizable

After recording ends, the SKILL.md file automatically generated by Codex is in plain text format—and this is critically important.

SKILL.md uses plain-text Markdown format to describe automation workflows, a design choice with deep technical reasoning behind it. Traditional RPA (Robotic Process Automation) tools like UiPath and Automation Anywhere typically store recorded operations as structured XML or JSON configuration files that are difficult for ordinary users to read and modify. SKILL.md, on the other hand, uses natural language to describe each step. It's essentially an "executable document"—humans can read and modify it, and AI can parse and execute it. This approach falls under the computer science concept of Programming by Demonstration, where programs are automatically generated by observing user actions. The idea dates back to academic research in the 1980s, but it didn't become truly practical until the emergence of large language models.

You can perform secondary edits on the generated skill file:

• Modify step descriptions: Make the workflow more precise
• Add naming conventions: Standardize file naming rules
• Add edge case handling: For example, "If the thumbnail doesn't exist, skip the upload"
• Add prohibition rules: For example, "Never set the video to public"

After editing, Codex will follow your adjusted version for all future executions. This means the skill file isn't a one-time black box—it's a continuously iterable automation script, just written in natural language.

Prerequisites and Installation Setup

Before using Record and Replay, confirm the following requirements:

System Requirements:

• Currently macOS only; you need to install the Codex desktop app first
• Install the Computer Use plugin in Codex settings
• Grant screen recording and accessibility permissions (in System Settings → Privacy & Security)

The permissions Codex requires involve macOS's Accessibility API framework. The macOS Accessibility API allows third-party applications to read the UI element hierarchy on screen, retrieve text content, and simulate keyboard and mouse operations. For security reasons, Apple requires users to manually grant authorization in "System Settings → Privacy & Security" to prevent malicious software from controlling the computer without the user's knowledge. Screen recording permission allows the app to capture the screen, which is the foundation for Codex "observing" user operations. The current macOS-only support (without Windows) is likely related to the maturity and consistency of the macOS Accessibility API—while Windows' UI Automation framework offers similar functionality, implementation quality varies significantly across different applications.

Regional Restrictions:

• The initial launch does not include the European Economic Area, the UK, or Switzerland
• Enterprise users need to confirm that Computer Use is enabled in requirements.toml

The installation steps are simple: Open Codex → Plugins → Install the Computer Use plugin → Grant permissions in System Settings → Confirm the plugin status shows "Enabled."

Use Cases and Considerations

What Tasks Are Suitable for Codex Automation?

Any operation you repeatedly perform on your computer can be recorded once and automated by Codex going forward:

• Publishing videos (YouTube, Bilibili, and other platforms)
• Filing expense reports, creating tickets
• Downloading reports, organizing data
• Submitting Pull Requests
• Scheduling meetings, booking workspaces

Most importantly, you don't need a programming background or complex prompts. If you can do it once, Codex can learn to do it.

The Technical Evolution from RPA to AI Agents

The emergence of Record and Replay marks an important evolution in automation technology from traditional RPA to AI Agents. Traditional RPA relies on precise UI element positioning (such as XPath and CSS selectors), and is prone to breaking whenever the interface layout changes, resulting in extremely high maintenance costs. Solutions based on large model visual understanding offer greater robustness—even if button positions move or the interface is redesigned, AI can still find the correct operation target through semantic understanding. This is why Codex's skill files describe steps in natural language rather than coordinates. For enterprise users, this means automation workflow maintenance costs will drop significantly, eliminating the need for dedicated RPA development teams to handle every UI change.

Privacy, Security, and Recording Tips

Regarding privacy and security: During recording, Codex can see your screen content, so do not enter passwords, keys, or other sensitive information. Permissions can be revoked at any time in System Settings.

Recording tips:

• Don't operate too quickly—give the AI enough time to observe
• Keep the workflow focused; only perform the task you want Codex to learn, and avoid unrelated actions
• Split different workflows into separate skills; don't mix them together
• Maximum recording length is 30 minutes, which is more than enough for normal workflows
• After recording ends, be sure to review the generated SKILL.md and add your preference settings

Conclusion

Record and Replay is a significant upgrade for Codex in the automation space. It lowers the barrier from "writing prompts" to "showing the AI once"—you don't need to precisely describe every step, you don't need to worry about preferences being lost in translation, you just need to do your familiar task once and leave the rest to Codex.

From a technological evolution perspective, this represents an important trend in human-computer interaction: shifting from "telling AI what to do" to "showing AI what to do." When AI can learn tasks by observing screen operations, the barrier to automation drops dramatically, truly achieving "record once, use forever."

Currently, this feature still has platform limitations (macOS only) and regional restrictions, but as a directional product, it has already demonstrated significant potential. For users plagued by repetitive tasks every day, this may be one of the most worthwhile AI automation tools to try right now.