OpenAI Codex Hands-On Review: How Does It Compare to Claude Code and Gemini CLI?

OpenAI Codex Product Overview: Web and Command-Line Modes

OpenAI Codex, an AI-assisted programming tool with a similar positioning to products like Cursor, offers two distinctly different usage modes.

Technical Background: The Evolution of Codex OpenAI Codex first appeared in 2021 as the underlying model powering GitHub Copilot, fine-tuned on large amounts of public code data based on the GPT-3 architecture. The new generation of Codex deeply integrates more advanced model capabilities like GPT-4o, upgrading its positioning from a "code completion engine" to an "autonomous coding agent" capable of understanding natural language requirements, planning task steps, and autonomously executing multi-turn code generation. This transformation represents a paradigm shift in AI programming tools from "Copilot" to "Autopilot."

The first is the web version of Codex, which feels very similar to ChatGPT when opened — supporting text input and file uploads, with programming interactions happening through a chat window. One of its core features is connecting to projects via GitHub rather than opening local projects. Users can directly select repositories from their GitHub account, let Codex connect to the corresponding codebase, and then develop through conversation.

The Design Logic Behind GitHub Integration The Codex web version connects to remote repositories through GitHub OAuth rather than directly operating on local file systems like Cursor or VS Code plugins. There's a technical rationale behind this design: cloud sandboxed environments can isolate code execution risks while naturally supporting version control and multi-person collaboration. As the world's largest code hosting platform with over 100 million developers and 400 million repositories, GitHub's deep partnership between OpenAI and Microsoft (GitHub's parent company) also gives this integration strategic significance. However, this also means users must first host their projects on GitHub to use it, creating a barrier for purely local development scenarios.

The second is the command-line (CLI) version, which is very easy to install — just one command and a few minutes. The command-line version's programming experience is similar to Gemini CLI and Claude Code, with interactive programming happening in the terminal. However, for users without traditional programming experience, this approach isn't particularly user-friendly.

Test Task: Using AI to Convert a Novel into a Visual Novel Game

To test Codex's actual programming capabilities, the tester presented a task with considerable complexity: converting a novel into an interactive visual novel game. This task had previously been tested on both Claude Code and Gemini CLI, making it excellent for horizontal comparison.

Why Visual Novels Make a Good Benchmark Test A visual novel is an interactive game format that combines text narration, character sprites, background music, and branching storylines. Notable titles include Fate/stay night and Steins;Gate. From a technical implementation perspective, a complete visual novel engine needs to handle: a script parsing engine (converting dialogue scripts into game event sequences), character sprite rendering and expression switching, background layer management, audio playback control (BGM fade-in/fade-out, sound effect triggers), save/load systems, and state machine management for branching choices. This task presents considerable complexity for AI programming tools and serves as an excellent benchmark for testing multi-module coordinated generation capabilities.

Codex's Task Planning Capabilities

After receiving the requirements, Codex first entered a Syncing phase, displaying its entire thought process including creating the codebase, creating static pages, meeting the user's voice-over requirements, etc. After clarifying the overall workflow, Codex broke the task down into five steps:

1. Create a static webpage — build the basic page framework
2. Parse the novel text — convert the source text into a visual novel script
3. Create the visual novel UI — design the interactive interface
4. Generate art assets — including character portraits, background music, background images, sound effects, etc.
5. Generate a shareable link — deploy as an accessible online version

From the task breakdown perspective, Codex's planning capabilities are decent — the logic is clear and the steps are reasonable. But the real problems emerged during execution.

Code Execution: Severe Step-Skipping and Rough Output

Automation Control and Code Generation

During the execution phase, Codex displays the generated code in full, and users can set different levels of automation. The tester chose the manual approval mode, where each operation requires user confirmation before execution.

The Execution Architecture of AI Programming Agents Modern AI programming tools (such as Claude Code, Gemini CLI, Codex CLI) commonly adopt a "plan-execute-feedback" agent loop architecture (ReAct framework). During the planning phase, the model decomposes natural language requirements into executable subtasks; during the execution phase, it invokes tools like code generation, file read/write, and terminal commands; during the feedback phase, it self-corrects based on runtime results or error logs. Codex's "step-skipping" problem is essentially a misjudgment of dependency relationships between intermediate steps in the execution chain, or task state loss caused by context window limitations — a technical challenge shared by all current LLM-based programming agents.

The first step of static webpage creation was completed smoothly, and the second step also progressed normally. However, a clear problem emerged next: Codex directly skipped critical intermediate steps, including the generation of art assets (character portraits, background music, etc.), all of which were omitted, jumping straight to the final summary phase.

Even more disappointing, the final generated project consisted of only a single static HTML file with no backend whatsoever. The way to open it wasn't by starting a server, but by directly opening the HTML file in a browser.

The Importance of Front-End/Back-End Separation Architecture Front-end/back-end separation is the mainstream architectural paradigm in modern web development. The front end handles the user interface and interaction logic (HTML/CSS/JavaScript), while the back end handles data processing, business logic, and API services (Node.js, Python Flask, etc.). For a visual novel game, the back end typically handles save management, branching storyline data storage, audio resource serving, and other functions. Codex only generating a single static HTML file means all logic was compressed into the client side, which is not only architecturally unreasonable but also severely limits functional extensibility. The fact that Claude Code and Gemini CLI can automatically distinguish between front end and back end reflects a deeper understanding of engineering best practices.

By comparison, when testing the same task with Claude Code and Gemini CLI previously, despite also being command-line experiences, they at least distinguished between front-end and back-end architecture.

First Run Failed, Requiring Additional Debugging

The first generated code threw errors upon running and couldn't work properly. After the tester fed the browser error logs back to Codex, it began analyzing the problem and debugging. After one round of fixes, the code finally ran.

This shows that Codex's one-shot code generation quality is not high — the first output already contained errors requiring an additional debugging cycle. While bugs in AI programming tools are not uncommon, for a relatively simple static page project, this performance is indeed less than ideal.

Final Output Assessment: Severely Missing Features

The fixed project ultimately generated a directly accessible link, but the actual result was extremely bare-bones:

• Character portraits — not generated at all
• Background music and sound effects — missing
• Visual novel script — essentially just the original text pasted directly
• Speaker (character dialogue) labels — not handled at all
• Overall UI — extremely rough

In short, of the five steps Codex laid out during the task planning phase, only the first step (static webpage) was truly completed. The remaining steps were either skipped entirely or executed at extremely low quality. The final output was far from the expected "visual novel game."

OpenAI Codex vs. Claude Code and Gemini CLI: Horizontal Comparison

Comparing the OpenAI Codex command-line version with similar AI programming tools reveals significant gaps in execution quality and task completion.