Building an AI Coding Agent from Scratch: A Vibe Coding Livestream in Review

An Ambitious Starting Point: A "Iron Man-Style" Coding Assistant to Rival Cursor

This was a build-from-scratch development livestream, and the creator set a bold goal — to create a coding agent called "Mayday." As he described it, Mayday isn't a simple Cursor clone, but rather draws inspiration from Jarvis in the Iron Man films, with the aim of building "something better than Cursor."

The most compelling part of the vision was the interaction model. He planned to let users "call" the model via their phone — and have the model call them back: when something goes wrong in the code, the Agent phones the developer, reads out the available options, and once the developer chooses, the task continues executing remotely. This voice-driven, human-like coding assistant concept carves out a clear differentiation from the generic IDE tools on the market. He was explicit about not wanting to build a catch-all tool that "throws in every model," but rather to reach a higher level of quality in a specific use case.

The entire project was built entirely on the Claude Code Max plan. At the start of the stream, his usage showed "100 out of 100" — 20x quota — providing ample headroom for extended, high-intensity Vibe Coding sessions.

Vibe Coding is a concept introduced by Andrej Karpathy in early 2025 that quickly went viral. It describes a workflow where developers rely entirely on AI to generate code, while they focus only on stating requirements, reviewing results, and giving feedback — barely writing or deeply reading any code themselves. The core idea is "if it feels right, keep pushing forward," as opposed to the traditional approach of understanding and debugging line by line. It lowers the technical barrier to programming, but introduces new challenges: when AI-generated code has deep bugs, developers unfamiliar with the underlying logic often struggle to pinpoint the root cause. The Claude Code Max plan is Anthropic's subscription offering for power users, providing higher API call quotas and rate limits, designed specifically for long, intensive AI-assisted coding sessions. The 20x quota means users can make far more model requests in a single day than the standard tier without hitting rate limits.

The Practical Reality of Model Selection: Claude Leads, Codex Falls Behind

Throughout the stream, the creator repeatedly discussed the competitive landscape among today's leading coding models — a section worth paying attention to for developers tracking AI coding tools.

His core take: Anthropic's Claude is currently in a clear lead. He had previously relied mainly on Codex (OpenAI's offering), but as newer models rolled out, Claude became his preference. He cited a point that Claude is "twice as smart" on benchmarks, and repeatedly remarked that Claude has been "popping off" lately. That said, he acknowledged that Claude models — especially the Opus series — are noticeably slower than Codex: "better output, but much slower."

For cost-sensitive tasks, he shared an interesting practical strategy: use cheaper, faster models for the grunt work. He repeatedly mentioned the "Luna" model, noting it costs around 10 cents per million tokens — "fast, cheap, and gets the job done" — well suited to serve as the "AI brain" inside an app. His principle was blunt: "Unless you're building a very secure backend, just use whatever works." When someone mocked him for using budget models, his response was simple: if it works, it works.

As for OpenAI, his assessment was on the negative side. He felt they need to close a significant gap on intelligence while also bringing down usage pricing — "if the output is still this bad and the models are weaker, it'll never work."

Technical Details in Practice: Parallel Terminals and Permission Management

The most revealing part of the stream was how it exposed the real pain points of AI-assisted coding in complex scenarios.

The creator wanted Mayday to run multiple Claude Code terminals (Agents) simultaneously, enabling parallel development. He requested "work tree isolation" to prevent conflicts when multiple Agents edit the same file. Reality didn't cooperate — he kept running into terminals freezing. After investigation, Claude's diagnosis was this: file locks (claims) held by an agent are only released when the stop hook fires or the session ends normally. Processes that are interrupted or force-killed trigger neither, leaving file locks permanently stuck and blocking other Agents from proceeding.

Permission management was another recurring pain point. Even with "auto mode" enabled, the Agent kept popping up approval prompts — prompting him to ask, "why is it still asking me if auto mode is on?" His solution was an explicit directive: unless an operation is genuinely high-risk and could affect other terminals or damage the project, execute automatically without asking.

There was also an interaction design flaw he jokingly called a "flash bang" — when an Agent needed user approval, a red alert would cover the entire screen rather than just flagging the relevant terminal, severely disrupting the workflow. He ultimately changed it so only the border of the terminal requiring action would flash in a dim red.

Work Tree Isolation is derived from Git's git worktree feature, which allows the same repository to check out different branches simultaneously in multiple independent directories, each with its own working state. In multi-Agent parallel development, each Agent operates its own worktree, avoiding direct write conflicts on shared files. However, the file lock mechanism introduces a new problem: an Agent holds a lock while editing a file, and releases it via the stop hook on a clean exit. If the process is force-killed, the hook never runs, and the lock becomes a "zombie lock" that permanently occupies the file — blocking other Agents from acquiring write access and leaving them stuck waiting. This is a classic "unreleased resource" problem in concurrent multi-process programming. In AI Agent frameworks, it's especially prone to occurring because process lifecycles are difficult to control precisely, and typically requires heartbeat detection or timeout-based auto-release mechanisms to mitigate.

Experimenting with Voice Input and AI Video Generation

Beyond the core coding functionality, the creator also integrated speech-to-text (STT/dictation) capabilities into Mayday, hoping to dictate prompts directly using a hotkey (such as holding Control). Debugging this feature was an ordeal — poor recognition accuracy, sluggish response times, and hotkey conflicts cropped up repeatedly, with him testing "testing testing" over a dozen times. He also mentioned the third-party tool Whisper Flow but preferred to build his own STT system to avoid paid dependencies.

An unexpected highlight was AI video generation. He used code to generate a product teaser trailer, calling it "visually insane" — with only the music falling slightly short. He also mentioned the Hyperframes tool, calling it "the coolest tool ever made." These experiments show how today's AI toolchain already allows a solo developer to rapidly produce output across coding, design, and video in a single workflow.

Whisper is OpenAI's open-source speech recognition model, known for its high accuracy and multilingual support. Whisper Flow is a third-party desktop app built on Whisper, optimized for macOS, that supports system-wide hotkey triggering and real-time transcription with results inserted directly into the current input focus — making it one of the more popular "hands-free" input solutions among developers. The creator's preference for building his own STT rather than using Whisper Flow came down to avoiding third-party paid dependencies and the privacy risk of data passing through an external service, while also allowing tighter integration with Mayday's hotkey system and reducing context-switching friction between apps. The main value of voice-to-text in a coding context is allowing developers to quickly dictate long prompts without breaking their train of thought — especially useful when describing complex requirements or context, where typing speed itself becomes a bottleneck.

A Genuine Snapshot of "Vibe Coding" in Action

The most valuable thing about this stream wasn't the completeness of the final product — in fact, by the time it ended, Mayday was still in early stages, with the core vision of remote calling and voice interaction yet to be realized. Its value lies in the honest portrayal of what "Vibe Coding" actually looks and feels like: the developer acts more like a conductor, issuing natural-language directives to multiple AI Agents, and constantly mediating as they freeze up, ask repetitive questions, and misinterpret instructions.

The creator chose to end the stream because he needed to configure the real application and handle sensitive information — not suitable to do on camera for risk of exposure. The whole session both demonstrated the enormous potential of AI coding agents and honestly surfaced unresolved engineering challenges in multi-Agent coordination, permission control, and state management. For anyone who wants to understand what AI-assisted programming actually feels like in practice, this is a rare first-hand record.