L8 Principal Engineer in Action: Complete AI Agent Workflow from Zero to Production

Preface: A Real-World Full-Stack AI Agent Demo

"Kun," an L8 Senior Principal Engineer who previously worked at Meta, Microsoft, and Atlassian, published an autonomous engineering workflow video that racked up 600,000 views. By popular request, he followed up with a complete demonstration of how to use an AI Agent workflow to build a genuinely shippable iOS app — starting from a blank terminal.

The project came from a real need: his son Eddie needed an app to manage his allowance. The app had to track a virtual balance and carry educational value — helping kids understand balances, loans, credit limits, interest, and other basic financial concepts. It also needed separate "parent mode" and "child mode," with data stored in the cloud for multi-device sync.

The most striking aspect of the entire process: the developer wrote almost no code himself, barely reviewed most of it, and instead drove the whole project forward through continuous conversation with an AI Agent "first mate."

Core Methodology: The Captain and First Mate Collaboration Model

The author used his own open-source project FirstMate, whose core idea is building a hierarchical Agent collaboration system structured around a "captain and first mate" relationship. The developer plays the "captain" and only needs to communicate with a single Agent called the "first mate," who breaks tasks down and delegates them to different "crew" Agents running in parallel.

The key value of this design is dramatically reducing cognitive load. The author repeatedly emphasized that he doesn't need to constantly switch between multiple tasks or personally coordinate "which Agent needs guidance and which doesn't." The first mate handles all the complex task orchestration automatically.

Interestingly, the author chose "Calm Mode," which hides all the noise from tool calls — the terminal only shows his prompts and FirstMate's responses. His view: "With models from a year or two ago that weren't smart enough, it was useful to watch what the Agent was doing because they'd often go down the wrong path. But with today's models, I find myself needing to guide the agent less and less."

Not Chasing the Most Powerful Model

One counterintuitive choice: the author deliberately didn't use the most powerful GPT-5.6 High model, opting instead for the more efficient "Luna" model (GPT-5.6 Luna X High). He wanted to prove that you don't always need the best model for everything — you can still extract enormous value from more efficient models.

As it turned out, the only recommendation from Luna he disagreed with during the entire project was on the VPS cost decision — Luna suggested managed Supabase, which the author considered too expensive for MVP stage. Beyond that, Luna performed remarkably well.

FirstMate represents a concrete implementation of the "multi-Agent orchestration" paradigm. Unlike a single Agent executing tasks step by step, an orchestration framework allows a "coordinator" Agent to break down an overall goal into subtasks and distribute them to multiple "executor" Agents running in parallel. The core advantage of this architecture is throughput: research, design, and development steps that would otherwise run sequentially can proceed simultaneously, dramatically reducing total time. Engineering implementations of this kind must solve several key challenges: context isolation for subtasks (each Agent only sees what it needs), aggregation and conflict resolution of results, and unified permission management for external tool calls. OpenAI's Swarm, Microsoft's AutoGen, LangGraph, and others are open-source solutions tackling similar problems — FirstMate is the author's custom implementation tailored for engineering development scenarios.

From Requirements to Prototype: Market Research First

To kick off the project, the author fed the first mate roughly 3,000 characters of "stream-of-consciousness" input — everything he had in his head about requirements, ideas, and thoughts, all at once. This is his typical way of starting a new project.

He asked the first mate to handle two things in parallel:

• Market research: Confirm whether existing apps already cover these features. If so, just install one; if not, it's a unique gap worth filling.
• Technical research: How to build this complete tech stack end to end.

The first mate immediately spun up two parallel "scout" tabs to handle each. The author then further requested interactive HTML prototypes (rather than just scoped Markdown documents), focusing on wireframes, screen concepts, and user flows — not polished UI.

By actually interacting with the HTML prototype, the author caught quite a few details and gave feedback: for example, the currency should default to local currency (USD), the recent activity list had duplicate entries, and the flow for children requesting credits needed work. This feedback was ultimately compiled into a PRD (Product Requirements Document) and committed to the repository.

Design System: Building Consistency with Cloud Design

With product requirements clear, the author shifted to using a cloud design tool. He specifically emphasized creating a design system rather than individual designs — once a design system is established, building more UI components and screens later doesn't require reinventing every element from scratch, and consistency is guaranteed.

Interestingly, the author originally wanted to use a more powerful model for the design phase, but chose Sonnet in the spirit of frugality. The design system covered brand colors (gold, his son's favorite), icons, card designs, and more. His core feedback on the initial design was: "It doesn't feel like a kids' app" — he wanted it to feel more fun, friendly, and child-appropriate.

The app icon evolved from a piggy bank metaphor to an enlarged pig character, with icon thumbnails generated to meet iOS format requirements.

Security Practice: Managing Secrets with Atomic Vault

While setting up the backend, the author demonstrated a critical security practice — using Atomic Vault (an open-source project by Homebrew creator Max Howell) to manage secrets.

Whenever an Agent needs access to a Hetzner API token, Cloudflare API key, or other production credentials, an authorization dialog pops up, letting the author review and decide whether to approve. Tokens are passed as environment variables, and the Agent never sees the plaintext values.

The author's insight here was remarkably forward-looking: "As we delegate more and more things to Agents, this is going to become increasingly important. Otherwise, if you put all your keys in ENV files, you really have no idea when an Agent has already leaked them — maybe it's already published them somewhere else as part of a public API call."

The credential security problem for Agents is known in the industry as "credential exfiltration" risk, and it's one of the core security challenges AI Agents face on their path to production. The traditional approach is storing secrets in .env files, with Agents reading environment variables directly at runtime. The problem with this: the Agent holds plaintext credentials throughout the entire session, and once prompt injection introduces a malicious instruction — or the Agent accidentally writes secrets to logs or commits them to a repository — a leak occurs. Atomic Vault's solution is analogous to the operating system's "principle of least privilege" — secrets are only briefly exposed to the calling process when needed, and the Agent itself cannot persistently access plaintext values. OWASP's LLM Top 10 threat list includes "Insecure Plugin Design" and "Excessive Agency," both pointing to these kinds of risks — confirming that just-in-time authorization for secrets has become an important component of AI security best practices.

Tech Stack Decisions: Cost Control and Declarative Infrastructure

On infrastructure choices, the author demonstrated the judgment of a seasoned engineer. He vetoed managed Supabase and chose a low-cost Hetzner VPS, with costs well below the expected $30–60/month.

He also requested that infrastructure be managed as code using OpenTofu (an open-source alternative to Terraform) — machines and required resources all declared in a file that's auditable, committable, and source-controlled.

On the deployment environment, the author pushed back on the GPT-5.6 model's tendency to "over-engineer." The model originally wanted to deploy both development and production environments, but the author judged that for a brand-new app with no users yet, "there's really no need to set up different environment tiers" — a single production environment is sufficient. He admitted frankly: "These are the places where we humans still need to make some judgment calls."

OpenTofu is a community fork of Terraform, born in 2023 after HashiCorp changed Terraform to a non-open-source license (BSL). Hosted by the Linux Foundation, it maintains syntax compatibility with Terraform. The core value of "Infrastructure as Code" (IaC) is managing the configuration of servers, networks, DNS, and other resources as declarative files: you describe the "desired state," and the tool calculates and executes the steps to get from the "current state" to the "desired state." This is especially important for AI Agent workflows — Agents can directly modify .tf files and run tofu apply, leaving the entire infrastructure change process versioned, reviewable, and rollback-capable, avoiding the hard-to-trace problem of Agents directly manipulating cloud resources via the command line.

Workflow Governance: No Faults Mode and Compaction

The project evolved from a "push directly" to a "PR-based" workflow. The author proposed a practical rule of thumb for when to enable No Faults Mode:

"If this is a code change that another person will manually review, you should enable No Faults Mode; if you're working solo and don't want anyone to review it, then you should enable it (to set up local validation). Since the code isn't taking shape yet in the prototype stage, I'm not enabling it now; once we reach MVP, I'll switch to a stricter approval process."

On context compaction, the author shared an important observation. GPT models have a relatively short default context window of 272k, and compaction triggers automatically at around 80% capacity. He recommended the openai-server-compaction plugin — letting the PI Agent call OpenAI's server-side compaction when running GPT models, which works best for long-running tasks. The entire project went through four automatic compressions with almost no noticeable performance degradation. His conclusion: "Compaction has really become an implementation detail that most people don't need to worry about anymore."

Context compaction is the key mechanism that allows large language models to maintain coherence over long tasks. A model's context window — the total token limit it can process in a single session — ranges from roughly 128k to 272k tokens for the GPT series. When a coding task involves extensive file read/write operations, multiple rounds of conversation, and tool call records, the context can quickly approach the limit. The core idea of compaction is to replace parts of the conversation history that no longer need precise reference with a summary, freeing up token space. The advantage of server-side compaction (like openai-server-compaction) over client-side truncation is that the compression logic is optimized by the model provider, which can more intelligently preserve key code snippets, decision context, and current task state — rather than mechanically discarding the earliest messages. Compaction quality directly affects whether an Agent can still "remember" early architectural decisions and user preferences after a long-running task.

End-to-End Testing and MVP Delivery

As the project neared completion, the author refused to accept the Agent's verbal assurances that "everything is working" and insisted on real evidence. He explicitly instructed: "Unless there's something only I can handle, don't ask me — you should try to do end-to-end testing and see if you can find real bugs."

The final MVP acceptance criteria were clear and concrete: log in with a real Apple account in the iOS Simulator, connected to the real backend deployed on the VPS (via a Cloudflare-configured subdomain ediswallet.conchaingate.com with HTTPS enabled).

The test succeeded: after login worked, the parent mode added $10, and the child view synced to show $10; after logging out and logging back in with a different account, the cloud data was correctly retrieved. The full-stack flow ran end-to-end. The author planned to upload the app to TestFlight so his son could use it on his iPad the very next day.

Conclusion: A Maturity Signal for AI Agent Engineering

This demonstration carries several signals worth paying attention to:

First, AI Agent autonomy has reached a practical level. The developer's role has shifted from "coder" and "guide" to "decision-maker" and "reviewer" — the main work is providing requirements, making key judgment calls, and setting quality gates.

Second, multi-Agent parallel orchestration has become key to efficiency. Through the first mate's task delegation mechanism, market research, technical research, frontend/backend development, and infrastructure configuration could all proceed in parallel.

Third, security and governance mechanisms are becoming indispensable. Secret authorization management, No Faults Mode, PR workflows — these are all necessary guardrails for putting production-grade Agent workflows into practice.

As the author noted, even without using the most powerful models (cloud design used Sonnet rather than Opus, and the main flow used Luna), he still delivered a genuinely shippable full-stack application. That may be the best proof yet that AI-assisted development is reaching maturity.