Building an AI Teammate from Scratch with Claude Code: A Grok Bot Clone in Practice

From xAI's Grok Bot to Meta's Muse and OpenAI's Dots, AI teammates are becoming a new arena that big tech is racing to stake out. But long before these products appeared, the open-source project OpenClaw had already proven that AI could book flights and send emails on your behalf. It was just too geared toward technical power users, and the model capabilities weren't mature enough yet. Now, a Bilibili creator has used Claude Code to fully replicate a production-deployable AI teammate system—from the interface to the sandbox, from memory to billing—demonstrating an engineering methodology for "building AI products with AI" end to end.

Why AI Teammates Suddenly Took Off

Several companies zeroed in on this direction almost simultaneously, and the reason isn't complicated. OpenClaw^Ad proved that AI could operate a real computer to complete tasks, but its initial barrier to entry was too high for ordinary users. Big tech saw the opportunity: package OpenClaw's capabilities into a familiar interaction interface, provide official connectors, and drive it all with stronger models. That's the shared logic behind Grok Bot, Dots, and Muse.

The core goal of this project is to replicate those same capabilities while staying open-source and customizable. Users can choose the Grok Bot look, switch to a Dots or Muse style, or even fully customize it. The key point is that the entire development process isn't "vibe coding"—every step knows exactly what it's doing, using Skills and MCP to constrain the agent's behavior and avoid producing AI slop code.

Architecture Design: Next.js + Trigger.dev + Sandbox

The application interface is divided into three parts: a global sidebar, a conversation area, and a resizable desktop preview panel. Users can have the agent operate the remote desktop within the conversation, or take over at any time.

When a user submits a message, it first reaches a server action or route handler in the Next.js app. But the Next.js environment isn't suited for long-running AI tasks—it may time out, and deployments or crashes will interrupt the task. So the message is handed off to a stateless, persistent worker node, which in this case is Trigger.dev. This worker process is independent of Next.js, has no request timeout limit, and can automatically recover after a crash.

The real core operations happen within this worker process: calling the AISDK agent and giving it sandbox execution tools, sandbox file tools, and sandbox screenshot tools. Together, these three tools cover 99% of computer use tasks.

Trigger.dev is an open-source orchestration platform for AI workflows and background tasks, whose core value lies in providing a persistent execution environment unconstrained by the request lifecycle. Unlike traditional Next.js API routes or serverless functions, Trigger.dev tasks run on independent long-lived worker nodes, supporting execution times of up to several hours, automatic retries, resumable execution, and crash recovery. Its task model is based on the abstractions of "run" and "task," where each run has a unique ID and persistent state and can be queried, cancelled, or replayed via the SDK. In AI agent scenarios, this combination of "stateless scheduling + stateful storage" is especially well-suited: the worker process itself can restart at any time, but the task state is persisted to an external database, so even if a new version is deployed or a node crashes, an in-progress agent task can continue from where it left off without losing context.

Remote Desktop and User Takeover

The remote desktop is implemented based on the VNC protocol. Each time an agent is created, a sandbox instance is created alongside it. The sandbox uses the NoVNC-based open virtual network computing protocol and exposes a specific port. On the UI side, a page with a signed URL is loaded via an embedded frame; after the user clicks, they take over in a full-screen dialog with input enabled.

This mechanism makes "human intervention" very natural. For example, when the agent encounters information it doesn't know while booking a flight (such as a payment password), it gets stuck and waits. After the user takes over the desktop and enters it manually, the agent can remember these details and complete the task automatically next time.

VNC (Virtual Network Computing) is a remote desktop sharing technology based on the RFB (Remote Framebuffer) protocol, which captures framebuffer changes on the server side, encodes and transmits them to the client, and then sends keyboard and mouse events from the client to achieve remote control. NoVNC, on the other hand, is a VNC client implemented in HTML5 and JavaScript. It uses WebSocket to wrap the RFB protocol into a communication channel usable by browsers, so no plugin or local client is needed—it can be embedded directly into a web page. In this project, the VNC server runs inside the sandbox container, NoVNC provides browser-side rendering and input handling, and the embedded frame integrates the signed-URL NoVNC page into the application UI. The role of the signed URL is to temporarily authorize access to the sandbox port, avoiding directly exposing internal services while also supporting automatic expiration after the session ends, improving security.

Skill System: Making the Agent Work Reliably

The project makes extensive use of the "Skills" mechanism. After installing the ShadCN skill, the agent can reliably use the component library without having to scrape documentation from the web or rely on training data. After installing Firecrawl, the agent can bypass JavaScript rendering limitations, scrape complete webpage information, and even directly call the Y Combinator data registry to search for competitors.

According to the creator's testing, replacing the built-in web search with Firecrawl reduced token consumption by 93%. After installing the Clerk skill, the agent can automatically complete authentication integration; after installing the Neon skill, it can configure Drizzle ORM and database push; after installing the Railway skill, it can complete deployment and environment variable synchronization directly through conversation.

MCP (Model Context Protocol) is a standardized protocol proposed and open-sourced by Anthropic, designed to let AI models connect to external tools, data sources, and APIs in a unified way. It defines primitives such as tools, resources, and prompts, and models dynamically discover and invoke these capabilities through a client-server architecture without having to hard-code them during training. In this project, MCP complements the "Skills" mechanism: skills are more like high-level encapsulations for a certain type of task (such as "use ShadCN components" or "scrape web pages via Firecrawl"), while MCP handles the underlying communication and tool exposure. Constraining the agent's behavioral boundaries through skills can effectively reduce the uncontrollable output caused by the model improvising, while also lowering token consumption from repeatedly querying documentation or trial and error—this is the fundamental reason why token usage dropped 93% after replacing the built-in search with Firecrawl.

Persistence, Group Chats, and Tool Calling

The initial implementation ran AI conversations in the route handler, and the problem was obvious: refreshing the page interrupted the task. After migrating to Trigger.dev, each chat session has its own persistent storage, isolated by Chat ID. Even if the user closes the page while the agent is executing a long task, the task continues and the result is saved.

The group chat feature demonstrates the possibility of "one-round generation." Because the project had already accumulated enough form examples, conversation examples, and persistence foundations, the agent only needed to build on the existing base rather than invent something new. The creator generated a usable group chat feature with just one prompt, including group chat avatars (showing up to three bot avatars, with the remainder represented by a number), UI for switching conversation bots, and message metadata display.

On the tool-calling side, the agent is given basic tools such as file read/write, mouse click, keyboard input, and screenshots. When the agent needs to look at the screen, it uses the "view screen" tool; when the user asks what's on the screen, it uses the "show screen" tool and renders the screenshot in the chat.

Memory System and Billing

The memory feature is implemented through a memory.md file in the sandbox. Initially, it tried to write via file upload but hit an error because the Node runtime lacked the FormData module. Switching to using the Touch command to create the file directly solved the problem. Then get and set tools were added for memory, making invocation more stable.

Billing uses Clerk Billing. The free plan locks all features, while the paid plan unlocks bot creation, message sending, sandbox connections, and more. When users attempt an operation, they receive a notification directing them to the pricing page. The creator emphasizes that this approach is suitable for small demos, but true cost accounting requires a ledger system to track AI tokens and sandbox usage.

Clerk Billing is a subscription and billing module launched by the Clerk identity authentication platform. It integrates user management, organization permissions, and paywall capabilities, so developers can implement plans and feature gating without separately integrating payment gateways like Stripe. In this project, the free plan locks all agent features through Clerk Billing, while the paid plan unlocks operations such as bot creation, message sending, and sandbox connections. When a user attempts to trigger an unauthorized operation, the system returns a structured error response, based on which the frontend pops up an upgrade prompt and redirects to the pricing page. It's worth noting that Clerk Billing mainly solves the "who can use it" problem, while true cost accounting—tracking AI token consumption and sandbox resource usage per session—still requires an independent ledger system; otherwise, it's easy to fall into hidden losses where revenue and expenses don't match.

Observability: From Next.js to Stateless Workers

Sentry is integrated across the entire pipeline. Errors in the Next.js app and errors in the Trigger.dev worker process can all be captured uniformly. The session replay feature lets developers replay user actions and see precisely the mouse position and operation sequence at the time of the error. Sentry's code review agent Seer can also automatically analyze root causes, generate fix plans, and even submit PRs directly.

After completing observability, the creator remarked: if Sentry had been integrated earlier, many errors could have been discovered before deployment. End-to-end logging and error tracking, combined with AI's analytical capabilities, make debugging no longer a guessing game.

Takeaways from This Methodology

The value of this project isn't just that it replicates an AI teammate product, but that it demonstrates an engineering paradigm for building AI-native products: first teach the agent how to work using skills and foundational components, then let it implement specific features. Every step is supported by clear infrastructure, avoiding the uncontrollable code that comes from the agent improvising.

From open-source OpenClaw to the commercial product Grok Bot, and now to replicating the entire system with Claude Code, the barrier to entry for AI teammates is rapidly lowering. The real moat may no longer be model capability, but rather how to design a reliable engineering framework that lets agents complete complex tasks within human-controllable bounds.

One-line Summary

A Bilibili creator uses Claude Code to replicate a production-deployable AI teammate system, showcasing an AI-native engineering methodology.

Paragraph Summary

The article describes how a Bilibili creator used Claude Code to fully replicate a production-deployable AI teammate system, covering the interface, sandbox, memory, billing, and observability. The project adopts a Next.js + Trigger.dev + VNC sandbox architecture, using stateless worker processes to solve AI task timeouts and interruptions; through the skill system (ShadCN, Firecrawl, Clerk, Neon, Railway), it constrains the agent's behavior, reducing token consumption by 93%; it implements a remote desktop based on VNC and supports user takeover at any time, with the agent able to remember takeover details. Persistence is isolated by Chat ID, and the group chat feature is generated in one round by leveraging existing components. Sentry's end-to-end observability covers the entire pipeline, and session replay plus Seer's automatic fixes significantly improve debugging efficiency. The article ultimately points out that the moat for AI teammates is shifting from model capability to reliable engineering framework design.