AI Large Language Models + MCP Protocol: A Hands-On Tutorial for Fully Automated Unity Digital Twin Construction

Introduction: A Paradigm Shift in Digital Twin Construction

In the fields of industrial automation and Virtual Commissioning, building digital twin systems has long been a highly manual endeavor—engineers must repeatedly drag and drop components, set properties, and configure kinematic parameters in platforms like Unity. However, with the rapid advancement of Large Language Model (LLM) capabilities, all of this is being fundamentally rewritten.

A Digital Twin refers to a precise digital replica of a physical entity in virtual space. It encompasses not only geometric models but also physical behaviors, kinematic constraints, sensor logic, and control systems. Virtual Commissioning is the process of validating PLC programs and automation logic through digital twin models before physical equipment is actually installed. Traditionally, engineers need to individually model, configure joint degrees of freedom, set up collision detection, and map signals in platforms like Unity, Siemens NX MCD, or Visual Components. A production line digital twin of moderate complexity might require weeks or even months of manual construction work.

This article is based on a Bilibili technical tutorial (originally in English), showcasing a complete workflow: building a fully functional digital twin simulation system entirely automatically through an AI large language model paired with an MCP server, without any manual Unity operations. The tutorial author believes this could fundamentally change how people build simulation and virtual commissioning projects.

MCP Server: The Bridge Between Large Language Models and Unity

The key to letting a large language model drive Unity lies in an MCP (Model Context Protocol) server. MCP is a protocol standard open-sourced by Anthropic in late 2024, designed to provide LLMs with a unified interface for interacting with external tools and data sources. It adopts a client-server architecture: the LLM acts as a client issuing tool call requests, and the MCP server receives these requests, executes the corresponding operations, and returns results. Unlike traditional API integration, MCP's design philosophy enables AI to actively discover and invoke tools rather than relying on hardcoded call paths. This allows AI to dynamically compose tools to complete complex tasks, similar to how a human engineer flexibly uses various plugins in an IDE.

According to the tutorial demonstration, this solution called Virtual I/O MCP Server consists of two parts:

• Unity-side component: Responsible for executing operations within the Unity editor
• Python server: Acts as the interface hub between Unity and the LLM

MCP Server Installation Methods

The author provides two ways to obtain it: through the Unity Asset Store (simpler, recommended) or through the GitHub open-source repository. Search for "Virtual I/O MCP Server" in the Asset Store, add it to your assets, then download and import it into your project via Package Manager.

After import, the system automatically prompts you to install the Python server. This Python server is deployed to the project's Streaming Assets folder. In Unity's project structure, Streaming Assets is a special directory whose files are copied as-is to the target platform during build, without undergoing Unity's asset serialization and compression. Deploying the Python MCP server in this directory means it can run as an independent process, communicating with the Unity editor via WebSocket or local pipes while maintaining a connection to the external LLM. This architectural design allows the MCP server to deeply access Unity's internal APIs (through Editor scripts) while existing independently of Unity's runtime lifecycle. Its key advantage is that it can truly comprehensively control Unity, including taking screenshots, starting and stopping simulations, and more.

A Highly Extensible MCP Tool System

What's particularly noteworthy is the extensibility of this MCP system. After installation, the interface displays a complete list of all tools provided by MCP. The author emphasizes that if you know scripting, you can easily add custom tools simply by adding an annotation to your script—meaning you can extend functionality without writing complex custom logic for the MCP server. This annotation-based tool registration mechanism is similar to the dependency injection pattern in Java's Spring framework: developers only need to declare intent (by annotating methods), and the framework automatically handles discovery, registration, and routing. Additionally, the interface allows you to directly start and stop the server and view its current status information.

Connecting Claude Code: Letting AI Understand Your Unity Project

After completing the MCP configuration, the tutorial moves to the code editing environment. The author primarily uses Claude Code from Anthropic (integrated into VS Code), but emphasizes that users can freely choose any coding LLM agent or use CLI methods.

Claude Code is Anthropic's terminal-native AI coding agent that can directly understand codebases, execute commands, and perform multi-step reasoning. Unlike traditional code completion tools, Claude Code has full agent capabilities—it can browse file systems, run terminal commands, invoke external tools, and dynamically adjust strategies based on execution results.

Project Initialization and AI Memory Mechanism

The first step is to have Claude "learn" the entire project. The LLM automatically inspects the project structure and generates a CLAUDE.md file—this file serves as a cross-session "memory" carrier. CLAUDE.md is the core of Claude Code's project memory system: when AI first analyzes a project, it persists information about project structure, tech stack, key conventions, and available tools into this Markdown file. When subsequent sessions start, Claude prioritizes reading this file to restore context, avoiding redundant analysis. This solves the fundamental problem of memory loss between LLM sessions, enabling it to work continuously like a team member familiar with the project.

During the learning process, the AI automatically identifies all MCP tools, README documentation, and related data in the project, thereby grasping the project's foundational architecture.

The configuration is then written to Claude Code via the "Cloud" button in the interface, and VS Code and Unity are restarted to enable the new session connection.

First Verification: Letting AI Understand the Unity Scene

After restarting, the author issues the first command: check the current scene contents. At this point, Unity gains focus, and the MCP feeds scene information back through the LLM. The author then has the AI start a motor-driven conveyor belt and check its running status—the AI executes successfully, the view begins to move, sensors read data from the occupied zone, and everything runs normally.

This verification process demonstrates the complete closed loop of AI-driven simulation: command issued → MCP forwarded → Unity executed → status returned → AI confirmed. Throughout the process, the AI not only issues operation commands but also verifies execution results through screenshots and status queries, forming a workflow pattern similar to a human engineer's "operate-observe-adjust" approach.

Why Open Systems Are Critical for AI-Driven Simulation

During a break in the demonstration, the author raises a notably forward-looking viewpoint. He believes that in the future, cloud-based digital twins, simulation, and modeling will no longer be what they used to be—all the tedious dragging and dropping, property setting, and even kinematic configuration will soon be handled by AI and LLMs.

This is precisely why he emphasizes the need for open systems like ReVirtual. In the industrial software space, products from traditional giants (such as Siemens, Dassault) often adopt closed architectures where internal logic is invisible to external programs. When AI needs to understand and manipulate software, closed systems form natural barriers—LLMs cannot read proprietary formats or call unpublished APIs. Open systems (like ReVirtual's Unity-based solution) are different: source code is readable, interfaces are extensible, and data formats are transparent. This means AI can deeply understand the behavioral logic of every component, rather than operating only at the surface level.

When source code is open and AI can understand everything, AI can truly create productive outcomes for you and save enormous amounts of time. This also highlights a key trend: In the era of AI-driven industrial software, system openness and comprehensibility will become core competitive advantages. Comprehensibility equals automatability—this is the underlying logic reshaping the competitive landscape of industrial software in the AI era.

Hands-On Demo: Fully Automated Digital Twin Scene Construction from Scratch

To fully demonstrate AI's automated construction capabilities, the author issues a complex command in a completely new session (without old session memory):

"Stop and save the current scene, create a scene for me with a conveyor, a source above the conveyor, and a sensor at the end, name it AI Demo."

The AI's execution process reveals a clear working logic:

1. Check Unity status, confirm it's running
2. Create a new scene
3. Search for suitable prefabs in the project to build the conveyor
4. Generate the conveyor, source, and sensor beam
5. Check bounds to confirm the setup is correct
6. Take a screenshot to gain visual understanding of the scene

Unity's Prefab system allows fully configured game objects to be stored as reusable assets, including hierarchy structure, component parameters, and material references. In the industrial simulation context, prefabs typically represent standardized equipment—such as specific conveyor belt models, robotic arms, or sensors. When AI queries available prefab libraries in the project through MCP, it's essentially browsing a parameterized industrial equipment catalog. This "build library first, then invoke" pattern means domain experts' modeling experience is encapsulated in prefabs, while AI is responsible for understanding requirements and correctly assembling these modules.

Built-in Industrial Domain Knowledge Is Key

There's a particularly noteworthy detail here: the AI automatically added the ReVirtual controller component. The author explains that their solution "Virtual Professional" has MCP-centric domain knowledge built in, capable of providing the LLM with solid application expertise. In this example, the LLM "knows" it needs to add a ReVirtual component to the scene—this isn't common knowledge for a general-purpose large model, but rather encapsulated industry knowledge.

This domain knowledge encapsulation can be technically implemented through multiple paths: context information can be injected into the LLM through MCP tool descriptions, industry rules can be provided through system prompts, or relevant documentation can be dynamically loaded through RAG (Retrieval-Augmented Generation) mechanisms. Regardless of the approach, the core concept is consistent—making the tacit knowledge accumulated over decades by industry experts explicit and structured so it can be invoked by AI during reasoning. This is precisely the key mechanism through which "AI + domain knowledge" produces a 1+1>2 effect.

After several rounds of back-and-forth attempts, the AI ultimately completed the task: the simulation started and ran, with sensors successfully detecting materials. The author acknowledges that fine-tuning of the source and the number of generated parts might still be needed, but the AI essentially completed the core work.

Conclusion: The Future Outlook for AI-Driven Digital Twins

The author concludes the tutorial with a clear judgment: "We're just getting started." In the future, more and more AI capabilities will be integrated into product simulation, and virtual commissioning will be fundamentally transformed.

From a technical analysis perspective, this case reveals several deep signals:

• AI is moving from "assistive tool" to "executing agent": LLMs no longer just give suggestions—they directly operate professional software to complete end-to-end tasks. This transformation is similar to the leap from GPS navigation (telling you how to drive) to autonomous driving (driving for you)—AI no longer needs humans as an intermediary in execution.
• The MCP protocol is becoming critical infrastructure for AI integration with professional tools: It gives AI a complete closed-loop capability of control, observation (screenshots), and iteration. This "perception-decision-execution-feedback" cycle is the essential characteristic that distinguishes agents from simple conversational models.
• Domain knowledge encapsulation is a differentiating moat: General-purpose large models don't understand industrial kinematics, but encapsulated domain knowledge enables AI to make professional judgments. This means that the core assets of future industrial software companies will shift from "software features" to "domain knowledge bases that can be understood and invoked by AI."

For practitioners in industrial simulation and virtual commissioning, this represents both an opportunity for efficiency revolution and a harbinger of profound changes in skill structures. The engineer's role will transform from "manual builder" to "AI-collaborative architect"—with core value lying in defining problems, validating results, and handling edge cases, rather than repetitive modeling operations.

Key Takeaways