AI Agent Controls Your Computer: One Sentence to Complete a Return and Get the Pickup Code
One WeChat message triggers an AI Agent to complete an entire e-commerce return and retrieve the pickup code automatically.
This article documents a real AI Agent demo where a single WeChat instruction triggered the Agent to autonomously navigate an e-commerce platform, select a return reason, upload photos, schedule a courier pickup, and send the pickup code back to the user's phone — no manual computer use required. Two key engineering highlights stand out: a workflow reuse mechanism that converts first-time explorations into one-click templates, and real-time step-by-step reporting that keeps AI behavior transparent and traceable. The article also honestly addresses current limitations, including UI change-driven instability, permission and security boundaries, and the challenge of handling CAPTCHAs and unexpected exceptions.
When AI Starts Actually "Getting Its Hands Dirty"
For a long time, AI has mostly lived in the realm of conversation, text generation, and image creation. But the real turning point — the one that's genuinely exciting — is when AI gains the ability to control a computer. Like a human, it can click buttons, fill out forms, upload images, and complete an entire workflow that would otherwise require the user to do everything manually.
This article is based on a real hands-on demo showing how an AI Agent can handle one of life's most tedious yet frequent tasks: processing an e-commerce return and retrieving the pickup code. Simple as it sounds, this involves multi-step automated decision-making and execution — a genuine test of what AI Agents can do.
AI systems capable of directly controlling graphical interfaces are commonly referred to as "Computer Use" or "GUI Agents." Unlike traditional RPA (Robotic Process Automation), AI Agents don't rely on pre-recorded fixed scripts. Instead, they combine visual perception (screenshot recognition) with large language model reasoning to dynamically assess the current screen state and decide the next action. Anthropic's Claude Computer Use, released in October 2024, was a landmark milestone in this space, and since then multiple teams both domestically and internationally have launched similar products. The core technical pipeline typically looks like this: screenshot → visual understanding → action planning → simulated mouse/keyboard input — forming a closed loop of perception, decision-making, and execution.
The Full Demo: From Initiating a Return to Getting the Pickup Code
The entire demonstration centered on a real return request, walking through the complete automated flow from initiating the return to receiving the pickup code.
Triggering Automation via WeChat
Interestingly, the interaction didn't start from a traditional computer interface — it started with WeChat. The user sent an instruction from their phone to the desktop Agent: something like "schedule a return pickup and send me the pickup code." This design fits naturally with how users in China actually behave: no need to sit down at a computer, just send a message from your phone and the entire automated flow kicks off.
Once the Agent (called Handle in this demo) received the instruction, it began executing the task autonomously.
AI Autonomously Fills Out Return Information
During the return process, the AI demonstrated several key capabilities:
- Choosing the right return reason: Based on the user's text description, it automatically matched and filled in the appropriate return reason — no mindless clicking.
- Uploading photos of the received item: Photos the user had previously sent to the Agent were automatically uploaded one by one to the return portal, along with a message.
- Submitting and scheduling pickup: Finally, the AI clicked the submit button and arranged a time for the courier to come.
In essence, every step a person would take when processing a return was broken down and handed off to the AI to execute sequentially.
Scheduling Pickup and Returning the Code Automatically
After completing the return, the demo went a step further to showcase the sub-task of "scheduling a pickup time and returning the pickup code."
Reusable Workflows: From Exploration to One-Click Execution
There's an engineering detail worth highlighting here: the platform's return workflow had already been "summarized" into a reusable template. This means the AI wasn't starting from scratch each time — it was executing based on a pre-established workflow. Open a new session, issue the instruction "schedule a return pickup and return the pickup code," and the AI runs through the entire process smoothly.
This idea of "workflow consolidation and reuse" is central to making AI Agents practically useful. Completing a task once isn't hard — the hard part is turning it into a reliable, repeatable automation.
"Workflow consolidation" maps to what's typically called "Memory" or "Workflow" mechanisms in AI Agent engineering. After an Agent successfully completes a task for the first time, it can structurally store the successful action path, page element localization methods, and exception-handling strategies — forming a reusable "skill" or "Playbook." This mirrors the concept of function encapsulation in software engineering: turning a one-time exploration into a stable, callable module. Some frameworks (like LangGraph and AutoGen) achieve this through explicit workflow definitions, while others rely on the Agent's long-term memory system to automatically distill patterns. The core value of workflow reuse is that it dramatically reduces per-task reasoning overhead and error rates, while enabling non-technical users to trigger complex automation sequences through natural language.
No Need to Open the Platform — Get the Code Straight on Your Phone
The end result was impressive: without the user ever personally opening the e-commerce platform, the AI autonomously navigated the platform, scheduled the pickup, retrieved the pickup code, and sent it back to the user's phone.
For everyday users, this means the whole hassle of returning a package can genuinely be handled with a single sentence.
Transparent Operations: Every Step Is Traceable
On the question of trust, this demo offered a compelling answer. Handle sends every action step back to the user during execution, so users can check at any point in the conversation whether the AI did something unexpected.
This is critical for real-world AI Agent deployment. When AI is operating our accounts, uploading our information, and clicking submit on our behalf, the biggest concern is always: "What exactly did it do?" Making the process visible and traceable is the foundation of building trust — and an essential safeguard against the AI going off the rails.
Takeaways: Value and Limitations of AI Agents in Practice
Core Value: Fully Automating Tedious Workflows
Processing a return, scheduling a courier, getting a pickup code — none of these are individually complex, but together they're quite a chore, and they happen all the time. The value of AI Agents lies precisely here: they can take over those "not hard, just annoying" repetitive tasks from users. Once similar workflows are captured as reusable templates, future tasks — whether it's scheduling a pickup or grabbing a tracking code — can be triggered with a single command.
Current Limitations and Open Questions
Of course, there's still a meaningful gap between a demo and large-scale reliable deployment. A few key issues worth watching:
- Stability: E-commerce platform UIs change frequently. Do workflow templates need constant maintenance?
- Permissions and security: When AI operates in sensitive areas involving account details and shipping information, how should permission boundaries be defined?
- Exception handling: When the AI encounters platform pop-ups, CAPTCHAs, or unexpected pages, can it handle them gracefully — or does it just freeze?
The answers to these questions will determine whether "AI controlling your computer" can truly become part of daily life.
CAPTCHA is one of the most common technical obstacles facing Computer Use-type Agents today. Major e-commerce platforms trigger slider or image verification challenges when they detect abnormal behavior — such as unusually regular operation timing or anomalous user-agent signatures — and these human verification mechanisms were specifically designed to block automated scripts. Additionally, some platforms have begun actively detecting and restricting AI-controlled behavior, which means an Agent's stability faces adversarial pressure from the platform side, not just a pure technical capability problem. From an industry perspective, how to advance "proxy operations" within a compliance framework while avoiding account bans or data leakage risks is a real-world challenge that this field must confront head-on before achieving commercialization.
Closing Thoughts
This return-processing demo is a vivid snapshot of AI's transition from "can talk" to "can do." When AI not only understands our intent but can also physically carry out an entire computer workflow on our behalf, the boundaries of human-machine collaboration are being redrawn. There's still road ahead before this technology fully matures — but the experience of handling something annoying with a single sentence is already enough to make the future of AI Agents genuinely exciting.
Related articles
Supply Chain Hardware Implants: The Most Dangerous Security Threat You're Overlooking
A deep dive into supply chain hardware implant attacks: how they work, historical cases, and defense strategies. Learn why hardware backdoors are nearly undetectable and how to build a zero-trust defense.
Apple M6 and M5 Ultra Chips Unveiled: What the Major AI Performance Boost Really Means
Apple launches M6 and M5 Ultra chips with dramatically enhanced Neural Engine and on-device AI performance. A deep dive into architecture upgrades, unified memory, and real-world impact.
Fine-Tuning LLMs to Mimic Real Human Chat Styles: A Guide to Building Emotion-Aware Datasets
How to fine-tune an LLM to mimic real human chat styles? This guide covers emotion labeling, context-aware datasets, LoRA fine-tuning, and iterative optimization.