6 Self-Hosted AI Agents Compared: Which One Is a Truly Self-Evolving Digital Employee?

Why GitHub Stars Can't Determine an AI Agent's Real Ranking

In the open-source AI Agent community, GitHub stars have almost become the default basis for ranking. But in the latest comparison from Bilibili creator "Asteroid AI Observatory," a clear stance is put forward: stars only show how many people are watching—they don't show whether it will grow stronger on its own once it "goes to work."

GitHub Stars represent the act of a developer bookmarking or following a repository—essentially a "bookmark" feature rather than a quality certification. Star counts are strongly influenced by traffic events such as launch timing, social media buzz, Product Hunt debuts, and Hacker News rankings, and their correlation with a project's actual usability, stability, or depth of innovation is quite limited. Notably, GitHub Stars also suffer from "star inflation"—some projects rapidly accumulate stars through paid promotion or organized engagement, further undermining their credibility as a quality signal. Academic research (such as a 2019 MSR conference paper) has documented the lack of statistically significant correlation between star counts and code quality metrics. For infrastructure like AI Agents that requires long-term stable operation, stars are especially misleading for selection—they measure the excitement at discovery, not the reliability after use.

This ranking deliberately differs from encyclopedic introductions or installation tutorials—it is an opinionated, subjective list. The reviewer puts six self-hosted "digital employees" on the table with just one core question: who truly grows on its own, and who merely wraps a chatbox in a system shell.

The most counterintuitive conclusion is this: Open Claw, which has the most stars across the web (380,000) and the most mature ecosystem, ranks only third. The winner is a small project with a seed codebase of just 3,000 lines.

Three Thresholds for Evaluating Persistent Agents

Traditional chatbots are essentially stateless request-response systems: a user enters a message, the system generates a reply, and no action capability is retained after the interaction ends. AI Agents build on this by introducing an "Action Loop" architecture: perceive the environment → make a plan → invoke tools → observe results → update state, forming a continuously running closed loop. This loop architecture derives from the "perception-action-reward" paradigm in reinforcement learning, but modern LLM-based Agents concretize it as: decomposing tasks via Chain-of-Thought or Tree-of-Thought reasoning, invoking external tools through Function Calling interfaces, and injecting tool return values into context to drive the next iteration. The loop's stability depends heavily on the LLM's instruction-following ability and the idempotency design of tool calls. Core components of a modern AgentOS typically include: a task Planner, a Tool Use / Function Calling layer, a memory system (short-term working memory + long-term vector storage), and an Executor. This architecture lets Agents maintain state across sessions and proactively trigger operations, but it also means they gain continuous access to system resources, making the issue of permission boundaries critically important.

The reviewer offers a clear evaluation framework: a qualified persistent AgentOS must pass at least three tests:

• Stays up: Can it run stably 24/7?
• Can learn: Is there an intrinsic mechanism supporting it getting stronger with use, rather than a marketing gimmick?
• Stays under control: Do data, permissions, models, and costs ultimately remain in the user's own hands?

In his words: "Being able to chat is just the entry ticket—being able to grow on its own is the real skill." This framework also forms the three dimensions of the subsequent ranking: persistence and skill ecosystem, growth potential (self-evolution and long-term memory), and whether it can be "tamed" (support for open-source domestic models, transparent data ownership, and controllable token billing).

Sixth to Fourth Place: Middleware, Chinese Entry Points, and Cold-Start Solutions

Sixth Place — OpenScaler: A Token-Saving Intelligent Scheduling Layer

The cost of calling large language models generally correlates with model parameter count: a GPT-4-class model's token price can be 10-100x that of a small model. Token Routing or Model Cascading strategies were created precisely to solve this cost problem—the core idea is to use small models for simple sub-tasks and only invoke large models for complex reasoning. In engineering terms, routing decisions can be based on predefined rules (a task-type classifier decides model selection), dynamic confidence-based judgment (automatically escalating when the small model's output confidence falls below a threshold), or reinforcement-learning routing policies trained on historical task results. Notable industry references include routing services like Martian and Unify, as well as Meta's FrugalGPT framework—for model combinations with vastly different token prices, reasonable routing can reduce inference costs by 60%-90% while maintaining a relatively high overall task success rate.

OpenScaler's most impressive capability is its local model routing built on this idea—a task is first attempted with a low-cost model, then switched to a high-performance one when it hits a bottleneck. Combined with MetaSkill task orchestration, it acts more like a scheduling-savvy "money-saving middleware." For persistent Agents that need to run for long periods and frequently invoke tools, token cost control is often one of the key factors determining whether a project is production-viable.

But its positioning is also its limitation: it's closer to an infrastructure layer than a complete persistent Agent OS. The reviewer's advice: if you need to run long tasks and care about token bills, it has clear value; but if you want an all-around digital butler, don't make it your starting point.

Fifth Place — Hana Agent: Winning on Where Users Actually Are

Hana Agent's core strength lies not in technical papers, but in integration scenarios close enough to Chinese users—WeChat, QQ, and Feishu are the entry points domestic users actually open every day. It also designed a personality layer for the Agent, targeting not just the developer crowd.

The reviewer endorses this direction because it clearly knows where its users are. But in memory depth and self-evolution capabilities, Hana Agent is currently not mature enough to withstand production-level pressure. It suits users who want to first "raise" an Agent in Chinese chat scenarios—but don't let it take over core workflows on day one.

Fourth Place — OpenHuman: Brute-Forcing the Cold Start via Data Sync

The Cold Start Problem, in the recommendation system field, refers to the difficulty where new users produce poor initial recommendations due to lacking historical data. Transplanted into the AI Agent context, it refers to the problem of an Agent being unable to make high-quality decisions in its early stage when lacking user context. An Agent's cold start is essentially a Context Engineering problem—a concept that has evolved from Prompt Engineering in recent years, referring to a set of techniques and strategies for maximizing the injection of context information most valuable to the current task within a given token window limit. Its core challenge is that an LLM's effective context window is often smaller than its nominal maximum length, and the position of information within the prompt significantly affects the model's actual utilization—a phenomenon known as the "Lost in the Middle" effect, documented in empirical research from Stanford University. A model's capability ceiling is determined by training, but its actual performance quality is determined by the quality of the injected context.

OpenHuman's approach to solving cold start is quite aggressive: connecting email, calendar, code repositories, and notes, it first syncs the user's context into a local knowledge base, replacing messy raw data streams with structured local knowledge so the limited context window is used most efficiently. Where other Agents need you to explain a week's worth of background, it understands you better right after its first sync. This is an aggressive context-preloading strategy that can significantly compress the cold-start period, at the cost of requiring the user to grant read access to a large amount of sensitive data—a trade-off that reveals a universal tension in Agent design: the more capable and personalized, the wider the required data-access boundary.

It's genuinely smart, and the risk is genuinely real—it needs continuous read access to your inbox and work materials. It suits knowledge workers whose context is scattered across ten tools, but for sensitive accounts, phased integration is recommended.

Top Three: The Ecosystem King vs. the Truly Growing Digital Employee

Third Place — Open Claw: The Ecosystem Champion Deliberately Downgraded

380,000 stars, 17,000 skills, coverage across dozens of chat platforms—Open Claw is the undisputed ecosystem king of this circle. Whatever tutorials, plugins, or ready-made skills you need, it has them all.

But the reviewer publicly downgraded it a notch this time, for a thought-provoking reason: ecosystem scale does not equal a growth mechanism. Open Claw can of course implement various self-evolution setups through plugins, but its core architecture doesn't have a "learning loop" baked in. It's the industry benchmark, but it's not the best answer to the proposition of self-evolution.

Second Place — Hermes: The Skill Consolidator with Proven Results

Hermes has just one keyword—it truly grows. It observes how you complete tasks and consolidates successful paths into reusable skills. The longer you use it, the more capabilities it grows for itself.

An Agent's "self-evolution" typically has two engineering paths. The first is Skill Crystallization: the Agent abstracts the execution sequence of successfully completing a certain type of task into a reusable program or function, stores it in a skill library, and directly invokes it when encountering a similar task next time—avoiding repeated reasoning. Hermes takes this path. Representative academic work includes Stanford's Voyager (which automatically generates and accumulates an executable skill library in the Minecraft environment) and the ExpeL framework from ACL 2024; its advantage is that crystallized skills are deterministic executable code—fast and predictable—while its drawback is that the skill library requires ongoing management (deduplication, version control, failure detection). The second path is RAG-based Memory: vectorizing historical interactions, user preferences, and task results into a persistent database, and retrieving relevant context to inject into the prompt for new tasks, achieving a "understands you better the more you use it" effect. Representative frameworks include MemGPT and Zep. The core difference is: the former generates executable code, the latter generates retrievable knowledge; the two are not mutually exclusive, and some advanced Agent frameworks have already tried combining them.

This design direction is very close to the real vision of a "personal digital employee." The downside is also direct: it's fairly "dumb" during the startup period—you need to raise it for a few weeks first, and can't demand full-strength performance on day one. It suits users willing to invest long-term and treat the AI Agent as a genuine collaboration partner.

First Place — Generic Agent: A Self-Evolving OS in 3,000 Lines of Code

The champion isn't the biggest, but it's the closest to the reviewer's ideal form. Generic Agent is defined as "an operating system that grows new skills": 3,000 lines of seed code, 9 atomic tools, and a 100-line main loop. Every time it solves a new task, it crystallizes the execution path into a new skill—compact in design, ruthless in logic, and controllable in its boundaries.

It takes the skill-crystallization mechanism to the extreme: the concise design of a 100-line main loop means system state is highly transparent and behavior is predictable—which is precisely the engineering foundation for "controllable boundaries." In software engineering, there is a classic trade-off between code size and auditability—fewer lines of code mean a smaller attack surface, a more easily enumerable state space, and more easily verified behavior. Compared to projects that pile on functionality through massive plugin ecosystems, Generic Agent implements "self-evolution" in the most minimal, most auditable way: every piece of new code produced by each skill crystallization can be fully traced, the entire system's evolution path remains transparent to the user, and its design effectiveness is backed by academic papers.

It supports integration with domestic open-source models and is backed by academic papers. Compared to other projects that pile on massive ecosystems, Generic Agent handles "self-evolution" exceptionally cleanly. Its downside is that it requires a certain amount of tinkering ability—don't expect a luxurious out-of-the-box experience.

Complete Ranking and Selection Advice

The full list of this comparison:

1. Generic Agent — The purest self-evolution design
2. Hermes — A skill-consolidation mechanism with proven results
3. Open Claw — The most mature ecosystem and industry benchmark
4. OpenHuman — Breaks the cold start via data sync
5. Hana Agent — The most authentic Chinese-scenario entry point
6. OpenScaler — A cost-saving routing scheduling layer

The ordering logic is singular: those that truly grow on their own rank higher, and those that patch capabilities via ecosystems, plugins, and routing rank lower. This is a holistic judgment, not one to nitpick along a single dimension.

Choose by Workflow, Not by Ranking

The reviewer's selection approach is highly pragmatic:

• Want self-evolution ability: Look first at Generic Agent and Hermes;
• Want the largest plugin ecosystem: Open Claw is the top choice;
• Want a Chinese chat entry point: Start by trying Hana Agent;
• Just want to minimize token costs for long tasks: OpenScaler has clear value.

There is no all-around king—only whether it fits your task scenario, data environment, and patience budget.

Set Boundaries for Persistent Agents, Don't Let Them Run Naked

The final reminder of the comparison deserves particular attention: the potential risks of persistent Agents should not be underestimated. Their selling point is precisely that they can operate systems for you, read and write files, send messages, execute commands, and invoke external tools—the more capable they are, the more they need clear boundaries.

The industry has documented several attack vectors targeting Agents: Prompt Injection—OWASP has listed it as the top security risk for LLM applications (OWASP Top 10 for LLM Applications, LLM01), where malicious content triggers unintended instructions through external documents the Agent reads (web pages, emails, RAG databases); in 2023, research already demonstrated a complete attack chain that induces an Agent to leak private information by poisoning a RAG database. Goal Hijacking—intermediate steps in a long-chain task are tampered with, causing the final behavior to deviate from expectations. And Permission Creep—homologous to the "confused deputy" problem in operating systems, where an Agent, while executing an authorized task, is induced to use its already-granted high privileges to complete unauthorized operations. The NIST AI RMF and MITRE ATLAS frameworks have begun to systematically catalog such AI-specific attack surfaces, serving as important reference baselines for building a persistent Agent security strategy. The Principle of Least Privilege in security practice, in this context, means: sandbox environments to isolate execution context, interface whitelists to limit the invocable tool set, and budget caps to constrain resource consumption.

The correct approach is: first give it sandbox isolation, an interface whitelist, and a budget cap, clearly defining what it can access, what it can reach, and how much it can spend. For self-hosted Agents, the user is simultaneously the system administrator, so these configurations must be actively completed at the start of deployment rather than relying on the Agent to constrain itself. An AI Agent that can truly "go to work" must have permission management first. This may be the most easily overlooked yet most critical practical guideline in the entire comparison.

Key Takeaways