Deep Dive into Hermes Agent Architecture: Loop Mechanism, Memory System, and Gateway Design

As a continuously learning AI agent, Hermes has quickly become a standout project in the developer community thanks to its flexible gateway system and multi-layered memory architecture. This article provides a systematic breakdown of its core mechanisms based on an in-depth analysis of Hermes's source code and architecture, helping you understand how to use it and how to build similar agent systems.

Bird's-Eye View: Hermes's Overall Architecture

Hermes's architectural design is surprisingly clean. Its core consists of the following components:

• AI Agent Core: The central Agentic Loop, serving as the brain of the entire system
• Multiple Access Methods: CLI, Gateway (Telegram/Email/Slack, etc.), and API interfaces
• Built-in Services: Pre-installed Tools, Skills, and a multi-layered memory system

After installing Hermes, you get a ready-to-use set of tools and skills. What truly sets Hermes apart from ordinary chatbots is its memory system — divided into internal memory (conversation logs, soul.md, user.md, and other files) and external memory (third-party services like mem0, Super Memory, etc.). This dual-layer memory design enables the agent to continuously learn and evolve through ongoing interactions.

Agent Loop: How the Core Loop Works

Hermes's core loop is similar to minimalist agents like PiAgent and OpenCode, but with more thoughtful details. The loop kicks off every time a user sends a message.

The Agentic Loop is the most fundamental design pattern in modern AI Agent architecture. Unlike the traditional "request-response" model, an Agentic Loop allows the LLM to make multiple autonomous decisions after a single user input — it can repeatedly invoke tools, observe results, and decide on the next action until it considers the task complete. This pattern originates from the ReAct (Reasoning + Acting) paradigm, proposed by Yao et al. in 2022, with the core idea of having the model alternate between reasoning and acting. API designs like OpenAI's Function Calling and Anthropic's Tool Use were built specifically to support this kind of loop.

Step 1: Build Context

The system assembles the complete context from internal memory and preset prompts, including the system prompt, soul.md (agent personality), user.md (user profile), message history, and more.

Step 2: Send to LLM

The complete context and message history are sent to the large language model for inference.

Step 3: Tool Calling Loop

The LLM can decide to invoke tools (such as web search, file read/write, etc.), and the tool execution results are fed back to the LLM. This process continues until the LLM determines that no more tool calls are needed.

Step 4: Generate Final Response

After all tool calls are complete, the LLM outputs the final reply.

Step 5: Memory Update

This is the essence of Hermes — after each response, the system analyzes the current conversation turn, determines whether any information is worth remembering, and writes it to the memory system. This is the fundamental reason Hermes gets "smarter the more you use it."

Context Construction and Compression Strategy

Hermes's context is composed of several key Markdown files:

• soul.md: Defines the agent's personality, tone, goals, and behavioral patterns. It's empty upon initial installation and uses a default system prompt, but users are strongly encouraged to customize it
• user.md: A user profile file that Hermes automatically learns from and updates during conversations (e.g., your profession, work details, etc.)
• memory.md: A general-purpose memory file that stores tool usage methods, workflows, interesting facts learned during conversations, and more

Beyond these files, the context also includes past session summaries (requires external memory configuration), skill and tool descriptions, and recent message history.

Context Compression Mechanism

As conversations grow longer, context compression becomes critical. Hermes triggers compression by default when context usage reaches 50%, summarizing historical messages into a digest.

Compression checks occur at two points: before each message is sent and when the LLM returns a context window error.

For the first message, since there's no token usage data from the LLM yet, Hermes uses a clever approximation: total character count ÷ 4 ≈ token count. For subsequent messages, it uses the usage parameter returned by the LLM for precise token counting.

Some technical background on token estimation: LLM context windows are measured in tokens, not characters. A token is the smallest unit in the model's vocabulary. For English text, 1 token corresponds to roughly 4 characters or 0.75 words; for Chinese text, 1 Chinese character typically maps to 1-2 tokens. Context window sizes vary dramatically across models: GPT-4o supports 128K tokens, while Claude 3.5 supports 200K tokens. When input exceeds the window limit, the model returns an error or truncates the content. Therefore, accurate token counting is crucial for avoiding request failures, but running a tokenizer itself has computational costs, which is why the "character count ÷ 4" approximation is extremely common in engineering practice.

The compression prompt itself is quite rich, requiring the LLM to generate a summary covering multiple dimensions: overall goals, constraints, completed actions, active states, historical progress, current blockers, key decisions, resolved issues, relevant files, and more. Compared to Pi's minimalist compression prompt, Hermes's version is significantly more detailed, preserving more contextual information for the agent.

Gateway: The System That Connects Everything

The Gateway is arguably the feature that made Hermes truly break out — it allows your AI agent to communicate with you through platforms like Telegram, Slack, Email, WhatsApp, Discord, and more.

How the Gateway Works

Once started, the gateway runs an AsyncIO event loop that continuously listens for new messages across various messaging platforms.

AsyncIO is Python's asynchronous programming framework, implementing non-blocking I/O operations through coroutines and an event loop. In the gateway scenario, the agent needs to simultaneously listen to input from multiple messaging platforms. Using a traditional multi-threaded model, each platform connection would require a dedicated thread, resulting in massive resource overhead. AsyncIO uses a single-threaded event loop combined with the await keyword to efficiently manage hundreds of concurrent connections within a single thread. For a gateway system that needs to simultaneously handle Telegram polling, WebSocket long connections, and Webhook callbacks, this is the most appropriate technology choice.

Different platforms use different integration methods:

• Some use Webhook push notifications
• Some use polling (e.g., Telegram supports polling the API every second)
• Some use WebSocket long connections

These three message integration methods represent three classic communication patterns in distributed systems. Webhook is a "push" model — the platform proactively sends an HTTP POST request to your server when there's a new message. It offers the best real-time performance but requires a publicly accessible server address. Polling is a "pull" model — the client periodically sends requests to the platform API to check for new messages. It's simple to implement but introduces latency and wastes bandwidth. WebSocket is a full-duplex long connection — once established, both parties can send data at any time, combining real-time performance with efficiency, though it requires maintaining connection state. Telegram supports both Webhook and long polling modes, suitable for different deployment scenarios.

Each platform integration requires independent configuration (via the hermes setup gateway command), including Bot ID, allowed user IDs, and more.

Context Reconstruction and Session Management

The gateway receives individual messages, not complete conversations. Therefore, it needs to reconstruct the full message history from the SQLite database. The session ID is composed as: platform name + platform-returned Session ID + other identifiers.

The gateway also includes a built-in Session Manager that handles message concurrency scenarios: when the agent is still processing the previous message, new messages can be queued (default), interrupted (/interrupt), or steered (/steer).

Deep Dive into the Three-Layer Memory System

Hermes's memory system is one of its most distinctive design features, organized into three layers:

Layer 1: Markdown File Memory

These are the soul.md, user.md, and memory.md files mentioned earlier, always appended after the system prompt. This is the most direct form of memory.

Layer 2: SQLite Database Storage

Hermes stores the complete record of every interaction in a local SQLite database. The database contains multiple tables and data models, essentially serving as complete transcripts of all sessions. One noteworthy detail: the database also maintains a plain text table specifically designed for similarity search across historical conversations — a very practical design choice.

Layer 3: External Memory Services

External memory requires manual configuration and supports multiple providers including mem0, Super Memory, and Honcho. Each provider works differently — some use semantic similarity search, while others use LLMs to extract key memories.

Semantic similarity search is a retrieval technique based on vector embeddings. The principle is: text is converted into high-dimensional vectors through an embedding model (such as OpenAI's text-embedding-3-small), and semantically similar texts are closer together in vector space. During queries, the user input is similarly converted into a vector, and the most relevant historical memories are found through cosine similarity or Euclidean distance. Services like mem0 add an LLM extraction layer on top of this — first using a model to extract structured memory points from conversations, then vectorizing them for storage, thereby improving recall precision and interpretability.

An important detail: external memory queries happen after the first message. That is, when you start a new topic, the agent first responds to your first message, then queries external memory to try to anticipate what you might ask next — much like how humans recall related experiences after answering a question. Therefore, if the first message doesn't trigger memory recall, you can follow up in the second message, by which time external memory will have already been queried.

Cron Jobs: Scheduled Task Scheduling

Hermes's scheduled task system enables the agent to automatically perform periodic work, such as sending an AI news digest every morning or sending a work report to your boss every Friday.

Implementation Mechanism

Hermes's Cron doesn't rely on a system-level cron process. Instead, it maintains its own loop that executes once per minute, running a function called tick each time.

It's worth noting that the traditional cron in Unix/Linux systems is a daemon process that schedules tasks by reading crontab configuration files, using a five-field expression ("minute hour day month weekday") to define execution times. Hermes chose to implement its own scheduling loop rather than relying on system cron primarily for cross-platform compatibility (Windows doesn't have native cron) and more flexible task management capabilities (such as dynamically adding/removing tasks, task status tracking, etc.).

One interesting detail: although the official documentation claims Cron Jobs are stored in SQLite, actual source code analysis reveals that task configurations are stored in the .hermes/cron/jobs.json JSON file. During each tick, the system reads this file to determine whether any tasks need to be executed. Execution results are stored as Markdown files in the .hermes/cron/output/{jobID}/ directory.

Message Delivery Mechanism

Cron task results aren't delivered through the agent calling a "send message" tool. Instead, they're automatically sent to the messaging platform you've set as Home. When configuring the Gateway, the system asks whether to set a particular platform as Home, and Cron task notifications are delivered to that platform.

Summary and Takeaways

Although Hermes Agent's architecture doesn't have many components, each part is carefully designed: the clean Agent Loop ensures reliability, the multi-layered memory system enables continuous learning, the Gateway bridges everyday communication channels, and the Cron system grants autonomous action capabilities.

For developers looking to build their own agents, Hermes's architecture offers several design ideas worth borrowing: using Markdown files to manage personality and memory, using SQLite to store conversation history with full-text search support, approximating token counts with character count to reduce computational costs, and the strategy of asynchronously querying external memory after the first response. These are all highly practical techniques in engineering practice.

From a broader perspective, Hermes represents an important direction in AI Agent evolution: progressing from stateless conversational tools to continuously learning systems with memory, personality, and autonomous action capabilities. As context windows continue to expand and memory retrieval technologies keep advancing, agents like these will increasingly resemble true "digital assistants" — not just answering questions, but understanding you, remembering you, and proactively working on your behalf.