Cognee + Hermes in Practice: Giving AI Agents True Long-Term Memory

Most AI agents' memory is essentially just a rollback buffer—it remembers the current session, but once you close the terminal and come back, it no longer recalls your API rate limits or what database the backend runs on. Hermes's built-in memory handles short-term conversation fine, but it doesn't build a structured long-term knowledge picture. Cognee fills exactly that gap: treat Hermes as the brain, Cognee as the hippocampus, and the agent stops forgetting your world. This article documents the entire process from zero installation, through configuration, to real world stress testing.

Why Wire Cognee into Hermes?

The agent's default memory mechanism is like a rolling window: the current session's content is there, but as the conversation grows longer, earlier content gets summarized or discarded. That's fine within a single conversation, but when you close the terminal and come back the next day expecting it to still remember "the staging environment rate limit is 47 requests per minute" or "the backend runs on Postgres," it completely fails.

Hermes's built-in memory system can handle short-term information, but it can't build a structured long-term memory on its own—the kind of knowledge graph that treats "Alice," "engineering team," and "Project Falcon" as nodes with edges between them. Cognee is built precisely for this: it absorbs whatever you feed it, quietly builds a knowledge graph in the background, and lets retrieval walk along relationships instead of returning a pile of isolated text chunks.

The good news is that Hermes itself was designed with a memory-provider plugin system, and Cognee slots into it cleanly. The whole setup requires no GPU, no paid graph database, no cloud infrastructure—everything runs locally.

Knowledge Graph is a data model that organizes information in a graph structure: entities (such as people, teams, projects) as nodes, and relationships between entities (such as "belongs to," "owns," "depends on") as edges. Compared to traditional RAG, which splits text into isolated chunks and relies on vector similarity for retrieval, a knowledge graph allows retrieval to walk along relationship chains—for example, starting from "Alice" to find the "engineering team," then to the "Falcon project" that team owns. This lets the agent answer multi-hop questions that require reasoning about connections, rather than just returning the text fragment most literally similar to the query. The cost is higher construction overhead: entities and relationships must be extracted from raw text, which is exactly the work Cognee performs in the background using an LLM.

Preparation and Installation Steps

The Hermes installer checks whether the following dependencies are on your PATH:

Tool	Purpose
uv	Fast Python package management and virtual environments
Python 3.11	Runtime for Hermes and Cognee
Git	Cloning repositories and plugin discovery
Node.js	Required by parts of the toolchain
ripgrep	Fast in-repo search for the agent
ffmpeg	Media processing
LLM API Key	Only needed for local mode—Cognee uses an LLM to extract entities and relationships when building the graph

There's a noteworthy point about the last one you should know upfront: the LLM key is only needed when running Cognee locally. If you go the Cognee Cloud route, you don't need an LLM key; instead, you need a tenant URL and API key from the Cognee Cloud dashboard, and the tenant runs the models for you.

Step 1—Install Hermes:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

The installer will automatically run the dependency scan described above. Once everything passes, Hermes is installed.

Step 2—Install the Cognee plugin:

pip install cognee-integration-hermes-agent

That's the whole thing—once installed, Hermes will discover the plugin automatically. Upgrading uses the same command with the -U flag. Without -U, you're still running the old version. hermes cognee status will tell you whether version drift has occurred between your installed copy and the installed package.

Want to install from source? The integration code lives in Cognee's integrations monorepo:

git clone https://github.com/topoteretes/cognee-integrations.git
mkdir -p ~/.hermes/plugins/cognee
cp -R cognee-integrations/integrations/hermes-agent/. ~/.hermes/plugins/cognee/

A heads-up: this pulls in a long list of dependencies—vector stores, graph engines, LLM pipelines, and more. Don't panic when you see a wall of package names scrolling by.

Rule of thumb: when two tools conflict over a shared library version, an isolated environment beats forcing version alignment. Reproducibility over convenience.

Configuring a Model for Hermes

Hermes stores its configuration and secrets in ~/.hermes/.env. On the first attempt, PowerShell threw a path error because the directory didn't exist—a one-line fix, then write the file and fill in GOOGLE_API_KEY.

If you hit the same path error:

mkdir -p ~/.hermes

Running hermes model opens a menu with a large number of supported providers—OpenAI, Anthropic, Google, and more. The author chose Google AI Studio, pasted the API key, and selected the default Gemini model. After restarting, Hermes runs on Gemini (gemini-2.5-flash-lite).

Worth mentioning: switching models is completely decoupled from the memory layer. Gemini today, Opus tomorrow—Cognee doesn't care. It treats both the same underneath.

How the Memory Provider Plugin Works

Hermes discovers memory providers by scanning folders under $HERMES_HOME/plugins/ that contain a plugin.yaml plus a Python implementation. The YAML declares metadata and—crucially—which lifecycle hooks the provider implements. The Cognee plugin lives in ~/.hermes/plugins/cognee/, and the folder name is the provider's identity.

Here's what Cognee does at each key moment:

Moment	What Cognee Does
Session start	Ensures the local Cognee server is running—or connects to one a sibling plugin already started—and connects as a thin client
After each turn	Writes the turn into Cognee session memory. Fast, no graph building.
cognee_recall	Searches session memory first, falls back to the knowledge graph
cognee_remember	Writes persistent facts directly into the graph
cognee_forget	Handles deletion requests
Explicit Hermes memory write	Mirrored via on_memory_write
Session end	Runs cognee.improve() to bridge that session's turns into the permanent graph

That last row is the single most important design decision here. Cognee's most expensive operations—text chunking, LLM-based entity and relationship extraction, embedding, committing edges—are not executed inline on every turn. If they were, every turn would freeze while the graph is built. Instead, turns go into cheap session memory first, and graph construction happens all at once at the end, in a separate process—at session end, the plugin hands off shutdown to a detached worker that runs improve() to completion before deregistering from the server. Exiting Hermes returns immediately; you never sit in the terminal watching the graph build finish.

This ordering is mandatory: the local server runs in agent mode and auto-retires 60 seconds after the last agent deregisters. Deregistering before building would cut the promotion process off mid-stream.

Three connection modes:

Mode	When to Use	Communication Method
local-server (default)	COGNEE_BASE_URL unset, COGNEE_EMBEDDED unset	Ensures local server runs, connects as a thin client
remote	COGNEE_BASE_URL set	Thin client connects to Cognee Cloud or a self-managed instance
embedded	COGNEE_EMBEDDED=true	Runs Cognee in-process

The author initially assumed embedded would be a sensible default, but it isn't—for good reason. Cognee's local storage (SQLite, Kuzu/Ladybug, LanceDB) is single-writer. Driving them from the agent's background thread, or from a second Hermes process sharing the same data root, leads to "database is locked" errors and eventually data corruption. The local Cognee server is the single owner that serializes all access; the agent just makes HTTP calls. This is the same design used by the Claude Code and Codex plugins. Embedded is opt-in and only suitable for single-process or offline use.

Another design worth appreciating: there are no silent fallbacks. If COGNEE_BASE_URL is set but unreachable, or the local server fails to start, initialization throws an error rather than quietly switching modes. If initialization fails, memory stays off: Hermes logs the error and starts normally, and the provider refuses all calls rather than running a half-connected backend. You get a loud failure instead of discovering three weeks later that your memory has mysteriously vanished.

A shared brain across agents:

This is the author's favorite part. By default, the Hermes plugin joins the same memory as the Claude Code, Codex, and OpenClaw^Ad Cognee plugins: same dataset (agent_sessions), same local storage (~/.cognee/{data,system}), same server port (8011), and the same issued API key. Whichever plugin starts the server first, the rest connect to it. A fact you remember in Claude Code, you can recall in Hermes—and vice versa.

Want Hermes, or one Hermes profile, to be independent? Give it its own COGNEE_PLUGIN_DATASET. Full isolation requires its own COGNEE_DATA_ROOT / COGNEE_SYSTEM_ROOT and COGNEE_LOCAL_PORT. The server belongs to whoever reaches its port first, so private storage needs a private port.

Single-writer means a given storage allows only one process to hold write access at a time. Embedded databases like SQLite, Kuzu, and LanceDB, to simplify deployment, typically default to a single process exclusively owning the file or directory, and throw a "database is locked" error or corrupt data when concurrent writes are detected. When the agent's main loop, background threads, and another Hermes process all try to directly open the same data root, conflicts are almost inevitable. The solution isn't to add lock retries, but rather to introduce a long-running local server as the sole owner, routing all access through HTTP so it can be serialized. This is also the fundamental reason why local-server mode became the default and embedded mode is only optional.

Setting Cognee as the Memory Provider

Run:

hermes memory setup

Hermes lists available memory providers—built-in memory plus Cognee. After selecting Cognee, it asks how you want to connect, i.e. the three mode decisions from earlier presented as prompts. Local keeps everything on your machine—this is also the mode that requires an LLM API key, since Cognee needs a model to extract entities and relationships from text when building the graph. Cloud points at a hosted tenant: a base URL and a Cognee Cloud API key, and you don't need your own LLM key.

This choice gets written to ~/.hermes/cognee.json, and that file takes precedence over environment variables. A local setup writes an empty service_url into it, which is why exporting COGNEE_BASE_URL later won't switch you to cloud. Changing your mind requires re-running hermes memory setup, not editing variables.

Confirm the configuration took effect:

hermes memory status

Showing Installed, Available, Active—that's the whole handshake, and Hermes now routes memory and learning to Cognee.

Lifecycle hooks are agreed-upon points where the host program calls back into plugin code at specific moments, such as session start, end of a turn, or session close. Hermes declares which hooks a memory provider implements via plugin.yaml, then calls it at the corresponding moments. This is precisely why Cognee can attach "write cheap session memory each turn" and "run expensive graph promotion at session end" to different hooks without intruding into Hermes's core loop. Understanding the hook mechanism also helps with troubleshooting: if memory behavior is abnormal at some moment, it often means the corresponding hook wasn't triggered or initialization already failed.

Real Testing: Does It Actually Remember?

A configuration guide that claims to work proves nothing. The author actually tested memory recall—which is where most "memory" demos quietly cheat.

The input given to it was: "Use cognee_remember that my favorite editor is Helix."

It called cognee_remember and confirmed after 19 seconds. What happened next wasn't asked for—it turned around and called cognee_recall on itself to confirm the write landed. It returned "Your favorite editor is Helix," and more usefully, with a source tag: graph, dataset agent_sessions. It then wrote the same fact into Hermes's own memory store, this time taking 0.0 seconds because the fact was already there.

That source tag deserves a moment's thought. cognee_recall checks session memory first and only falls back to the graph when it's not found. So when it tells you the answer came from the graph, it isn't handing back the sentence you typed thirty seconds ago verbatim. The fact really is in the knowledge graph.

There are two write paths with different behaviors. cognee_remember is the persistent one—directly into the knowledge graph, effective immediately. Ordinary conversation turns don't do this; they fall into session memory and are only promoted to the graph by improve() at session end.

This means what you say offhand and what you explicitly ask it to remember aren't in the same place, at least not at first. There are two ways to prove the graph path was taken rather than accidentally testing a rollback buffer: first, use cognee_remember directly and read the source tag; second, take the slow route—fully close Hermes, give the detached worker a bit of time to finish session promotion, reopen, and ask after a cold start. The second is more convincing; the first takes just twenty seconds.

Cross-model verification: To show the memory layer is model-agnostic, the author switched Hermes to gemini-3.5-flash and told it: "our next product launch is September 12th." This triggered Cognee's full ingestion pipeline. When asked later, Hermes correctly recalled the launch date: September 12th. Different model, same memory, correct answer—that's the whole point.

Pitfalls and Practical Feedback

#	Problem	Fix
1	packaging 24.2 conflicts with packaging==26.0	Isolate Cognee in a uv virtual environment to avoid shared dependencies
2	PowerShell can't create ~/.hermes/.env because the directory doesn't exist	Run mkdir -p ~/.hermes first
3	First hermes session hangs for a few minutes after configuration	This is the one-time database migration on first local server startup. Use curl -s http://127.0.0.1:8011/health to tell when it's actually ready
4	Exporting COGNEE_BASE_URL to switch to cloud stubbornly stayed local	~/.hermes/cognee.json takes precedence over environment variables, and local setup wrote an empty service_url. Re-run hermes memory setup

None of these are fatal, but each one will cost you twenty minutes of head-scratching—now they're documented.

Directions to explore next:

• Shared brain: Install the Cognee plugin for Claude Code, Codex, or OpenClaw as well—they share dataset, storage, and port by default
• Scale the backend: When the graph outgrows local storage, switch to Postgres/pgvector, Neo4j, or Qdrant
• Feed it documents: Cognee can ingest files, JSON, and more—point it at your project docs and let the graph build itself
• Explore search modes: Cognee offers over a dozen retrieval strategies, from classic RAG to graph traversal completion
• Tune session-end behavior: COGNEE_IMPROVE_BACKGROUND controls where graph construction runs; the auto default hands it to a detached worker

FAQ

Does it need a GPU or paid cloud services? No. Everything runs locally, based on a local Cognee server—SQLite, embedded vector store, embedded graph engine. No GPU, no managed database, no cloud account. "Local" doesn't mean "in-process"—the default is a small local server at 127.0.0.1:8011, because Cognee's local storage is single-writer and the server is how a single owner is maintained.

What happens if Cognee fails to start? Memory stays off, and it's a loud failure. The provider never silently degrades—if a remote URL is unreachable or the local server can't start, initialization throws. Hermes logs the error and starts normally, and the provider refuses all calls. If you really want the in-process mode tradeoff, you need to explicitly set COGNEE_EMBEDDED=true.

Can I switch models without losing memory? Yes, and that's the core of the plugin's design. The memory layer is fully decoupled from the model.

Can multiple external memory providers run at once? No. Hermes enforces only one external memory provider at a time; a second registration is rejected with a warning. This prevents two backends from fighting over the tool schema and breaking each other.

Where does memory data live? Not in Hermes config files. In ~/.cognee/data and ~/.cognee/system, with the dataset agent_sessions—the same root as other Cognee agent plugins, which is the foundation of the shared-brain mechanism across Claude Code, Codex, OpenClaw, and Hermes. Because these roots live outside HERMES_HOME, the plugin reports them to hermes backup.

Memory gone after upgrading from an older plugin? It's not lost, just relocated. Version 1.0 changed three defaults: the local port went from 8000 to 8011, the default dataset went from hermes to the shared agent_sessions, and local storage defaults to ~/.cognee/{data,system}. Point COGNEE_PLUGIN_DATASET=hermes at the old data to recover.

One-line Summary

Using Cognee as Hermes's memory provider to achieve cross-session, cross-model long-term structured memory via a knowledge graph.

Paragraph Summary

The article documents the complete process of wiring Cognee into Hermes as an external memory provider. Hermes's built-in memory only covers the current session and can't build structured long-term knowledge; Cognee absorbs content into a knowledge graph in the background, letting retrieval walk along relationships. The setup defaults to local-server mode rather than embedded because Cognee's local storage is single-writer, and the server as sole owner serializes access to avoid locks and corruption. Expensive operations—LLM-based entity/relationship extraction, embedding, committing edges—aren't performed inline each turn, but are run by a detached worker at session end via improve(), keeping conversations fluid. The plugin also by default shares dataset and storage with Claude Code, Codex, and OpenClaw, forming a cross-agent shared brain. In testing, cognee_remember took about 19 seconds to write, and recall's source tag showed graph, verifying the fact truly entered the knowledge graph rather than a session buffer.