The Complete Guide to Building an AI-Powered Second Brain with Obsidian + Hermes Agent

From "Dead Files" to "Living Files": A Knowledge Management Revolution in the AI Era

As AI Agents gradually integrate into daily workflows, a long-overlooked problem is coming to the surface: the vast majority of our files are "dead files."

A "dead file" refers to a file that an AI agent cannot access, cannot process, and cannot use as context. Whether it sits on a disconnected hard drive or quietly resides in Google Drive, as long as a powerful AI agent cannot easily read it, such a file is essentially wastepaper. This distinction reflects a fundamental shift in information architecture in the AI era: in traditional knowledge management systems, the value of a file is determined by human readability—a PDF, Word document, or scanned image only needs to be comprehensible to a human. But with the rise of AI Agents, the value dimension of a file has gained a new axis: Machine Accessibility.

This concept is closely tied to RAG (Retrieval-Augmented Generation) technology. RAG is a core component of today's mainstream AI application architectures—the knowledge of a traditional large language model is "frozen" in its training weights and cannot be dynamically updated; RAG achieves dynamic expansion of the model's knowledge by retrieving relevant snippets from an external knowledge base at inference time and injecting them into the context. Its technical pipeline typically includes: document chunking → embedding → storing in a vector database → cosine similarity retrieval at query time → concatenation into the prompt. Markdown format holds an inherent advantage in this pipeline: heading levels (#, ##) naturally provide semantic boundaries, making chunking easier; plain text with no formatting noise reduces embedding errors; and bidirectional links provide an explicit semantic graph that can assist graph retrieval (Graph RAG). In contrast, PDFs require OCR processing, Word documents require format parsing, and scanned images are a hotbed of noise. This is precisely why RAG systems require documents in the knowledge base to be in a structured, indexable, and chunkable format, and Markdown—with its plain-text, semantically clear nature—becomes the top choice.

In contrast stands the "living file"—a file that an AI agent can use as a skill reference, background material, or part of a prompt. For a file to become a "living file," it must enable an AI agent to help you complete tasks more efficiently. And the core tool combination for achieving this transformation is precisely Obsidian + Hermes Agent.

The true value of this approach lies in redefining the goal of knowledge management: it's no longer about humans organizing everything neatly for their own reference, but about building a knowledge infrastructure that both humans and AI can understand and collaborate on.

Why Choose Obsidian + Hermes Agent

The core advantages of this combination manifest on three levels.

Visualization Brings a Real Sense of Control

Obsidian is a knowledge management tool based on local Markdown files, founded by Shida Li and Erica Xu in 2020. Its core design philosophy is "Local-First"—user data should be stored in an open format on devices under the user's own control, with network syncing being an add-on feature rather than a necessary dependency. Unlike cloud-based knowledge bases such as Notion or Confluence, Obsidian stores all notes as plain-text .md files. This "Local-First" philosophy has gained new strategic value in the AI Agent era: when an Agent needs to read and write notes directly via the file system API, local .md files are more efficient than cloud API calls, with lower latency and a simpler permission model. The AI Agent can read and write directly through the file system API, without needing to call proprietary APIs or handle complex data format conversions.

Its Graph View builds a knowledge network by parsing bidirectional links [[filename]]—the underlying mechanism is real-time parsing of [[link]] syntax across all files to construct a directed graph data structure. This aligns closely with the technical principles of a Knowledge Graph, giving the knowledge base an inherent structural foundation for AI retrieval and laying the data groundwork for future graph neural network-enhanced retrieval (GraphRAG).

Typically, AI agents operate on files that users never look at—you don't know what it created or what it modified. Once Hermes Agent is connected to Obsidian, with its clear graph view, connecting lines, and clean Markdown files, users can intuitively see what the agent is doing. This visibility transforms you from "blind delegation" to "effective orchestration," truly becoming the conductor of your workflow.

Deep Customization Without Touching Code

Hermes comes with about half of its skills built in, and the other half can be customized by the user. If you want to modify a specific skill, simply edit the corresponding Markdown file in Obsidian—the AI agent will immediately use the new version. The entire process requires no interaction with code, configuration files, or terminal commands.

This solves a core pain point of traditional approaches: skill configuration interfaces are not intuitive enough, causing most users to go months without maintaining them, so custom skills gradually become outdated and lose their value.

Continuous Knowledge Accumulation and Cross-Device Sync

Having to repeatedly tell the AI "remember this" or "save this" every time is a double waste of compute and time. By systematically accumulating knowledge in your second brain, you can easily reference specific files and completely avoid redundant research. With Obsidian Sync, the same file can exist simultaneously on your MacBook, phone, and the VPS running Hermes. Even if a device breaks or is lost, your knowledge base, standard operating procedures, scripts, and skills remain intact.

Core Mechanism: Persistent Memory and On-Demand Loading

There is a key technical misconception that needs to be clarified: an AI agent does not read the entire note library into memory—doing so would consume a massive number of tokens and overflow the context window.

What truly makes the system run efficiently is a two-layer structure:

• Core Memory (System Prompt): A small piece of critical information that is always loaded online, such as your name and your preferences for the AI's response style (e.g., "keep answers concise"). In Hermes, this is called the "Soul MD file."
• On-Demand Skills and Notes: The bulk of the knowledge content, which is only dynamically pulled into context when a relevant task is triggered.

This "persistent memory + dynamic retrieval" architecture is precisely the core concept of today's Context Engineering.

The reason Context Engineering became the central topic of AI engineering in 2024-2025 fundamentally lies in the "token window economics" of large language models. The context windows of mainstream models expanded from GPT-4's 8K to Claude 3.5's 200K and even Gemini 1.5 Pro's 1 million tokens, but a larger window does not mean you can pile on information infinitely—research shows that information suffers from "attention decay" in ultra-long contexts (Lost in the Middle), where the model's recall of content in the middle positions is significantly lower than at the head and tail. The concept of Context Engineering, popularized by Andrej Karpathy and others, is precisely a systematic response to this phenomenon: its core craft lies in placing the most critical task constraints at the beginning of the system prompt, arranging dynamically retrieved relevant knowledge in descending order of importance, and putting summaries of historical conversations rather than the original text into context. The "Soul MD file" is precisely the engineering implementation of this principle—it is a carefully human-curated core of high-density information that guarantees the highest attention weight during each inference.

Traditional "prompt engineering" focuses on optimizing the wording of a single conversation, whereas context engineering focuses on systematically designing the flow of information—which information should reside permanently in the system prompt, which should be dynamically injected when a task is triggered, and which should be recalled on demand via vector retrieval. The token window is a finite resource, and a reasonable context architecture can achieve a multi-fold improvement in task completion quality under the same token budget. Your notes are your context, and only a model combined with your personal context is truly useful to you—which also explains why tools like Obsidian have multiplied in value in the AI era.

Setup Process: Using an AI Agent to Configure an AI Agent

The most counterintuitive and also most valuable aspect of this entire setup process is: using an AI agent to set up another AI agent.

Preparing the VPS Environment

It's recommended to run the Hermes agent on a VPS, which is equivalent to handing over an entire cloud computer to AI management. Deploying the AI Agent on a VPS rather than a local machine involves several key engineering considerations: first is persistence—once a local machine shuts down, the Agent stops running, whereas a VPS provides 24/7 uninterrupted execution, suitable for handling long tasks that take hours; second is isolation—running the Agent on a VPS means it has an independent operating environment, so even if it performs destructive operations, it won't affect the host machine.

The technical difference between KVM (Kernel-based Virtual Machine) and OpenVZ has a substantial impact on AI Agent deployment. OpenVZ is OS-level virtualization, where multiple containers share the same kernel, meaning tenants cannot load custom kernel modules—features that many AI tools rely on, such as the FUSE file system and eBPF tracing, cannot be used in an OpenVZ environment. KVM, on the other hand, performs full virtualization at the hardware level, with each VM having its own kernel space and supporting nested virtualization, providing the technical guarantee for the Agent to run Docker containers and install arbitrary kernel extensions. For AI Agents that need to process sensitive data, KVM also provides stronger security isolation—even if some Agent task triggers a security vulnerability, its impact scope is confined within the VM boundary and won't penetrate to the host or other tenants. This is why serious AI Agent deployments typically choose KVM over the cheaper OpenVZ option. A KVM2-spec plan is usually sufficient to run multiple agents such as Hermes, OpenClaw, and Codex CLI simultaneously.

Letting the Agent Complete the Installation Autonomously

The most impressive part is that the entire installation process requires almost no manual intervention. Open PI Agent through CMux (an open-source, free terminal tool designed specifically for the AI agent era), let PI connect to the VPS via SSH, and then issue instructions in natural language: "Install Hermes Agent on this VPS with Root privileges."

PI Agent autonomously monitors the entire installation, handles dependencies and packages, and automatically troubleshoots and resolves any errors it encounters. You don't need to be a Linux expert or DevOps engineer—as long as you're clear about your goals and pair up with a powerful agent, you can easily surpass those who are still operating manually.

When it comes to model selection, the open-source route offers outstanding cost-effectiveness. GLM (General Language Model) is an open-source large language model series jointly developed by Tsinghua University's KEG Lab and Zhipu AI. GLM 4.6 is a version optimized for Agent tasks, with targeted enhancements in function calling, long-context processing, and Chinese comprehension. Calling it through OpenRouter is a noteworthy option—OpenRouter is a model aggregation API platform that allows developers to call dozens of models from different vendors through a unified interface and pay based on actual usage, avoiding the risk of being locked into a single vendor. PI also supports one-click switching to other models such as Kimi.

Obsidian Sync and Skill Visualization

After completing the Hermes installation, the next step is to connect Obsidian to the VPS, achieving full synchronization among your MacBook, phone, and cloud vault. There are two mainstream options to choose from:

• Obsidian Sync: The official solution, about $5 per month, with end-to-end encryption and simple configuration
• Syncthing: Open-source and free, with slightly more configuration steps, suitable for users with some hands-on ability

Once sync configuration is complete, PI Agent automatically handles tedious operations such as installing Obsidian Headless on the VPS and configuring a System service for continuous sync. The final result is impressive: a note created on the VPS syncs to the MacBook in less than half a second.

More importantly, all of Hermes's skills are rendered into the Obsidian vault in clean Markdown format, including detailed descriptions, code blocks, headings, quotes, and directly copyable shell commands, with automatically established semantic internal links between different skills. This visual, interactive management approach is precisely the concrete embodiment of context engineering moving toward practical application.

Hands-On Demo: The Slash Goal Feature

To demonstrate the system's real capabilities, we can use one of Hermes's most powerful features—/goal—as an example.

Example task: Scrape the top 50 most-viewed YouTube videos about Claude Code, pull the transcript text, and store each video's content as a separate Markdown file in the Obsidian knowledge base.

The essence of the /goal feature lies in its clear and verifiable end state. This embodies an important principle in AI Agent design: a clear Termination Condition.

The design quality of the termination condition directly determines the reliability of AI Agent task execution. One of the biggest pain points of early Agent frameworks (such as AutoGPT, which went viral in early 2023) was precisely the lack of a clear termination condition, causing the Agent to fall into infinite loops or stop prematurely. Modern Agent designs introduce the "Verifier" pattern: after the Agent believes a task is complete, it must verify through an independent verification subtask whether the result meets predefined success criteria, and if not, trigger a retry loop. This is fundamentally different from traditional script automation—a script ends once it executes its preset steps, regardless of whether the result meets expectations; whereas a goal-based Agent continuously evaluates the gap between the current state and the goal state until the verification condition is met. This design pattern shares the same lineage as the Goal-Oriented paradigm in reinforcement learning, and also echoes the specification verification ideas in Formal Methods, making it a key engineering path for Agents to move from "random execution" to "trustworthy execution."

Back to the example task: "the top 50 most-viewed videos" is a quantifiably verifiable termination condition—it can be programmatically judged through counting (==50) and sorting verification (whether ordered by view count in descending order). If only 45 videos are found, the task is not complete; if these are not the most-viewed, the task is likewise not complete. Therefore, it will keep working until the result fully meets the criteria—whether that takes 10 minutes, 30 minutes, or 5 hours.

Completing this task manually would take at least three or four hours, whereas with /goal it only takes about 10 seconds to describe the requirement, and everything else is handed over to Hermes Agent. This is precisely the leverage effect of AI agents in the truest sense.

Configuration Matters More Than Tools

Many people fail to gain substantial benefits from AI, often not because AI isn't powerful enough, but because they lack the right configuration or haven't thought through their use cases. The application scope of AI Agents is virtually boundless: calorie tracking, autonomous research, code development, podcast interview preparation... The key is whether you've built a knowledge infrastructure that AI can actually use.

This Obsidian + Hermes Agent solution is essentially a systematic upgrade of personal knowledge infrastructure—storing private data encrypted in a second brain that syncs across devices, then letting AI agents fully participate in management and execution. For users who want to truly boost efficiency in the AI era rather than stay at the level of "just chatting with AI," this approach is well worth in-depth study and practice.

Note: This article is compiled based on video demonstration content, in which specific plan choices, model evaluations, and the like carry strong personal preferences. Readers can flexibly adapt them to their own needs.

Key Takeaways