Stop Using Claude and ChatGPT? A Deep Dive into Hermes Self-Hosted AI Agent Architecture

An AI systems implementation consultant named Gui Costa argues in a YouTube video that if companies are still using Claude or ChatGPT for business tasks, they may be missing out on what AI-first companies have already achieved. His core claim: the problem isn't the model itself, but that companies use AI in too fragmented a way, lacking governance and unified context.

The Typical Dilemma of Enterprise AI Use Today

Gui Costa uses an org chart to describe the current state of most companies: directors, managers, and employees each use their own Claude or ChatGPT accounts on their personal computers. Even if the company pays centrally, everyone still has separate projects, tool connections, and skill configurations, with information fragmented across individual accounts.

This fragmented model creates a cascade of problems. First, the AI can't access complete company or project context, so answers tend toward the generic. Second, tool connections are piecemeal—Google Drive, Slides, website SEO, SEMrush and other tools can't work together in a unified view. Project collaboration is also difficult: everyone works in their own project and can't share context under the same account.

More critically, there are governance and security gaps. Everyone accesses the same information with no permission tiers; the process of employees entering passwords and sensitive data into AI is invisible; and managers can't confirm whether a piece of copy was AI-generated or human-written. Gui Costa emphasizes that the larger the company, the more serious these problems become.

The Core of the Hermes Approach: Agents, Second Brain, and VPS

The alternative architecture Gui Costa proposes isn't complicated: keep Claude and ChatGPT as the interaction interface, but build a company-level agent system behind them. Each agent has specific skills, context, and tool access permissions, serving the whole company like an employee.

Orchestrator Agent and Department Agents

In his recommended architecture, the top layer is an "Orchestrator" agent facing the board, which grasps the whole company picture, can access all documents, and can see what other agents and employees produce with AI. The next layer is divided by department—the sales agent connects to CRM, HubSpot, Pipedrive, Clay, Instantly, Apollo and other GTM tools; the operations agent connects to HubSpot but doesn't touch sales-specific tools; each agent has only the minimum permissions and context needed to do its job.

The key to this design is permission isolation: the sales team doesn't need access to employee salary data, and the operations team doesn't need to see all sales-side details. Agents connect to company tools via API or MCP, and meeting transcription tools (such as Granola, tl;dv) are also connected to agents rather than scattered across individuals.

MCP (Model Context Protocol) is an open protocol launched by Anthropic in late 2024 to standardize how AI models connect to external data sources and tools. Think of it as the "USB interface" of the AI world: whether the backend is local files, a database, a CRM, or third-party SaaS, as long as it implements an MCP server, the model can discover and invoke these capabilities in a unified way. When the article says agents "connect to company tools via API or MCP," it means agents don't need to develop separate integration logic for each tool but can reuse existing connectors in the MCP ecosystem, lowering integration and replacement costs. MCP itself is model-agnostic; Claude, GPT, and others can all be used as clients, which is one of the technical foundations for the model decoupling the author emphasizes.

Second Brain: Using GitHub to Carry Company Context

Gui Costa repeatedly emphasizes the concept of a "second brain"—a knowledge base storing all company context, covering customers, departments, employees, and more. Without a second brain, an agent is just a generic chatbot; with it, AI output can move from generic to specific.

His actual approach is to use private GitHub repositories to carry the second brain. The reasons: GitHub is free, agent-friendly, and easy to govern. Different repositories correspond to different permissions: the company-level brain is accessible only to the orchestrator agent, the sales agent has its own separate repository, and the backup agent is responsible for syncing the orchestrator's output back to the main repository. This design also brings antifragility—if Claude or ChatGPT raises prices, degrades, or shuts down in the future, the company context remains safely stored in GitHub and the model can be switched at any time to continue the conversation.

"Second Brain" originates from the personal knowledge management field, referring to externalizing and structuring scattered information to reduce memory burden and support retrieval and reasoning. In the enterprise AI context, it refers to a knowledge base that centrally stores company facts, processes, customer information, and decision records, serving as a retrieval-augmented (RAG) source for agents. Before each answer, the agent retrieves relevant context from the second brain, then combines it with model capabilities to generate results, thereby avoiding "generic answers." Its quality depends on how the content is organized—clear structure, appropriate granularity, and continuous updates, otherwise what's retrieved is just noise. The GitHub repository approach in the article is essentially using version control to manage this knowledge asset.

VPS: The Runtime Foundation for Agents

Agents need an always-on cloud computer. Gui Costa recommends using a VPS and running multiple agents in it via Docker. His resource estimates are fairly specific: each agent typically needs 2 to 2.5 GB of memory; the KVM 1 spec can run 1 to 2 agents, KVM 2 can run 3 to 4, KVM 4 can run 8, and KVM 8 is for companies with many agents and intensive tasks. Small companies should choose KVM 2; mid-sized companies should choose KVM 4 or KVM 8.

On hosting provider choice, he mentions that he and his clients use Hostinger, and explains the installation process: choose server location, Ubuntu system, set root password, enable Docker Manager and malware scanning. Note that this is the video author's personal recommendation; hosting providers differ in price and specs, and readers should evaluate based on actual needs.

Docker is a containerization technology that packages applications and their dependencies into lightweight, portable containers that run isolated from each other on the same server. "Running multiple agents via Docker" in the article means each agent has an independent runtime environment and resource quota, so when one agent crashes or has a security vulnerability, it's less likely to affect other agents, and it also facilitates version rollback and batch deployment. For self-hosted AI agents, container isolation is also part of the permission boundary: different agents can mount different repositories and hold different keys. The cost is the extra learning required for container orchestration, image updates, and log management—exactly the hidden operational costs the article reminds SMBs to evaluate.

Why Choose Hermes Instead of Building Agents Inside Claude

Gui Costa acknowledges that Claude, ChatGPT, and Grok can all create agents today. But he advocates using solutions like Hermes or OpenClaude, with the core reason being model independence: the agent's intelligence layer should not be bound to a single LLM. If both the agent and the second brain are built inside Claude, the company will be at a disadvantage once Claude changes in price or capability. By using Hermes to decouple the agent layer from the model layer, switching costs are greatly reduced.

This view has merit, but it should also be viewed objectively: self-hosted solutions mean the company must bear responsibility for operations, security hardening, and troubleshooting itself, and for SMBs lacking a technical team, the hidden costs cannot be ignored. The video doesn't discuss this in depth.

Calling Agents Directly in Slack and Other Communication Tools

After deployment, company members don't need to open Claude or ChatGPT; they can talk directly to agents in Slack, WhatsApp, Telegram, Microsoft Teams, or Google Chat. Gui Costa shows his own company's Slack channels: divided by function such as SEO and sales, each team can only access the corresponding channel, and agents respond within the channel. He also demonstrates an ad placement agent that can create, pause, and resume ad campaigns in Slack, fully visible to the team.

This approach of embedding AI into existing workflows is closer to real collaboration scenarios than requiring employees to switch to a new chat window. Agents are no longer personal tools but shared "colleagues" for the team.

A Calm Assessment Before Implementation

Gui Costa's solution provides a clear enterprise-level AI architecture idea: agent layering, permission isolation, centralized context, and model decoupling. For teams already using Claude or ChatGPT but struggling with generic results, this framework is worth referencing.

Before implementation, however, there are several questions to answer yourself: Does the company have the basic capability to maintain VPS and Docker? Can the second brain's permission design withstand auditing? How should data boundaries be defined after agents connect to CRM and email? And as the number of agents grows, who is responsible for version management of prompts and skills? The video provides the architectural skeleton, but governance details still need to be filled in according to the company's actual situation.

Tool choice is never the end point. The value of Hermes or similar solutions lies in forcing companies to seriously think about organizational context and permission structures in the AI era—which may be more important than switching tools.

Background Supplement

Common security risks of self-hosted AI agents deserve separate mention: first, prompt injection, where external documents or web pages may contain malicious instructions that induce agents to leak context or perform unauthorized actions; second, key management, where agents connecting to CRM and email need credentials, and if these are written in plaintext in container configurations, a repository or server leak means total compromise; third, lack of auditing, where if agents autonomously perform actions such as pausing ads or sending emails without logs, accountability becomes difficult. Mitigation measures include assigning least-privilege tokens to each agent, setting human confirmation steps for tool calls, writing operation logs to append-only storage, and regularly rotating credentials. These measures should be incorporated at the architecture design stage rather than remedied afterward.