Acrux Core: Open-Source LLM Observability Platform, a Self-Hosted Alternative to LangSmith
Acrux Core is an open-source, self-hosted LLM observability platform that combines prompt management, tool catalogs, and feedback loops.
Acrux Core is an open-source LLM observability platform designed as a self-hosted alternative to LangSmith and Langfuse. It offers prompt versioning with dynamic tool binding, user feedback loops for iterative optimization, full-chain tracing with cost tracking, and native Python/TypeScript SDKs. Built for teams who prioritize data sovereignty and cost control, it provides lightweight Agent orchestration capabilities at the platform level.
When LLM Applications Meet the Observability Challenge
When building AI applications, many developers encounter the same recurring pain point: LLM Observability and Prompt Management. As call volumes grow, this problem becomes increasingly difficult to manage.
LLM observability draws from the three pillars of observability in traditional distributed systems—Logs, Metrics, and Traces—but with important adaptations for the non-deterministic nature of large language models. In traditional software, the same input typically produces the same output, making debugging relatively straightforward. LLM outputs, however, are stochastic—the same prompt can produce vastly different results at different times. Therefore, LLM observability needs to record not just input/output pairs, token consumption, and latency metrics, but also track prompt versions, model parameters (such as temperature, top_p), context window contents, and other dimensions directly related to generation quality. This field matured rapidly between 2023-2024 and is now considered a critical infrastructure layer for AI Engineering.
According to the project author's sharing on Reddit, most existing tools suffer from three notable shortcomings:
- Heavyweight SaaS platforms are costly: As call volumes increase, costs balloon quickly, making them unaffordable for small and medium-sized teams.
- Prompt iteration is inflexible: Dynamically adjusting prompts often requires redeploying backend code, elongating iteration cycles.
- Rigid tool-prompt binding: There's no way to freely combine arbitrary tools with arbitrary prompts without modifying code, limiting prompts' "agentic" capabilities.
To address these problems, the author built Acrux Core—an open-source platform for developers who want full control over their telemetry data and workflows. It simultaneously provides LLM observability, a Tool Catalog, and prompt management capabilities.
Acrux Core's Core Features Explained
Acrux Core positions itself as an open-source alternative to LangSmith and Langfuse, with a focus on self-hosting and data sovereignty. Its feature design is closely aligned with real-world development pain points.
Prompt Management and Dynamic Binding
Acrux Core allows developers to edit and version-manage prompts directly in the dashboard. This means prompt iteration no longer depends on backend code redeployment—product managers or operations staff can also participate in prompt optimization.
The core challenge of prompt management is that prompts are essentially a special artifact that sits between code and configuration. In early prototype stages, developers typically hardcode prompts in application code; but as applications move to production, prompts need frequent iteration—sometimes multiple adjustments per day to handle edge cases. If every modification requires a full CI/CD pipeline, iteration efficiency suffers dramatically. Mature prompt management systems typically need to support version control (Git-like diff and rollback capabilities), A/B testing, environment isolation (development/staging/production), and permission management (allowing non-technical staff to safely participate in optimization).
More noteworthy is its dynamic binding mechanism: you can mount tools from the tool catalog directly onto prompts without hardcoding execution flows in application code. This design gives prompts inherent "agentic" potential—the combination relationship between tools and prompts is abstracted to the platform level, significantly reducing coupling in building agent applications.
The Tool Catalog concept originates from the "Tool Use/Function Calling" paradigm in LLM Agent architectures. In this paradigm, LLMs no longer merely generate text but can decide when to invoke external tools (such as search engines, database queries, API calls, etc.) to complete tasks. OpenAI's Function Calling and Anthropic's Tool Use are specific implementations of this capability. The traditional approach is to hardcode tool definitions and invocation logic in code, meaning every new tool or adjustment to tool-prompt combinations requires code changes and redeployment. Abstracting tool definitions into a "catalog" with dynamic binding support is essentially declarative Agent orchestration—developers only need to declare "which tools this prompt can use" rather than imperatively writing invocation flows.
User Feedback Loop
Non-determinism is one of the biggest challenges in LLM applications, and real-world user feedback is often the best basis for optimizing prompts. Acrux Core provides a User Feedback Loop feature:
You can collect end-user ratings, thumbs up/down, and even correction content for specific prompt executions. This feedback is directly associated with execution records, allowing developers to continuously refine prompts in the UI based on these insights. This "execute—feedback—iterate" loop is exactly what many teams need most in production environments yet find hardest to build.
Human-in-the-Loop Feedback plays an irreplaceable role in LLM application optimization. Unlike traditional software where correctness is verified through unit tests, the "quality" of LLM output often requires human judgment. RLHF (Reinforcement Learning from Human Feedback) proved the value of human feedback during model training, and at the application level, collecting explicit end-user feedback (such as thumbs up/down, ratings) and implicit feedback (such as whether users adopted suggestions or manually modified outputs) is equally important. The key is precisely associating feedback data with specific execution contexts (which prompt version was used, what model parameters, what was the input), so that root causes can be identified and targeted optimizations made.
Full-Chain Tracing and Cost Tracking
For observability, Acrux Core supports complete tracing of nested LLM executions, covering function calls, token usage, and latency breakdowns across runs. For teams increasingly concerned about inference costs, the built-in LLM cost tracking capability is especially practical—it makes the cost of every call clearly visible, providing a data foundation for cost optimization.
LLM pricing models are typically based on token count, with enormous price differences between models—for example, GPT-4o's input token price is roughly half that of GPT-4 Turbo, while Claude 3 Haiku costs approximately 1/60th of Claude 3 Opus. In complex Agent applications, a single user request may trigger multiple rounds of LLM calls (chain-of-thought, tool calls, result summarization, etc.), with actual token consumption far exceeding what's visible on the surface. Cost tracking needs to solve several key problems: accurately counting input/output tokens for each call, correctly mapping to the corresponding model's pricing, supporting unified accounting across multiple providers, and attributing costs to specific feature modules or user groups, thereby providing data support for architectural decisions (such as when to substitute cheaper smaller models).
Developer-Friendly SDKs and Integration
Acrux Core provides native Python and TypeScript SDKs, both built on a clean REST API. This means even if your backend uses a different language, you can integrate directly via the REST API, offering solid cross-language integration capability.
For AI application developers, dual-language SDK coverage essentially meets the needs of today's mainstream tech stacks—Python serves data science and backend logic, while TypeScript covers full-stack and frontend scenarios.
Why Choose Open-Source Self-Hosting Over SaaS
The author offers a compelling explanation for the open-source philosophy: Debugging non-deterministic LLM behavior shouldn't come at the cost of sending all your prompts and trace data to a third-party vendor.
This viewpoint directly addresses the core concern with current SaaS observability tools—data sovereignty. Data Sovereignty concerns are particularly acute in AI application scenarios. When enterprises use SaaS-based LLM observability tools, all prompt templates, user inputs, model outputs, and execution traces are sent to third-party servers. For enterprises handling sensitive data (such as finance, healthcare, and legal sectors), this may violate compliance requirements like GDPR and HIPAA. Even in less regulated industries, prompts themselves may contain core business logic and competitive advantages—exposing them to third parties carries commercial risk. Self-hosted solutions fundamentally eliminate this concern by keeping data entirely within enterprise-owned infrastructure, while also avoiding the risk of Vendor Lock-in.
Acrux Core supports 100% self-hosting, runnable locally via Docker or deployable to your own infrastructure, with no data lock-in or forced dependency on third-party clouds.
Of course, for users who prefer a managed solution, the project also offers a complete cloud-hosted version, achieving a balance of "flexible choice": enjoy the convenience of managed hosting while maintaining full control over your telemetry data when needed.
Comparative Thoughts: LangSmith, Langfuse, and Acrux Core
From a market perspective, LLM observability is becoming a critical component of AI engineering infrastructure. LangSmith, Langfuse, and similar tools have already validated market demand, while Acrux Core differentiates with its open-source + self-hosted approach, targeting developers who are sensitive to costs and data sovereignty.
Its real highlight lies in integrating the tool catalog, prompt management, and feedback loop into a unified platform, rather than merely providing tracing capabilities. The dynamic binding of tools and prompts effectively provides lightweight Agent orchestration at the platform level—an approach worth watching.
As a project still under active development, Acrux Core is currently best suited for early adopters willing to experiment and contribute. The author has explicitly stated that community early feedback, code contributions, and feature requests are welcome. For teams struggling with LLM application observability but unwilling to be locked into SaaS platforms, this is a new open-source option worth adding to the evaluation list.
Project repository: https://github.com/AcruxCore/AcruxCore
Related articles
How to Interview Engineers in the AI Era: Practical Insights on Restructuring the Interview Process
When AI coding tools render traditional algorithm interviews ineffective, how should teams restructure? Insights from a year of practice on evaluating systems thinking, problem decomposition, and human-AI collaboration.
AI Agent Observability: A New Paradigm for Production Debugging and Hallucination Governance
Deep dive into AI Agent observability tools for production debugging and hallucination governance, covering full-chain tracing, semantic evaluation, and continuous improvement strategies.
How Theoretical Physicists Can Efficiently Get Started with Machine Learning: Optimal Paths and Resource Guide
A systematic guide for theoretical physicists transitioning to ML, covering math advantages, a three-stage learning path, classic textbooks, and physics-ML cross-disciplinary research directions.