AI Agent Observability in Backstage with Langfuse and OpenTelemetry

Why AI Agents Need Observability

As AI Agents move from experimentation into production, operations teams face a new challenge: how do you manage and monitor a fleet of agents at scale? Traditional logging and metrics tools don't do a great job of capturing an agent's decision chain, tool calls, or reasoning process. An open-source Backstage plugin is attempting to answer this question by embedding AI Agent observability directly into the developer platform.

The project (backstage-plugin-ai-agents) is a plugin for Spotify Backstage, designed to help teams centrally manage and observe an entire fleet of agents within their self-hosted Backstage instance. For organizations that have already adopted Backstage as their internal developer portal, this means no additional monitoring system is required — you get full visibility into agent health right from the familiar interface.

Technical Architecture: OpenTelemetry + Langfuse

The core design of this plugin relies on OpenTelemetry (OTEL) signals as the data collection standard. OpenTelemetry is the de facto observability specification in the cloud-native world, unifying three categories of telemetry data: traces, metrics, and logs. By having agents emit signals in OTEL format, the plugin gains the ability to decouple from any specific backend — in principle, it can integrate with any observability backend that supports OTEL.

First Supported Backend: Langfuse

According to the project documentation, the first available backend is Langfuse. Langfuse is an open-source observability platform focused on LLM applications, specializing in recording and analyzing traces of model calls, prompts, token consumption, and latency. Using Langfuse as the backend means developers can trace every model call within an agent's internal chain, rather than just seeing a black-box pass/fail status.

This combination of "OTEL for collection + Langfuse for visualization" reflects a mainstream architectural choice in current LLM engineering: use an open standard to collect data, then use a tool purpose-built for LLM analysis.

---

OpenTelemetry is governed by the CNCF (Cloud Native Computing Foundation) and emerged from the merger of OpenCensus and OpenTracing. Its core value proposition is "instrument once, consume anywhere": developers only need to integrate the OTEL SDK into their application code, and the resulting telemetry data can be forwarded via a Collector to Jaeger, Prometheus, Datadog, or any backend that supports the OTLP protocol — without touching business logic. For AI Agent use cases, OTEL's Trace concept is especially powerful. A single Trace can span the entire decision chain — "receive user request → call LLM → execute tool → return result" — with each step recorded as an independent Span, making an otherwise opaque agent reasoning process auditable and debuggable.

Langfuse's core data model is organized around three layers: Trace → Span → Generation. A Trace corresponds to one complete agent run; Spans record intermediate steps such as tool calls and retrieval operations; and Generation specifically captures each LLM inference's input prompt, output content, token usage, and latency. This level of granularity lets engineers answer production questions like "which step caused the agent to time out" or "which prompt template led to degraded output quality." Langfuse offers a self-hosted version (via Docker Compose or Kubernetes), which aligns well with this plugin's emphasis on self-hosted deployment — data never needs to leave your private environment.

What It Means to Integrate with Backstage

Backstage is Spotify's open-source internal developer platform framework, widely adopted by enterprises to build unified service catalogs, documentation portals, and tooling hubs. Building agent observability as a Backstage plugin — rather than a standalone application — delivers several practical benefits:

• Reduced context switching: Engineers can view services, documentation, and agent runtime status all in one portal.
• Reuse existing permissions and catalog infrastructure: Agents can be managed within Backstage's entity model just like microservices.
• Designed for self-hosted deployments: The project emphasizes self-hosted operation, making it more suitable for teams with data sensitivity or compliance requirements.

For teams actively pushing agents into production, this kind of plugin fills the observability gap that is so often overlooked.

---

Backstage's plugin system is built on a "frontend/backend plugin pair" architecture using React on the frontend and Node.js on the backend. Frontend plugins render custom pages or cards within the Backstage UI, while backend plugins can expose APIs through Backstage's routing and authentication layer, or aggregate data from third-party systems on the server side. Backstage also maintains a Software Catalog that manages organizational resources — services, APIs, teams, and more — using YAML-format entity descriptor files. Bringing AI Agents into this catalog means each agent can have a clearly defined owning team, dependency relationships, and lifecycle status, keeping them consistent with existing microservice governance workflows rather than floating outside the operational framework.

Use Cases and Limitations

This plugin is a good fit for engineering teams that are already using Backstage and have begun scaling AI Agent deployments. If your organization hasn't adopted Backstage, the cost of introducing a full developer platform just for this purpose would likely be too high.

Based on publicly available information, the project is still in early stages — OTEL as the collection layer is in place, but currently only Langfuse is available as a backend option. Given OTEL's open nature, extending support to additional observability backends is a natural and expected future direction. Teams looking to try it out should start by validating in a test environment that agent traces are properly reported and that complete chains are visible within Langfuse.

Summary

This open-source project demonstrates a pragmatic direction for AI Agent engineering: rather than reinventing the wheel, it connects a mature observability standard (OpenTelemetry), an LLM-specialized tool (Langfuse), and an enterprise-grade developer platform (Backstage). For teams focused on deploying agents in production, it offers a well-considered integration pattern worth referencing.