Configuring OpenTelemetry Logs in Rails: From Integration to Production

In modern distributed systems, observability has evolved from a "nice-to-have" to an absolute necessity. When a Rails application is split into multiple services or needs to collaborate with external systems, the traditional approach of relying solely on Rails.logger to write to files quickly falls short. OpenTelemetry (OTel for short), the observability standard led by the CNCF, is becoming the de facto standard for unifying the three signal types: Traces, Metrics, and Logs.

CNCF (Cloud Native Computing Foundation) is a sub-organization under the Linux Foundation, responsible for incubating and promoting key projects in the cloud-native technology ecosystem, including Kubernetes, Prometheus, Envoy, and more. OpenTelemetry is one of the most active projects within the CNCF, born from the merger of two earlier projects — OpenTracing and OpenCensus — with the goal of providing a vendor-neutral observability standard for distributed systems. OTel defines a unified API, SDK, and data protocol (OTLP), allowing developers to instrument once and export to multiple backends, avoiding vendor lock-in with any specific APM provider. OTel currently supports over a dozen mainstream programming languages, with Java, Go, Python, and JavaScript having the most mature implementations, while the Ruby ecosystem is rapidly catching up.

This article focuses on how to configure OpenTelemetry logs in Rails, covering the core concepts, common approaches, and practical considerations.

Why Introduce OpenTelemetry Logs in Rails

The traditional Rails logging system has several obvious pain points: logs are plain text with no structure; logs from different requests are interleaved and hard to correlate; and most critically, there's no inherent connection between logs and traces. When tracking a slow request across services, you often need to examine the Trace in your APM tool, then switch to the logging system to hunt for the corresponding log entries — an extremely inefficient workflow.

The core value of OpenTelemetry lies in unified signals. The three pillars of observability each serve different purposes: Traces record the complete call path of a request through a distributed system, composed of multiple Spans forming a call tree, where each Span represents an operation (such as a database query or HTTP call); Metrics are numerically aggregated data like request rates, error rates, and latency percentiles, ideal for alerting and trend analysis; Logs are event-level detailed records containing specific error messages, business context, and more. Each is valuable on its own, but their true power lies in correlation. By collecting logs through OTel, every log entry can automatically carry a trace_id and span_id, enabling bidirectional navigation between logs and traces. When you spot an anomalous Span in the Trace view, you can jump directly to all logs produced during that Span, and vice versa.

Core Configuration Strategy for OTel Logs in Rails

Adding Dependencies and Initializing the SDK

In the Ruby ecosystem, OpenTelemetry provides an official SDK along with a series of instrumentation gems. The typical approach is to add the following three core dependencies to your Gemfile:

• opentelemetry-sdk: Provides core SDK capabilities, including Span management and context propagation.
• opentelemetry-exporter-otlp: Exports collected data to a Collector or backend via the OTLP protocol. OTLP (OpenTelemetry Protocol) is the native data transport protocol defined by OpenTelemetry, supporting both gRPC and HTTP/protobuf transport methods. Compared to the custom data formats used by earlier backends (such as Jaeger's Thrift or Zipkin's JSON), OTLP provides a unified data model capable of carrying all three signal types — Traces, Metrics, and Logs. Nearly all major observability backends (Jaeger, Grafana Tempo, Datadog, New Relic, Elastic, etc.) now support direct OTLP ingestion, meaning your application only needs to export data via OTLP, and you can freely switch backends without modifying application code.
• opentelemetry-instrumentation-all: Automatically injects instrumentation into common components like Rails, ActiveRecord, Net::HTTP, and more.

Initialization is typically placed in the config/initializers directory, using OpenTelemetry::SDK.configure to set up the service name, resource attributes, exporters, and other configuration. One key point: the service name (service.name) should align with your service naming conventions, as it's a critical dimension for backend data aggregation.

Making Logs Carry Trace Context

Collecting logs alone isn't difficult — the real challenge is establishing the correlation between each log entry and its Trace. The core approach is to inject the context information of the currently active Span when outputting logs. OpenTelemetry provides an API for retrieving the active Span from the current context, from which you can extract the trace_id and span_id.

This relies on the Context Propagation mechanism — the foundation of distributed tracing. It solves a fundamental problem: how to pass trace_id and span_id across processes and services so that data produced by all participants can be correlated to the same Trace. OTel uses the W3C Trace Context standard by default, propagating context information via the HTTP headers traceparent and tracestate. Within an application, OTel maintains the currently active Span context through Thread-Local Storage. For Ruby's multi-threaded and Fiber concurrency models, context propagation requires special attention — the OTel Ruby SDK manages these lifecycles through the Context module. The reason logs can carry a trace_id is precisely because, at the moment a log entry is written, the active Span's identifiers can be retrieved from the current thread's context and injected into the log fields.

A common implementation approach is to customize the Rails.logger formatter, or use a structured logging library (such as lograge or semantic_logger) to append trace-related fields to log entries. lograge is a widely used log simplification library in the Rails community that consolidates the verbose multi-line logs Rails generates per request into a single structured line, significantly reducing log noise and making machine parsing easier. It supports output in JSON, Logstash, and other formats, and is often the first step toward structured logging in Rails projects. semantic_logger is a more comprehensive structured logging framework that supports multi-destination output (files, Syslog, Elasticsearch, etc.), thread-safe named loggers, automatic call context capture, and other advanced features. In OTel integration scenarios, both can serve as the foundation for the log formatting layer: by injecting trace_id and span_id as custom fields, structured logs inherently gain the ability to correlate with traces. The choice between them depends on your team's complexity requirements — lograge is lightweight and easy to adopt, while semantic_logger is more feature-rich but has a slightly steeper learning curve. Regardless of which approach you choose, the ultimate goal is the same: ensure that logs carry correlatable identifiers no matter which backend they're exported to.

Structured Logs and the Export Pipeline

Migrating from Text Logs to Structured Formats

Rails' default logs are in a human-readable text format, but for machine collection and analysis, structured formats like JSON are the ideal choice. Once logs are converted to a structured format, each field (timestamp, log level, message, trace_id, etc.) can be independently indexed and queried by the backend.

For production environments, it's recommended to adopt a unified JSON log format and use a log collection agent (such as OpenTelemetry Collector's filelog receiver, or Fluent Bit) to read, parse, and forward logs to the backend.

OpenTelemetry Collector is a core infrastructure component in the OTel ecosystem, serving as the intermediary layer for data collection, processing, and forwarding. It employs a pipeline architecture composed of three major modules: Receivers, Processors, and Exporters. Receivers are responsible for ingesting data from applications or other collectors, supporting multiple protocols including OTLP, Prometheus, Jaeger, and more. Processors can perform batch processing, sampling, attribute modification, filtering, and other operations. Exporters forward processed data to the final backend. By deploying a Collector as an intermediary agent, you can decouple collection logic from the application, achieve unified data governance, reduce resource consumption on the application side, and support flexible multi-backend fan-out.

Fluent Bit is the lightweight version of Fluentd, also part of the CNCF ecosystem, designed specifically for resource-constrained and high-throughput scenarios. It consumes minimal memory (typically just a few MB) while collecting, parsing, and forwarding log data at extremely high rates. In Kubernetes environments, Fluent Bit is typically deployed as a DaemonSet on every node, automatically collecting container stdout/stderr output. It supports a rich set of input plugins (tail, systemd, TCP, etc.), filter plugins (parsing, field modification, Kubernetes metadata injection, etc.), and output plugins (Elasticsearch, Loki, S3, OTLP, etc.). When used alongside OTel, Fluent Bit can serve as the first-tier agent for log collection, forwarding parsed JSON logs to OpenTelemetry Collector or directly to a backend.

This pattern of "applications are responsible only for producing structured logs, while collection is handled by external agents" effectively reduces application complexity and performance overhead.

Choosing Between Direct Export and Collection Agents

For the specific export approach, there are typically two paths:

• Direct export from the application: Using OTel's Logs SDK and OTLP exporter, logs are pushed from within the application process to the Collector. The advantages are centralized configuration and natural trace correlation; the downside is added load on the application process, and Ruby's Logs SDK is slightly less mature than its Traces counterpart.
• File collection + agent forwarding: The application outputs JSON logs to stdout or files as usual, and an independent Collector handles collection and forwarding. This approach better aligns with the cloud-native philosophy of "logs as data streams," offers better decoupling, and is the more commonly recommended production practice.

Practical Considerations

Based on community discussions and real-world deployment experience, several key points deserve attention.

SDK maturity: Compared to Traces and Metrics, the Logs signal implementation varies in progress across language SDKs. The Ruby ecosystem is continuously evolving, so before deploying, confirm the version and stability of the gems you're using, and avoid using APIs that are still in experimental stages in production.

Performance overhead assessment: Auto-instrumentation is convenient but introduces runtime overhead. In high-throughput scenarios, benchmark testing is essential. If necessary, disable instrumentation for components you don't need, keeping only those most valuable for troubleshooting.

Sampling and cost control: Log volume is often far greater than Trace data, and the cost of full collection and storage should not be underestimated. Sampling is a critical technique for controlling observability data volume and costs. OTel supports multiple sampling strategies: Head-based Sampling decides at the request entry point whether to collect the Trace — simple to implement but may miss valuable error requests; Tail-based Sampling decides whether to retain a Trace after it completes, based on its characteristics (such as whether it contains errors or exceeds latency thresholds) — it can more intelligently preserve valuable data but must be implemented at the Collector layer and has certain memory requirements. For log scenarios, sampling strategies are typically more flexible: you can filter by log level (e.g., only retain WARN and above in production), set different verbosity levels by service or module, or configure filter processors in the Collector to drop logs matching specific patterns. A well-designed sampling strategy can maintain troubleshooting capability while keeping storage costs within an acceptable range.

Summary

Configuring OpenTelemetry logs in Rails is essentially about incorporating logs into a unified observability system, enabling logs, traces, and metrics to corroborate one another. The core steps can be distilled into three: integrate the OTel SDK and instrumentation, make logs carry trace context, and export logs to a backend through an appropriate pipeline.

For Rails teams building microservices or looking to improve troubleshooting efficiency, the return on investing in this system is significant — you'll gain seamless navigation from Traces to logs, transforming fault diagnosis from "finding a needle in a haystack" to "following the thread." Of course, you also need to be mindful of the Ruby Logs ecosystem's maturity, runtime performance overhead, and storage costs, and choose the deployment approach best suited to your scale and stage.