What Is the OTel Collector? Architecture, Components, and Use Cases

The OpenTelemetry Collector, usually shortened to the OTel Collector, is a standalone, vendor-agnostic service that receives, processes, and exports logs, metrics, and traces between instrumented applications and the backends that store and analyze their data.

Its main job is consolidation. Instead of running separate agents for formats such as Jaeger and Prometheus, teams can use one Collector deployment to receive data from multiple sources and route it to one or more backends.

The Collector is one part of the broader OpenTelemetry project alongside its specification, the language-specific APIs and SDKs, and the instrumentation libraries. The API defines how code creates telemetry, while the SDK implements that API inside the application process.

OpenTelemetry graduated from the CNCF in 2026. The CNCF cited the project’s independent security audit and strong development activity, ranking it second in velocity among more than 240 cloud native projects.

Why Telemetry Pipelines Need a Collector

In distributed environments, telemetry tooling piles up in layers. A team adds a metrics agent for one backend, a log shipper for another, and SDK exporters that ship traces to a vendor endpoint. Each layer brings its own configuration, upgrade cycle, credentials, and resource footprint, all on the same hosts.

Direct SDK-to-backend export is brittle in a different way. Backend endpoints and credentials live in application configuration, so changing them can mean redeploying services. Retry and batching logic also runs inside the application process and competes with your workload for CPU and memory on the same host.

The Collector moves that work out of your application into a separate service. It can receive data from multiple sources, apply retries, batching, encryption, or even sensitive data filtering, and then send the data to one or more destinations.

Centralized collection also gives platform teams one place to set policy. Sampling rules, attribute scrubbing, and routing go into Collector configuration, so you edit one config set in place of hundreds of service configs. The same stream can feed several destinations, which helps during a backend migration or when different teams want to use different tools.

OpenTelemetry makes instrumentation vendor-neutral, so changing backends may only require changing an exporter. But that doesn’t eliminate backend lock-in: stored data, dashboards, and alert logic still depend on where telemetry lands. And the Collector only moves the data. If signals end up in separate backends, teams may still have to piece together service and infrastructure context during an incident.

Inside the Collector: Receivers, Processors, Exporters, Connectors, and Extensions

Five component types make up the Collector: receivers, processors, and exporters handle data in flight; connectors join pipelines together; and extensions add operational capabilities outside the data path. You configure each component by name, and the distribution you run determines which components are available to you.

Receivers

Receivers are the way in. They work in two directions: push-based receivers such as the OTLP receiver expose an endpoint that SDKs and other Collectors send to. The Prometheus receiver works the other way, scraping targets on a schedule.

Receivers for formats like Jaeger translate incoming data into the Collector’s internal representation, so downstream processors work on one model whatever the source. OTLP receivers can receive trace export calls via HTTP/JSON in addition to gRPC.

Processors

Processors work on data between receipt and export. They filter, enrich, batch, and sample it: memory_limiter refuses data when memory crosses a limit, and the attributes processor inserts, updates, or deletes attributes. The batch processor groups data for efficient export, and tail_sampling keeps or drops whole traces once it has seen their spans.

They run in the order you list them. The processor README recommends memory_limiter first, then sampling or filtering, then context-dependent processors such as k8sattributes, then enrichment. Batch goes at the end of the chain, partly because batching clears the request context.

Exporters

Exporters send processed data to one or more destinations. A single pipeline can list several. The components overview treats OTLP as the default because OTLP exporters emit OTel data without any loss of information and many tools accept it.

Format-specific exporters cover destinations that lack OTLP support. Many share a common exporter helper that adds a sending queue and retry, which govern behavior when a backend slows or goes offline.

Connectors

A connector bridges two pipelines. A connector consumes data as an exporter at the end of one pipeline, then emits that data as a receiver at the start of another, with the same signal type or a different one.

Deriving metrics from traces is a common use. The spanmetrics connector consumes spans in a traces pipeline and emits request, error, and duration metrics into a metrics pipeline. You get service-level metrics without adding metric instrumentation.

Extensions

Extensions handle work outside the data path. They are components for tasks not directly involved with processing telemetry data, such as health checks and profiling.

The health_check extension exposes an HTTP endpoint that orchestrators can call for liveness and readiness probes. The pprof extension exposes Go runtime profiles for CPU and memory investigations. zPages runs locally on port 55679 and shows live data from receivers and exporters.

How a Telemetry Pipeline Works

A pipeline is the path data takes through the Collector. Receivers accept data, optional processors transform it, and exporters forward it. Data moves through this chain in memory, according to the Collector architecture documentation.

Pipelines are typed by signal, and can operate on three telemetry data types: traces, metrics, and logs. Profiling is emerging as a fourth signal, but it is still in public Alpha; the Profiles announcement advises against using it for critical production workloads.

Fan-out lets one pipeline send each data element to several backends, with each exporter receiving a copy. Components can also be shared across pipelines. Because a shared receiver runs as one instance, a blockage can affect every pipeline it feeds. A slow exporter in a logs pipeline, for example, can stall metrics arriving through the same OTLP receiver. Separate receivers or Collector instances give each signal its own failure domain.

Collector vs. SDK vs. Agent

Vendor docs often blur these terms. The SDK runs inside the application process, generating telemetry through the OpenTelemetry API and exporting it, while the Collector runs as a separate service. That service receives, processes, and exports telemetry from many sources.

In OpenTelemetry’s agent deployment documentation, “agent” means the same Collector binary deployed close to the application, on the same host or as a sidecar. Language auto-instrumentation agents, like the OpenTelemetry Java agent, are different. That kind of agent attaches to the application runtime and injects instrumentation without code changes.

In practice, an SDK or auto-instrumentation agent produces the data, and one or more Collectors take over once it leaves the process.

Deployment Models: Agent, Gateway, and Combined

Topology decides where processing happens. It also determines how many Collectors you operate and how many egress points leave your network. The OpenTelemetry project documents two primary patterns, plus a hybrid that combines them.

Collector deployment patterns

On Kubernetes, a common setup pairs a DaemonSet Collector on each node for node and workload telemetry with a Deployment for cluster metrics and events. The Helm chart supports daemonset, deployment, and statefulset modes, and the OpenTelemetry Operator can handle upgrades and automatic sidecar injection. See the Kubernetes getting-started guide, Helm chart guide, and Operator guide for more information.

Distributions, Core vs. Contrib, and the Collector Builder

Collector distributions bundle different sets of components. Core contains a small set of broadly applicable components; Contrib offers a much wider range from the community and vendors. A practitioner account of building custom Collectors describes how components come from the core and Contrib codebases.

Contrib makes it easy to try many receivers and exporters, but unused components add to the Collector’s footprint and the code you need to secure and patch. The OpenTelemetry Collector Builder (OCB) lets you compile a distribution with only the components you need. That can produce a leaner binary, but your team must rebuild and test it as components and Collector versions change.

Configuring the Collector

A Collector configuration is a YAML file with top-level sections for receivers, processors, exporters, connectors, and extensions. Defined components stay idle until wired in. Components run once the service section adds them to a pipeline or lists them as active extensions.

Before you deploy

If you’re sending data to LogicMonitor, the OpenTelemetry Collector installation overview covers supported platforms and required permissions. To customize the configuration file before deployment, follow the installation wizard guide.

Scaling and Transforming Telemetry, plus Operational Trade-offs

In mixed environments, the Collector can normalize telemetry from different sources. For example, Jaeger spans from a legacy application and Prometheus metrics from infrastructure. Processors can rename attributes and add service or environment tags. For more complex changes, the transform processor uses the OpenTelemetry Transformation Language. Normalizing there keeps naming decisions in one place.

At scale, the agent-to-gateway pattern concentrates the expensive decisions in the gateway tier, where most heavy processing already runs. Stateless processing such as attribute edits and batching scales horizontally when you add gateway replicas behind a load balancer. Stateful processing is harder to scale.

Tail sampling is the clearest case. An OpenTelemetry post on tail sampling notes that all spans of a trace have to be processed in the same collector. Because that requirement leads to scalability challenges, agents typically use the load-balancing exporter to route spans by trace ID to one gateway instance.

The same constraint creates memory pressure. A practitioner write-up on tail sampling gives one example: “at 1,000 traces per second, a 60-second wait means 60,000 traces in memory simultaneously.” Buffering traces on disk can reduce memory use at the cost of higher CPU, and the results depend on your workload.

A well-tuned pipeline controls retention and routing. If traces, metrics, and logs land in separate backends, you can’t line up a sampled trace against infrastructure state at the moment of failure. Operators then rebuild that context across tools during incidents, and that reconstruction adds to resolution time.

Security, Management, and Troubleshooting

Receivers and exporters act as network servers and clients. The TLS configuration README says clients use TLS with server certificate verification by default; setting insecure: true disables the protection. Receivers can serve TLS once you supply a certificate and key, and hosting best practices recommend authentication extensions to restrict senders.

Sensitive data needs attention in two places. According to the configuration best practices, configuration files can hold API tokens and private keys, which you should move into a secret store. Inject them through environment variable expansion.

Telemetry itself can carry credentials in headers or personal data in attributes, and the redaction processor removes any attribute missing from its approved list.

Resilience comes from the health_check extension for orchestrator probes and from disk-based queuing. The exporter helper README explains that pointing sending_queue at a storage extension makes the queue persistent, with no in-memory queue when set. After a Collector restart, it picks up queued items and continues exporting.

Collector releases ship often. Many components remain below version 1.0, so pin your versions and read the release notes for breaking changes and critical fixes before you upgrade.

Most OTel Collector failures follow recognizable patterns. “Context deadline exceeded” means an export didn’t finish within its timeout, and common causes include an unreachable endpoint, a blocked port, or a slow backend. TLS handshake errors point to a plaintext-versus-TLS mismatch, an expired certificate, or an untrusted CA.

A 401 or 403 response means credentials are missing or rejected. If memory_limiter is dropping data, its README is clear that it “is not a replacement for properly sizing and configuring the collector.” An “unknown type” error means the configuration references a component missing from your distribution, as in this Contrib issue involving bearertokenauth.

When something breaks, work through it in this order:

1. Read the Collector logs at the failing component.
2. Check zPages and internal metrics to find the point where data stops moving through the pipeline.
3. Change one setting at a time.
4. Confirm the configuration validates before you restart.

Teams that would rather hand off this lifecycle can run a managed distribution instead of maintaining their own. LogicMonitor provides a pre-configured, wrapped version of the Collector that forwards traces from instrumented applications to the platform. You can manage it centrally from your LogicMonitor portal.

Bringing OTel Data Into a Unified View

The OTel Collector gives you a vendor-neutral pipeline for moving telemetry out of applications and infrastructure. The harder work starts after export. Consolidated telemetry still needs to turn into insight, and then into governed action.

LogicMonitor is the AI-first platform for Autonomous IT, built on a simple principle: you can’t automate across missing data. LM Envision ingests OTLP-standardized traces, metrics, and logs* (coming soon), giving teams a foundation for bringing telemetry together with infrastructure, cloud, and application context. Plus, Edwin AI analyzes telemetry and service relationships to help teams investigate root causes across signals and support governed workflows. Your team sets the boundaries for action.

The Collector moves telemetry into your platform. The work that follows is turning that data into action your team can trust.

FAQs