The primary goal of OpenTelemetry (OTEL) is to offer vendor-neutral ways of application instrumentation such that customers are able to switch between Telemetry backends. There are three main components of OpenTelemetry: OpenTelemetry SDK, OpenTelemetry API, and OpenTelemetry Collector.
In this blog, we will be covering the architecture, deployment, and best practices of consuming an OTEL collector.
What Is an OpenTelemetry Collector?
An OTEL collector is one of the core components of OpenTelemetry. OTEL collectors can optionally be deployed between instrumented applications and the Telemetry backends as a Telemetry pipeline component. With an OTEL collector, Telemetry signals can be ingested in multiple formats, translated to OTEL native pdata format, and finally exported out to backend native formats.
Of the three pillars of observability (metrics, logs, and traces), traces API is the most mature, while metrics and logs are in different stages of draft or development.
The OTEL Collector Architecture
The OpenTelemetry collector consists of three main components: receivers, processors, and exporters, which are then used to construct telemetry pipelines.
Receivers
Receivers get the data into the collector. Receivers can be of two types: push-based and pull-based. Generally, a receiver accepts data in a specified format, translates it into the internal format, and passes it to processors and exporters defined in the applicable pipelines. The format of the traces and metrics supported is receiver-specific.
Processors
Processors are used for processing the data before sending it out to the exporters. Processors can be used to transform the metrics or modify the name of the span. You can also do things like batching the data before sending it out, retrying in case the exporting fails, adding metadata, and tail-based sampling.
Exporters
Exporters are used to export data to multiple open-source backends and commercial backends. For example, a logging exporter can be used to export the data to the console output, while a file exporter can be used to dump the data to the file.
Traces, metrics, and logs pipelines can be set up by a combination of these three components. One more component of the Opentelemetry collector, which is not part of the pipeline, is an extension.
Extensions
Extensions provide capabilities on top of the primary functionality of the collector. They do not require direct access to Telemetry data and are mostly used for managing and monitoring an OTEL collector. Extensions are optional.
Service
The service section is used to enable the components which are configured within receivers, processors, exporters, and extensions sections. The service section consists of two sub-sections: extensions and pipelines.
The extensions consist of a list of all of the extensions to enable. Pipelines can be of traces, metrics, and logs type and consist of a set of receivers, processors, and exporters. Each receiver/processor/exporter must be defined in the configuration outside of the service section to be included in a pipeline.
An example configuration would look like this:
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
exporters:
otlp:
endpoint: otelcol:4317
extensions:
health_check:
pprof:
zpages:
service:
extensions: [health_check,pprof,zpages]
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
logs:
receivers: [otlp]
processors: [batch]
exporters: [otlp]OTEL Collector Best Practices
- The components defined under respective sections will be enabled only if they are added to the pipeline in the service section.
- The same receiver, processor, exporter, and/or pipeline can be defined more than once.
- For processor(s) referenced in multiple pipelines, each pipeline will get a separate instance of that processor(s). This is in contrast to the receiver(s)/exporter(s) referenced in multiple pipelines, where only one instance of a receiver/exporter is used for all pipelines.
- The order of processors dictates the order in which data is processed.
- At least one receiver and exporter must be configured in the pipeline.
- Extensions and processors are optional.
LogicMonitor’s OpenTelemetry Collector
LogicMonitor OpenTelemetry Collector comes with a wrapped version of an OTEL collector that is pre-configured to forward traces from your instrumented applications to the LogicMonitor platform. With LogicMonitor’s central management offering, users can install, upgrade, configure, and monitor an OTEL collector as a native resource in the LogicMonitor portal.
