Distributed tracing is an essential process in the modern world of cloud-based applications. Tracing tracks and observes each service request an application makes across distributed systems. Developers may find distributed tracing most prevalent in microservice architectures where user requests pass through multiple services before providing the desired results.
Contents
- Distributed Tracing
- A Closer Look at Spans in Distributed Tracing
- Span Composition
- Differences Between Spans and Traces
- A Quick Summary of Spans in Distributed Tracing
- Advantages of Spans and Distributed Tracing
Distributed Tracing
Developers can acquire a comprehensive perspective of their software environment by combining distributed traces, metrics, events, and logs to optimize end-to-end monitoring and operations. Spans serve as the fundamental blocks in distributed tracing and represent the smallest measure of work in the system.
DevOps engineers can set up distributed tracing across their operations by equipping their digital infrastructures with the necessary data collection and correlation tools, which should apply to the whole distributed system.
The collected system data gives insightful information while offering the earliest signs of an anomalous event (e.g. unusually high latency) to drive faster responses.
A Closer Look at Spans in Distributed Tracing
A trace comprises a combination of spans, with each span serving as a timed operation as part of a workflow. Traces display the timestamp of each span, logging its start and completion. Timestamps make it easier for users to understand the timeline of events that run within the software. Spans contain specific tags and information on the performed request, including potentially complex correlations between each span.
Parent Spans
The parent, or root spans, occur at the start of a trace upon the initial service request and show the total time taken by a user request. Parent spans contain the end-to-end latency of the entire web request. For example, a parent span can measure the time it takes for a user to click on an online button (i.e. user request) for subscribing to a newsletter. During the process, errors and mistakes may occur, causing parent spans to stop. These spans branch out to child spans, which may divide into child spans of their own across the distributed system. It is important to note that parent spans may finish after a child span in asynchronous scenarios.
Detailed visualization of parent-child references provides a clear breakdown of dependencies between spans and the timeline of every execution.
Developers should refer to every span – parent/root and subsequent child spans – in distributed tracing to gain a comprehensive breakdown of request performance throughout the entire lifecycle.
Span Composition
Every span contains specific descriptors that comprise the function and details of logical work performed in a system. A standard span in distributed tracing includes:
- An operation/service name – a title of the work performed
- Timestamps – a reference from the start to the end of the system process
- A set of key:value span tags
- A group of key:value span logs
- SpanContext includes IDs that identify and monitor spans across multiple process boundaries and baggage items such as key:value pairs that cross process boundaries
- References to Zero value or causally related spans
Span Tags
Essentially, span tags allow users to define customized annotations that facilitate querying, filtering, and other functions involving trace data. Examples of span tags include db.instances that identify a data host, serverID, userID, and HTTP response code.
Developers may apply standard tags across common scenarios, including db.type (string tag), which refers to database type and peer.service (integer tag) that references a remote port. Key:value pairs provide spans with additional contexts, such as the specific operation it tracks.
Tags provide developers with the specific information necessary for monitoring multi-dimensional queries that analyze a trace. For instance, with span tags, developers can quickly home in on the digital users facing errors or determine the API endpoints with the slowest performance.
Developers should consider maintaining a simple naming convention for span tags to fulfill operations with ease and minimal confusion.
Span Logs
Key:value span logs enable users to capture span-specific messages and other data input from an application. Users refer to span logs to document exact events and timelines in a trace. While tags apply to the whole span, logs refer to a “snapshot” of the trace.
SpanContext
The SpanContext carries data across various points/boundaries in a process. Logically, a SpanContext divides into two major components: user-level baggage and implementation-specific fields that provide context for the associated span instance.
Essentially, baggage items are key:value pairs that cross process boundaries across distributed systems. Each instance of a baggage item contains valuable data that users may access throughout a trace. Developers can conveniently refer to the SpanContext for contextual metrics (e.g. service requests and duration) to facilitate troubleshooting and debugging processes.
Differences Between Spans and Traces
At its core, a trace represents a service or transaction under a distributed tracing structure. Spans represent a single logical structure within a given trace. Trace context is a significant component for traces within a distributed system as they provide components with easy identification through the use of unique IDs.
Implementation of a trace context typically involves a four-step process:
- Assigning a unique identification to every user request within the distributed system
- Applying a unique identification to each step within a trace
- Encoding the contextual information of the identities
- Transferring or propagating the encoded information between systems in an app environment
Traces capture the data of a user service request, including the errors, custom attributes, timelines of each event, and spans (i.e. tagged time intervals) that contain detailed metadata of logical work. Therefore, a trace refers to the execution path within a distributed system, while a span represents a single request within that execution path.
A Quick Summary of Spans in Distributed Tracing
Distributed tracing enables developers to track and observe service requests as they flow across multiple systems. A trace serves as performance data linked to a specific user request in a function, application, or microservice. Each trace comprises spans representing the smallest measurement of logical data and contains metrics that direct users to specific events.
Specifically, a trace is the complete processing of a user request as it moves through every point of a distributed system (i.e. multiple endpoints/components located in separate remote locations).
Spans in distributed tracing provide IT specialists with granular control over data transferred between multiple end-users, improving the monitoring and diagnostics of IT operations.
Advantages of Spans and Distributed Tracing
Modern digital operations involve complex technologies such as cloud, site reliability engineering (SRE), and serverless functions. Software managers and engineers typically accustomed to managing single services lack the technological capabilities to monitor system performance on such a scale.
As such, remote online processes involve multiple user requests passing through distributed tracing to different functions and microservices, resulting in increased system speed and reduced delays in transforming code into products.
Distributed tracing (and spans that serve as the essential logical measurement of work within these functions) optimizes observability strategies for developers within complex and remote app environments.
Combining distributed tracing and a good understanding and implementation of spans allow software teams to pinpoint challenges or faults when managing user requests from multiple endpoints for expedited troubleshooting. Some immediate benefits of a distributed tracing and span-based approach include:
- Improved user experiences that lead to a more favorable business reputation and outcomes
- Holistic management of software systems that minimize downtime for maximum efficiency
- Creation of a proactive software environment that gives the company an edge over other companies in the increasingly competitive digital landscape
- Accurate and responsive identification of user priorities so system managers can quickly determine the steps and measures to keep digital users/customers satisfied
Developers may implement distributed tracing through various methods with differing difficulties. Choosing a method depends on the user’s current programming knowledge, infrastructure, and skill sets. Building a distributed tracing system from scratch provides the most flexibility and customization.
At LogicMonitor, we help companies transform what’s next to deliver extraordinary employee and customer experiences. Want to learn more?