LogicMonitor Query Language (LMQL) Reference for Advanced Metrics Widget
Last updated - 29 September, 2025
LogicMonitor Query Language (LMQL) extends the platform by giving you greater control and flexibility in how you query, filter, and visualize your data. LMQL uses a query-based approach, enabling you to analyze large data sets, customize visualizations, and extract meaningful insights.
The Advanced Metric widget leverages LogicMonitor’s LMQL to query for the data you want to visually represent in the widget. For more information, see Advanced Metrics Widget.
When you build the query, you start with an instant vector query, and use a combination of filters to narrow down the data and functions to perform operations or calculations for the data you need.
Instant Vector Queries
Instant vector queries are used to refine instant queries so they return only the most relevant datapoints. Instant queries themselves retrieve the latest single value of a specified datapoint (for example, cpuUsage
) across all instances where that datapoint exists.
Instant vector queries are paired with filters to limit the scope—for example, restricting results to certain resources, groups, or instances—making the output more targeted and efficient. Without filtering, these queries can generate very large result sets, since they pull one value per instance.
Filters for Instant Vector Queries
A filter is a mechanism for narrowing down which instances or datapoints are included in query results. Filters use property values, datasource names, instance names, or datapoints to select only the relevant time series data.
Recommendation: Use filters to limit result counts in instant queries and avoid overwhelming dashboards.
Each filter consists of the following:
- Filter name—Filter names can be the name of the property you want to use to limit the scope of the instant vector query.
- Filter operator—Operators are what you want to use to match the filter name to a value.
- Value—A value is appended after the operator and what you want the filter name matched to.
Multiple filters can be combined within {}
using commas, and an instance must satisfy all filter conditions to be included in the result set. This enables precise, flexible data retrieval tailored to specific use cases.
The following table describes the various filters you can use to further target your data with LMQL:
Type of Filter | Description | Example |
Operators | LMQL supports four main operators:
Recommendation: Use IDs or exact labels over loose regex when possible. | = (Equals):cpuUsage{system.displayname = "collector_1" != (Not Equals)cpuUsage{system.displayname != "collector_1"} =~ (Regex Match):cpuUsage{system.displayname =~ "collector.*"} !~ (Regex Not Match):cpuUsage{system.displayname !~ "collector.*"} Important: To leverage glob syntax, you must use the following: .* |
Property | Property filters use metadata and property values stored in LogicMonitor. You can combine multiple properties, separated by a comma (,). | cpuUsage{lm_pod_name="p02.us-west-2.prod"} cpuUsage{system.categories="tsdb", lm_pod_name="p02.us-west-2.prod"} |
Instance | Instance filters limit queries to specific instances. | cpuUsage{instance="cpu core-1"} cpuUsage{instance =~ "cpu core.*"} |
DataSource | DataSource filters restrict queries to a particular DataSource. Note: Supported operators are | cpuUsage{datasource="NetSNMPCPUwithCores"} |
Datapoint | Datapoint filters retrieve multiple datapoints in a single query. Note: Supported operators are | {datapoint =~ "cpuUsage|diskUsage", system.displayname="collector_1"} Note: The datapoint name can only be specified either outside or inside the {}, but not both. |
You can combine multiple filters for granularity. For example, the following query retrieve both CPU and disk usage datapoints, limits results to resources that belong to the Product group, and filters results to only those instances whose names start with “cpu core”:
{datapoint =~ "cpuUsage|diskUsage", system.groups = "Production", instance =~ "cpu core.*"}
Resource Aggregation Functions for Instant Vector Queries
Resource aggregation functions summarize values across multiple time series at a single point in time. The result is displayed as a single data point on the graph because the resource aggregation function returns a single value.
If you display the data as a graph, resource aggregation return one time series, with values for each time stamps. Displaying the data as big number or pie chart visuals—where only one value can be displayed—returns aggregated last sample value for end timestamp.
You can use the following basic syntax to build a query that contains an time series aggregation function:
function_name(<instant_query>)
LMQL supports the following resource aggregations:
Function | Description |
avg(q) | Calculates average for all time series returned by q |
min(q) | Finds the minimum value from all the time series returned by q |
max(q) | Finds the maximum value from all the time series returned by q |
sum(q) | Calculates sum for all the time series returned by q |
absent(q) | Returns a value of 1 if no series matches q Returns a value of 0 if any time series matches q |
count(q) | Counts time series returned by q |
For example, the following query returns the maximum CPU usage across all resource in pod p02.us-west-2.prod
:
max(cpuUsage{lm_pod_name="p02.us-west-2.prod"})
Group By with Resource Aggregation Query for Instant Vector Queries
Resource aggregation normally collapses multiple series into a single result (for example, max CPU usage across all hosts in a region), but you may need aggregated values per group rather than a single rollup. For that, LMQL supports by()
grouping modifiers with resource aggregation functions.
With by()
, you can group data byhost properties (such as pod name, application, or region). The function then applies separately within each group, returning one result per group. If multiple properties are specified, LMQL creates groups based on the unique combinations of those property values.
You can use the following basic syntax to build a query that contains a group by function:
function_name by(property_key [, property_key2, ...]) (instant_query)
For example, the following query returns the maximum CPU usage reported for each pod in that region:
max by(lm_pod_name) (cpuUsage{auto.aws.region="us-west-2"})
Recommendation: Use (by)
for structured rollups (for example, per pod, per app).
Nested Grouping
You can compose multiple resource aggregations so that an aggregation does a detailed breakdown within another aggregation that rolls the results up further. The aggregation doing the detailed breakdown is the inner aggregation, and the aggregation doing the rollup is the outer aggregation.
Note: For the outer step to find the grouping keys, every label in the outer aggregation must also display in the inner aggregation.
For example, the following query returns the average CPU per pod, after first averaging per (pod, application), restricted to us-west-2
:
avg by(lm_pod_name) ( avg by(lm_pod_name, application) (cpuUsage) )
The following is how the previous example queries data:
- The inner aggregation,
avg by(lm _pod_name, application)
, computes per-application averages within each pod. - The outer aggregation,
avg by(lm_pod_name)
rolls those application-level averages up to a pod-level average.
Note: The outer aggregation, lm_pod_name
is included in the inner aggregation set.
Arithmetic Expressions for Instant Vector Queries
LMQL supports binary arithmetic on instant vectors only. This lets you derive new, point-in-time values from existing datapoints (for example, compute free space, ratios, or percentages). You can also combine arithmetic with filters to keep results tight and relevant.
Note: Arithmetic expressions are not supported in the built-in Query Builder in the Advanced Metric widget.
LMQL support the following expressions:
Operator | Definition |
+ | Add |
- | Subtract |
* | Multiply |
/ | Divide |
Note: Arithmetic between two metrics is supported only when both datapoints come from the same DataSource.
For example, the following query returns data on free disk space:
StorageSize - StorageUsed
Recommendation: Narrow with property or instance filters first so arithmetic runs on the intended set and avoids high-cardinality explosions.
Range Vector Queries
A range vector query extends the concept of an instant vector query. Instead of returning just the most recent sample for each metric, it retrieves a range of samples over a specified duration back from the current timestamp. The duration is expressed in square brackets ([]
) after the query. Where a filter defines which data to query, a range vector query defines how much historical data to retrieve for each data.
For example, the following range vector query retrieves CPU usage samples from the last five minutes:
cpuUsage[5m]
Range queries override the global time window if a duration is explicitly specified. The boundaries are inclusive, so samples at the start and end of the range are included.
The following time units are supported with range vector queries:
Unit | Description |
ms | Milliseconds |
s | Seconds |
m | Minutes |
h | Hours |
d | Days Note: A day represents 24h. |
w | Weeks Note: A week represents 7d. |
M | Months Note: A month represents 30d. |
Y | Years Note: A year represents 365d. |
The LMQL range vector query returns up to 1,000 data samples per time series. If more are available, LMQL consolidates them into 1,000 points using the average (avg) function, similar to custom graph widgets.
Time Series Aggregation Function for Range Queries
Time-series aggregation functions take a range vector and collapse all samples in that window into one value per series. This helps to consolidate data samples for large duration of queries. The result is displayed as a single data point on the graph for each selected time series since the time series aggregation function returns a single value per time series.
You can use the basic syntax to build a query that contains an time series aggregation function:
function_name( <range-query> )
LMQL supports the following time series aggregations:
Function | Description |
avg_over_time(<range-vector>) | Calculates average value of all sampling data collected over a specified duration |
min_over_time(<range-vector>) | Finds the minimum value of all sampling data collected over a specified duration |
max_over_time(<range-vector>) | Finds the maximum value of all sampling data collected over a specified duration |
sum_over_time(<range-vector>) | Calculates the sum of all sampling data collected over a specified duration |
absent_over_time(series_selector[d]) | Returns 1 if the given lookbehind window d does not contain raw samplesReturns 0 for other scenarios |
present_over_time(series_selector[d]) | Returns 1 if there is at least one single raw data on the given lookbehind window d Returns 0 for empty results |
For example, the following query retrieves time series data for a specified time range:
sum_over_time( cpuUsage{system.displayname = "collector_1"} [4w] )
Offset Modifiers
By default, LMQL queries evaluate data from the current time. The offset modifier shifts the evaluation time backwards by a specified duration, enabling retrieval of historical data. It can be applied to instant or range vector queries. You can use the basic syntax to build a query that contains an offset modifier:
<instant_query | range_query> offset <duration>
The following describes how you can leverage the offset modifier in various scenarios:
Query | Description |
cpuUsage offset 10m | An instant vector query that evaluates CPU usage starting ten minutes in the past and plots 24 hours from that shifted point as a graph visual. When not used in a graph, the query displays the data as one sample value with a timestamp of ten minutes in the past. |
cpuUsage{system.displayname="collector_1"} offset 10m | A filtered instant vector query that returns CPU usage for collector_1 starting ten minutes in the past |
cpuUsage{system.displayname="collector_1"}[5m] offset 10m | A range vector query that retrieves five minutes of CPU usage, starting ten minutes in the past |
Note: Offset modifiers override the global time window and use the same time units as range vector queries.
Advanced Queries and Compositions
You can use advanced queries and composition in LMQL when you need control, accuracy, and complexity in retrieving information.
The following describes common types of advanced queries along with the use case where the query is helpful:
Type of Query | Query Example | Description | Use Case |
Aggregation of Aggregated Values | sum( max(cpu) ) + sum( max(fail) ) | Apply an aggregation over the result of another aggregation. | Rolling up already-summarized signals. |
Sum of Group-By Results | max by(system.displayname)(aliveTsf) + max by(system.displayname)(inactiveTsf) | Perform arithmetic (+ , - , * , / ) between two or more sets of group-by aggregations. Computes results by matching value of group by properties. | Comparing or combining aligned group-by dimensions (for example, per display name). |
Aggregation of Group-By | max( max by(lm_pod, system.displayname)( s3callsFailed{system.displayname=~"tsdb.*"} ) ) | Nest aggregations to extract a single “best” or “worst” per higher-level group. | Extracting a single “worst” or “best” case across all members of a group. |
Nested Grouping and Multi-Field Filtering | max by(lm_pod)( max by(lm_pod, system.displayname)( s3callsFailed{system.displayname=~"tsdb.*"} ) ) sum by(system.hoststatus)( max by(system.hoststatus, system.displayname)( s3callfail ) ) | Apply multiple group-by levels combined with label filtering. | First compute metrics at a detailed level (for example, by pod + display name), then collapse to a higher level (for example, by pod only). |
Literal Queries
LMQL can evaluate pure literals or mix constants with vectors when you need exactness over flexibility. This enables you to ensure LogicMonitor matches only what you type, exactly as you type it. Literal queries are helpful if you need to perform quick math, apply scaling factors or offsets to entire metrics, and mix constants with metric vectors for normalization, thresholds, or alerting logic.
The following describes literal queries that LMQL can perform with examples and outputs:
Type of Query | Query Example | Description | Output |
Pure Literal Arithmetic | 2 + 3 | Both operands are constants, with no time dimensions or labels. LMQL evaluates directly, similar to a calculator. Results are always one scalar. | Graph results: A flat constant line at value 5 Instant results: Scalar 5 |
Constant + Metric Vector | 2 + activeTsf | Adds 2 to each series in activeTsf .LMQL applies element-wise addition. For each sample in activeTsf , LMQL adds 2 .Labels on activeTsf are preserved. | Graph results: Each activeTsf series is added by 2, plotted as a line.Instant results: One scalar per time-series label, each being latest(activeTsf) + 2 .If activeTSf has the following series:system.displayname = "collector_1" → latest = 7system.displayname = "tsdb_1" → latest = 10Then: For collector_1 : 2 + 7 = 9 For tsdb_1 : 2 + 10 = 12 |
Metric Scaling by Literal | activeTsf * 7 | Each sample of activeTsf is multiplied by the constant 7 .Useful for unit conversion, rescaling metrics, or applying weight factors. Labels of the original metric are preserved. | Graph results: Scaled ines Instant results: One scalar per label, latest(activeTsf) × 7 If activeTSf has the following series: system.displayname = "collector_1" → latest = 7system.displayname = "tsdb_1" → latest = 10Then: collector_1 : 7 * 7 = 49 tsdb_1 : 7 * 10 = 70 |
Comparisons, Parenthesis, and Logical (Set) Operators
LMQL supports comparisons and logical (set) operations on metrics, literals, and expressions. These extend LMQL beyond raw values, enabling you to build conditional expressions, filters, and alert-like queries.
The following describes the types of operations you can leverage in LMQL:
Type of Operation | Description | Query Examples |
Comparison Operators | Compares each sample in a vector against either a scalar or another vector using the following:== , != , > , < , >= , <= You can leverage a boolean modifier to return time-series if condition is true ( 1 ) or false (0 ) for each point using the following:bool Recommendation: Use bool for converting results into 0 or 1. bool does not allow label matching flexibility. Comparison operators always return the value of the first operand if the condition is met. If the condition is not met, 0 is returned. | The following returns 1 if activeTsf > 100000, otherwise 0 :activeTsf > 100000 In the following, parentheses force (aliveTsf + inactiveTsf) to be evaluated first. Each sample of activeTsF is compared element-wise against the sum:activeTsf > (aliveTsf + inactiveTsf) The following returns 0 or 1 where up is exactly 0 :up == 0 The following returns series for all matching series, treating the comparison purely as boolean, even if vector labels differ: up == bool 0 If using bool keyword with a comparison operator, 1 is returned if the condition is met. The following returns 1 if the value of activeTsf is greater than 100. If the value is not greater than 100, then 0 is returned:activeTsf > bool 100 |
Parentheses | Parentheses () control the order of evaluation.Useful when mixing arithmetic with comparisons. Recommendation: Contain complex math with parentheses to make intent clear. | In the following, the average of active + inactive is computed first, followed by a check if the average is greater than aliveTsF :(activeTsf + inactiveTsF) / 2 > aliveTsF |