Advanced Search Operators

Advanced searches include aggregation, processing, and formating operators that can help you to refine and modify your search results. 

You can add advanced search operations to your query after filtering. Search operations work sequentially on a set of events, and the result from one operation is piped into the next one. Some operations work on an event at search time (parsing fields), others require a partial set (limit) or the full set (sort) of events to produce a result. You can view the results of an advanced search in the Aggregate tab.

Note: Advanced search operations are available for LM Logs Enterprise and LM Logs Unlimited customers.

Operator Description Type
as Displays the column name as the alias value. Formatting
avg Calculates an average of values in a specified source field. Returns one specific value. Aggregation
count Counts the number of events returned by a search, optionally grouped by one or more fields. Aggregation
limit Limits the results of an aggregation operation to a fixed number of results. Formatting
max Extracts the maximum value for a set of values for a specified source field. Returns one specific value. Aggregation
min Extracts the minimum value for a set of values for a specified source field. Returns one specific value. Aggregation
parse Extracts the contents of a specified source field into a new field based on a string pattern match. The new field is added as a field:value pair to the log metadata. If the source field is not specified, it defaults to the message field. Processing
sort Sorts the results of an aggregation operation by a set of fields, ascending, or descending. Formatting
sum Calculates the sum of the values of the specified field, optionally grouped by one or more fields. Aggregation
where Limits the result of an operation by a specified condition. Can be used for logs processing or aggregation. When used on processing it limits the results of the logs. When used on aggregation it limits the aggregated result. Processing or Aggregation

Note: The following examples begin with the filtering query for the wildcard * which searches for all logs. You can replace this wildcard with any filtering query. For more information, see Writing a filtering query.

Aggregation Operators

avg

Calculates an average of values in a specified source field. 

Syntax

* | avg(<FIELD>) (as<ALIAS>)
* | avg(<FIELD>) (as<ALIAS>) by <FIELD> (,<FIELD>)

Examples

Example 1: The following example searches through numbers in the 0-9 range in a four-digit sequence pattern, and calculates the average for the numbers found, if any. The average number will be extracted and added to the log event as a field called “number”.

* | parse / ([0-9]{4}) / as number | num(number) as number2 | avg(number2)

count

Counts the number of events returned by a filtering query, and optionally groups the events by one or more fields. You can also count events that contain specific fields.

Syntax

Count the number of events in the specified time range:

* | count

Count the number of events optionally grouped by one or more fields:

* | count (as <ALIAS>) by <FIELD> (, <FIELD>)

Count the number of events only if the event contains the field specified in the argument to count():

* | count(<FIELD>) (as <ALIAS>) by <FIELD> (, <FIELD>)

Examples

Example 1: Count the number of events for each unique resource and show the first 15 results.

* | count by _resource.name | sort by _count desc | limit 15

Example 2: You can count the number of unique values for a field by passing the results of one count operation into another count. The following example counts the number of unique resource names sending logs during the specified time range.

* | count by _resource.name | count

max

Extracts the highest value for a set of values for a specified source field. Returns one field if no “by <FIELD>” is specified. Otherwise, an average for each of the specified fields is returned.

Syntax

* | max(<FIELD>)

Examples

Example: 1 The following example searches for numbers in the 0-9 range in a four-digit sequence pattern. If found, the highest number is extracted and added to the log event as a field called “number” .

* | parse / ([0-9]{4}) / as number | num(number) as number2 | max(number2)

min

Extracts the lowest value for a set of values for a specified source field. Returns one field if no “by <FIELD>” is specified. Otherwise, an average for each of the specified fields is returned.

Syntax

* | min(<FIELD>)

Examples

Example: 1 The following example searches for numbers in the 0-9 range in a four-digit sequence pattern. If found, the lowest number is extracted and added to the log event as a field called “number” .

* | parse / ([0-9]{4}) / as number | num(number) as number2 | min(number2)

sum

Calculates the sum of the values of specified fields, and optionally groups the calculations by one or more fields.

Syntax

Counts the number of events in the specified time range:

* | sum(<FIELD>) (as <ALIAS>) by <FIELD> (, <FIELD>)

Examples

Example 1: Calculate the sum of the size field for each unique resource name, and sort the results.

* | sum(_size) by _resource.name | sort by _sum desc

Processing Operators

parse

Extracts the contents of a specified source field into a new field based on a string pattern match. The new field is added as a field:value pair to the log metadata.

Syntax

* | parse <SRC-FIELD> <PATTERN> as <FIELD1> (, <FIELDN>)
  • If the source field, <SRC-FIELD>, is not specified, the message field is used by default.
  • <PATTERN> is a string with wildcards, or a Regex.
  • If parsing multiple fields, the number of field names must match the pattern.

Examples

Example 1: The following example uses a wildcard to search words containing “unix” in the message field and extracts the parts before “unix”. If found, the extracted words are added to the log event as a field called “unix”.

* | parse "* unix" as unix

Example 2: The following example uses a Regex search. The query looks for the phrase “Failed to pull image” in the message field, and anything that is not a space, tab, or newline. If found, the image name will be parsed out and added to the log event as a field called “image”.

"Failed to pull image" | parse /Failed to pull image "(\S+)"/ as image

You can then search for a specific image using the query image=<imagename>.

where

Limits the result of an operation by a specified condition. Can be used on logs processing, or on aggregation. When used on processing it limits the results of the logs. When used on aggregation it limits the aggregated result. If a source field is not specified, the operation defaults to the message field.

Syntax

Extracts a specified value from the Source field. The following example limits the logs and not the aggregation. You must use an aggregation function to limit the aggregation. 

...where <FIELD>="value"

Examples

Example 1: The following example searches for the exact phrase “unix” in the event messages. If found, the “unix” part will be extracted and added to the log event as a field called “unix”.

* | parse /(unix)/ as unix | where unix="unix"

Formatting Operators

as

Displays the column name as the alias value.

Syntax

... as <ALIAS>

Examples

Example 1: Calculate the sum of the size field for each unique resource name, and sort the results.

* | sum(_size) as log_volume by _resource.name | sort by log_volume desc

limit

Limits the result set to a fixed number of results. The limit operator can only be used after an aggregation operation.

Syntax

* | limit <number>

Examples

Example 1: Count the number of events for each unique resource and show the first 15 results.

* | count by _resource.name | sort by _count desc | limit 15

sort

Sorts the result by a set of fields, ascending or descending.  The sort operator can only be used after an aggregation operation. By default, results are sorted in descending order by timestamp and ID.

Syntax

* | sort by <FIELD> desc

Examples

Example 1: When sorting by an aggregation operator, include a leading underscore on the operator.

* | count by _resource.name | sort by _count desc

Example 2: When sorting by an alias, no underscore is needed.

* | sum(_size) as log_volume by _resource.name | sort by log_volume desc

More Examples

These examples demonstrate how you can evaluate several operations at the same time in the same query.

Example 1:  Display the list of resource group names with count of logs and sum of log size, sorted by count of logs, limited to top 25 resource group names.

* | count(_size), sum(_size) by resource.group.name | sort by _count desc | limit 25

Example 2: Parses the image name from the message and stores name in image field (image=imagename).  Then display the unique image names and sorts by imagename with most occurrences in the specified time range.

“Failed to pull image” | parse /Failed to pull image "(\S+)"/ as image | count by image | sort by _count desc | limit 15
In This Article