Log Alert Conditions

Log alerts are triggered when specific patterns or keywords appear in log data. To manage these alerts, you use log pipelines. A log pipeline defines how and where alerts are applied. Pipelines group related alert conditions, similar to how folders or resource groups organize items in the Resource tree.

Pipelines are typically structured by Resource or Resource group. This structure enables scalable and intuitive alert management. For example:

• Use a pipeline named “Cisco ASA” to group alerts related to firewall logs.
• Use “Domain Controllers – East Region” to apply alerts to domain controllers in a specific geographic region.

Each pipeline targets a specific set of Resources or Resource groups. The alert conditions within the pipeline define when alerts are triggered—such as when a log message contains a specific keyword or matches a pattern.

Log alert conditions are based on log queries that match specific conditions in ingested logs. These matches can be based on a keyword, string, key-value pair, or a regular expression (regex) pattern. When a match occurs, LogicMonitor triggers an alert and sends a notification to inform you about the log event or anomaly.

With log alerts, you can speed up your investigation by adding alert conditions to track the logs you always want to know about or creating alerts on a detected anomaly.

Viewing Alert Conditions

On the Logs page, select the Pipeline Alerts icon to open the Alert Conditions page where you can review and manage alert conditions.

• Severity—Displays the level (Critical, Error, Warning) configured for the alert condition.
• Display As—Shows the configured display name of the alert condition.
• Logs query—Displays the regular expression pattern to match log events that will trigger the alert.
• Condition type—Displays the type of alert condition, such as basic or stateful alert.
• Pipeline—Displays the pipeline that the alert condition applies to.
• Description—Provides a short explanation of the alert condition.
• Active—Lets you toggle the alert condition on and off.
• Filter alert conditions associated with a specific pipeline.

Considerations for Log Alert Conditions

• Alert rate limit—Each pipeline can generate up to 90 alerts per minute. Alerts that exceed this rate will be dropped and not processed.
• Pipeline limits—Each portal supports up to 20 log pipelines.
• Per-Pipeline Alert condition limits—Each pipeline supports up to 55 log alert conditions, which includes:
  • Time-Based Alert Conditions—Up to 10 time-based alerts conditions per pipeline can be created. The time period based alerts are generated when at least one log matches a defined condition within a selected time window (10 min, 30 min, 1 hr, 8 hrs, or 1 day).
  • Stateful Alert Conditions—Up to 10 stateful alert conditions per pipeline can be created.

Adding Log Alert Conditions

You can add alert conditions directly from any log event or anomaly listed in the Logs page, or from the Pipelines and Pipeline Alerts pages.

• From a log event or anomaly on the Logs page: Open the action menu at the beginning of the log line and select “Create alert condition” to open the Add Alert Condition dialog.
• From the Pipelines page: Select the Alert Conditions icon or count to open the Alert Conditions page, and select the plus sign to add an alert condition.

You can add the following alert conditions:

• A basic alert condition—time-based alerts
• A stateful alert condition

Adding a Basic Alert Condition

In the Add Alert Condition dialog, enter information as follows:

1. Select the type of Alert severity level (Critical, Error, Warning) to generate when conditions are met.
2. Enter a Display name and a Description to be displayed in the list of alert conditions.
3. Select a Pipeline to apply the alert condition (unless prefilled).
4. Unless prefilled, enter a Logs query to filter the events in the pipeline. For more information see Writing a Filtering Query. Select the arrow to preview the results and refine your query before you save.
5. From Trigger alert, to activate an alert condition, do the following:
   1. for each matching log – Select this option if you want to generate an alert for each log matching a condition.
   2. only after matching logs within – Select this option to generate an alert if a certain number of logs match a condition within a defined time period. To generate an alert, there should be at least 1 matching log. You can select the following time intervals: 10 minutes, 30 minutes, 1 hour, 8 hours, and 1 day.

6. To define Alert clear conditions, enter the time that the alert will persist. Default is 60 minutes.
7. Toggle the Auto-clear after acknowledge switch on to clear the alert after it is acknowledged.
8. Toggle the Active switch on to activate the alert condition.
9. Select the Save icon to add the alert condition.

When you return to the Alert Conditions page, you can review the alert you created in the table.

Example

The pipeline Windows Servers (_resource.group.name="Windows") targets a subset of logs related to a “Windows” resource group. The alert triggers on specific keywords or values (eventcode 1100 and application log files). You do not need to include the resource group name in the query for the alert condition (however it would work).

Acknowledging Alerts

When log alerts conditions are matched, they trigger standard LogicMonitor alert notifications based on the alert settings (Critical, Warning, Error) and will route through the configured escalation chain.

You can also:

• View the log alert in the Logs graph, where it will display as a line alongside the alerting log event.
• Acknowledge or clear the alert in the Alerts list.

Adding a stateful alert condition

To add a stateful alert condition, do the following:

1. From a log event or anomaly on the Logs page, open the Action menu at the beginning of the log line and select Create alert condition to open the Add Alert Condition dialog.
   Alternatively, from the Pipelines page, select the Alert Conditions icon or alert condition count to open the Alert Conditions page, and select the plus sign to add an alert condition.
2. In the Alert condition name field, enter a name for the alert condition.
3. From the Alert condition type dropdown list, select Stateful.
4. From the Alert severity dropdown, select the type of alert severity level, such as Critical, Error, or Warning when the conditions are met.
5. Enterthe description for the alert.
6. Select the toggle Existing Pipeline or New Pipeline to choose an existing pipeline or a new pipeline to create the alert condition.

7. Unless prefilled, enter a Logs query to filter the events in the pipeline. For more information, see Writing a Filtering Query.
8. Select the arrow to preview the results and refine your query before you save.
9. In the Alerting condition field, to add an alerting condition,type a query or select the existing queries by clicking inside the field. Use this condition to define the condition to generate an alert.

10. Under Alert clear conditions, to add a condition to clear an alert condition, do the following:
    • Type a query for Clear condition or select the existing queries from the list. Use this condition to define the condition to clear an alert.
    • For Clear after, enter the time that the alert will persist. Default is 60 minutes. You an choose time in days, hours, and minutes.
    • Toggle the Auto-clear after acknowledge switch on to clear the alert after it is acknowledged.

11. In the Alert grouping criteria table,to add alert grouping criteria, do the following:
    1. Select the add icon to add the alerting criteria.
    2. In the Alert grouping criteria and Clear grouping criteria fields, select or type the alerting and clearing criteria.
    3. Select Apply.

12. Toggle the Active switch on to activate the alert condition.
13. Under the Log metadata fields included in alert section, do the following to add metadata in the alert information.
    1. Select Add Record ➕ to add a new field.
    2. In Logs fields, search for or enter the name of the metadata field, and select Apply.
    3. To delete a field, select Delete in the Actions column for the corresponding field.

14. Select the Save icon to add the alert condition.