Log Processing Pipelines

There are certain types of log events or anomalies that you may want to always track and take action on, such as errors or exceptions that should notify to be resolved immediately.

Log pipelines are a series of processing steps executed on log events that match a certain set of filters. Once you have defined filters for the logs you want to track, you can define other processing steps, such as setting alert conditions on them.

Reviewing Pipelines

On the Logs page, click the Pipelines icon to open the Pipelines page, where you can review and manage existing log pipelines as well as add new ones. 

• Actions allow you to edit the pipeline settings or delete the pipeline.
• Filter lists the filtering conditions that define the log events in the pipeline. For example, the resources where the logs are received from.
• Alert Conditions lists the number of alert conditions defined for that pipeline. Click on the icon or count to open the Alert Conditions page for that pipeline. See Log Alerts.

Adding new Pipelines

To define a new log pipeline:

1. In the Pipelines page, click the plus sign to add a new pipeline.
2. In the Add Pipeline dialog, enter a Name (required) and Description.
3. Under “Match logs using resource”, select the resource that the pipeline will match events from.
4. Click Add.

When you return to the Pipelines page, you can review the pipeline you created in the table.

After adding a pipeline, you may want to define alert conditions to notify when certain logs are received in the pipeline. See Log Alerts.