Windows Event Log Monitoring

Overview

LogicMonitor can detect and alert on events recorded in most Windows Event logs.  An EventSource must be defined to match the characteristics of an event in order to trigger an alert. When a collector detects an event that matches an EventSource, the event will trigger an alert and escalate according to the alert rules defined.

Note: LogicMonitor does not currently support the monitoring of any logs located under the “Application and Services Logs” folder in the Windows Event Viewer snap-in console, as these logs aren’t natively exposed to WMI. If you would like to monitor your Application and Services Logs in LogicMonitor, you can use Microsoft Subscriptions to ‘copy’ logs from your Application and Services folder to another log folder (e.g. “Application” logs). For instructions on how to correctly set up both Microsoft Subscriptions and your EventSource to accomplish this, see the Filter Required When Using Subscriptions to Copy Events to Another Destination section of this support article.

Creating a Windows Event Log EventSource

You can define a new Windows Event Log EventSource from Settings | LogicModules | Eventsource | New | Eventsource.

To begin, set the Type field to “Windows Event Logging”.

Other fields that must be configured include:

• Applies To. This field accepts LogicMonitor’s AppliesTo scripting as input to determine which resources will be associated with this EventSource. In the example above, “isWindows()” indicates the EventSource will be associated with any Windows devices. For detailed information on using this field (including its wizard and test functionality), along with an overview of the AppliesTo scripting syntax, see AppliesTo Scripting Overview.
• Group. (Optional) This field allows you to group several EventSources in a folder.
• Filters. This area of the configuration dialog sets characteristics that events must have in order to trigger an alert. You can filter based on the alert event ID, level, log name, message, and source name. An event must satisfy every line in the Filters section. In the example above,  the Event ID must equal 7040  AND  the string ‘disabled’ must be in the event message. 
• Alert Settings. This area of the configuration dialog defines the characteristics of the alert that is triggered when an event is detected that matches the filter criteria for this EventSource. The settings available for configuration here are largely identical across all EventSource types and discussed in Creating EventSources. However, there are two alert settings unique to Windows Event Logs:
  • Alert Level for Matching Events. By default, LogicMonitor maps Windows severity levels to LogicMonitor severity levels (e.g. error to error, critical to critical). You can use this field to override this default mapping so that LogicMonitor alerts triggered for particular Windows Event Log EventSources are assigned a static alert severity.

    Note: LogicMonitor’s pre-built Windows Event Log EventSources have filters that restrict alerting to error and critical level events.

  • Suppress duplicate Events. Sometimes the system may create events with the same EventId for a given host. If you have two EventSources with the same Event ID filter applied to filter out events, and if you do not select the Suppress duplicate Events option, then you will get two alerts and they will not be suppressed. You will get one alert in each EventSource. To prevent this from happening, you can select Suppress duplicate Events. By default, this option is selected. This suppresses duplicate Windows Event Log events that have the same event ID for the duration of the interval set in the Clear After field. As a result, LogicMonitor captures a single event per host and generates an alert.
    
    If this option is not selected, duplicate events are not suppressed. As a result, though a single event is captured per EventSource, multiple duplicate events are captured and multiple alerts are generated. Events captured by a single EventSource are suppressed within the specified interval. 
    
    Note: If all the duplicate events are mapped to a single EventSource, a single alert is generated.
    To capture duplicate events, you have to create different EventSources with specific filters.
    The ability to override duplicate EventID suppression requires Collector version 29.104 or later.
    
    Example 1
    Scenario: One EventSource is applied to a device. The filter is: EVENTID – 725
    LogicMonitor gets two events with EventId 725 but with different messages. 
    
    In this scenario, both the events are mapped to this EventSource.
    If the Suppress duplicate Events option is selected, a single alert is generated and the other event is suppressed.
    If the Suppress duplicate Events option is not selected, a single event is captured and the other event is suppressed. The reason being, there can be only one alert per EventSource.
    
    Example 2
    Scenario: Two EventSources are applied to a device.
    For EventSource 1, the filter is – EVENTID – 725 and MESSAGE – Message 1
    For EventSource 2, the filter is – EVENTID – 725 and MESSAGE – Message 2
    LogicMonitor gets two events for the EVENTID 725. Each event has its own message in the filter.
    
    In this scenario, one event is mapped with EventSource 1 and the other event with EventSource 2.
    If the Suppress duplicate Events option is selected, a single alert is generated and the other event is suppressed.
    If the Suppress duplicate Events option is not selected in both EventSources, two alerts (one alert for each EventSource) are generated.

Event Filtering

To match multiple event IDs, you can use the In comparison operation, and list the event IDs separated by the pipe “|” character – spaces are ignored.  

Notes:

• Each line in the Filters section is combined using a logical AND with the other lines, so adding more lines makes the filter more restrictive.
• Windows Event Log uses the same event level for both error and critical events reported through WMI. Ensure that you take this into account when choosing the alert level for routing. Selecting anything over warning will catch both error and critical events. Critical events route the same as error events and are also designated as error-level alerts in the LogicMonitor console.
• Regexmatch should not be used – use the In operator instead.
• RegexNotMatch does not support regular expressions – it only supports a bar separated list of eventIDs to exclude.
• By default, LogicMonitor excludes all events of level Informational. Thus if you specified a filter of Logname equals System, with no other filters, you would see all events except those of informational level. In order to raise alerts on Informational level events, you must specify the event explicitly by ID using the EQUAL or IN operator.
• In some cases, a resource’s Windows Event Log will generate an event without populating the INSERTIONSTRINGS attribute of the message. This may result in LogicMonitor capturing an event with a matching EventID and generating an alert, passing along the null value read from the host side. You can add a filter into the EventSource to discard messages that the resource generates incorrectly:

  

You can use the ##FILTEREDEVENTS## token to filter event Ids based on the value of the FILTEREDEVENTS property set at the device, device group, or global level. Consider the following EventSource filters:

The first three lines in the Filters section above will find all events in the System Event log with a severity greater than warning, excluding events with EventIDs 1111 or 10009 . The fourth line (boxed in red) will then exclude events where the EventID matches the IDs contained in the ##FILTEREDEVENTS## property.

• You can set the FILTEREDEVENTS property to exclude events. This property can be set on the global, group, or device level for granular control.
• Initially, when the FILTEREDEVENTS property has not yet been set on any device or device group, it is an empty string, and so does not match (or exclude) any events.

Example

For example, you want to exclude event ID 123 from triggering an alert on all windows devices. To do this, set the property FILTEREDEVENTS to 123 on the top level of the device tree. This setting will be inherited by all lower nodes.

For one group of servers, you want to exclude event IDs 123 as well as 456 and 789 triggering alerts. In this case, you can set the filteredevents property to the expression 123|456|789 on the group level. This will override the inherited property that has been set on the root level group just for that group and lower nodes.

For a specific server within this group, you might be interested in alerting on all events including 123, but not 456 or 789. You can set the FILTEREDEVENTS property again on this specific server to 456|789 to override the inherited property from all levels above.

If you want to filter on a composition of both the filtered events set on the upper group + the lower group, define an additional ##FILTEREDEVENTS1##, ##FILTEREDEVENTS2##, etc. token variable in the EventSource definition, and then define these separately at different levels of your device group tree structure. Make one token for each device group level in your device group tree structure that you would like to cumulatively match.

The standard filters described above are all combined with an AND clause – i.e. an event must meet the criteria specified in each and every filter in order to be alerted on.  This is sufficient for most cases – but not for cases where more specificity is required (e.g. ignore eventID 123 if it is from IIS, but alert on it otherwise), or where several conditions are sufficient to trigger the alert (e.g. alert if Level is error or higher, or if the alert is eventID 123)

Complex Event Filtering

If Complex filtering is checked, a Groovy script can be used to filter events. The script should return a boolean value as the filter’s result. Otherwise, the value of last expression will be the return value.

The event will only be detected and trigger alerts if the return value is true. If the return value is false or null, the event will not be detected. All other values will throw an exception and cause filtering to fail.

Complex Windows Event Filters do not have validity checking.  Thus, simple filtering should be used where possible.

For instance, if you were filtering based on a wide range of Event IDs, ie. between 4000 and 5000, you would use the following Groovy script:

(EVENTID>’4000′) && (EVENTID < ‘5000’)

Filter Required When Using Subscriptions to Copy Events to Another Destination

LogicMonitor does not currently support the monitoring of any logs located under the “Application and Services Logs” folder in the Windows Event Viewer snap-in console, as these logs aren’t natively exposed to WMI. If you would like to monitor your Application and Services Logs in LogicMonitor, you can use Microsoft Subscriptions to ‘copy’ logs from your Application and Services folder to another log folder (e.g. “Application” logs), as demonstrated in this video.

Once the logs are copied to a destination that is available to LogicMonitor, you will need to add a filter type of “LOGNAME” that equals <OriginalLogName>. As shown next, you can find your event’s original log name in the event details that display in the Event Viewer.

Predefined Variables

The following predefined variables can be used to access data about the event being processed:

Name	Type
LOGNAME	String
MESSAGE	String
SOURCENAME	String
LEVEL	Integer (Value: Meaning) 1: Error 2: Warning 3: Information 4: Security Audit Success 5: Security Audit Failure
EVENTID	Integer

For a more complete definition of each variable, see https://msdn.microsoft.com/en-us/library/aa394226(v=vs.85).aspx. Note the names of properties are translated to uppercase.

There are many operators in Groovy, but it is worth calling out the regular expression operator:

Examples