Where should I adjust thresholds?
You can customize your alert thresholds at the device or cloud resource, group and global levels. Changes made at a lower level (device/cloud resource level being the lowest) override changes made at a higher level (DataSource global definition being the highest level). When you customize alert thresholds, we recommend that you determine whether the changes you need to make should be made globally (in the DataSource definition) or for a specific group or device. You should only tune thresholds globally when a majority of the instances require tuning.
Keep in mind that in addition to changing alert thresholds, it may be beneficial to change the alert trigger interval and alert clear interval values. The alert trigger interval designates how many consecutive polling cycles a threshold must be exceeded before an alert triggers. The alert clear interval designates how many consecutive polling cycles the values must remain below the threshold before an alert clears. Both of these values can be configured in the DataSource datapoint definition.
Once you've decided where to adjust your alert thresholds, follow the appropriate instructions below:
Tuning thresholds globally for all instances (global level)
To tune a threshold globally, you'll need to make the change in the DataSource definition. Keep in mind that this change will apply to all instances of the DataSource in your account. You can find the DataSource definition for a DataSource by either navigating to Settings | DataSources and searching for the DataSource, or by finding the DataSource on a device or cloud resource and selecting the Edit Global Definition link, as shown next.
At the bottom of the DataSource definition you'll see all of the defined datapoints. Select the manage gear icon for the desired datapoint to edit the thresholds. The easiest way to edit thresholds is using the edit threshold wizard. Remember that you can have up to three thresholds, one for each alert severity level. The DataSource definition is also where you can set the alert trigger and clear intervals.
Tuning thresholds at the group level
To edit one of more datapoint thresholds for all instances in a group, navigate to the group from the Resources page and, once highlighted, open the group's Alert Tuning tab. Select the expand arrow for the DataSource and select the pencil icon for the desired datapoint, as shown next.
Tuning thresholds at the device level
Thresholds set at the individual resource level will override thresholds set at the group and global DataSource levels. There isn't an Alert Tuning tab for devices or cloud resources, so these threshold configurations must occur on the Alert Tuning tab for a particular instance or, if you'd like to set new thresholds for multiple instances of one DataSource, on the Alert Tuning tab of a multi-instance DataSource.
Tuning thresholds for a single-instance DataSource
For a single-instance DataSource, open the Alert Tuning tab from the DataSource or, as shown next, its instance, to edit a threshold.
Tuning thresholds for a multi-instance DataSource
For a multi-instance DataSource (e.g. disks or interfaces), you can open the Alert Tuning tab at the DataSource level to manage the thresholds for all instances of that DataSource
I need to tune the thresholds for a subset of instances in a group, but not all of them
One way to tune the thresholds for a subset of instances is to:
- Clone the DataSource whose instances you'd like to tune thresholds.
- Apply the cloned DataSource only to the desired group.
- Add Active Discovery filters to the cloned DataSource to limit the number of instances to just the desired subset of instance.
- Tune the thresholds for the desired datapoints in the cloned DataSource.
For more information on cloning DataSources, see Cloning a LogicModule.
A great example of this would be if you had a group of devices with a naming convention such that all uplink ports are named eth0 through eth4. In this scenario, we don't really care if any ports go down except our uplink ports. Setting thresholds for all instances across the interfaces in our device group won't work well because we don't want to be alerted on the interfaces that are going to go up and down naturally. We could go through instance by instance and set thresholds for each of our uplink ports, but for a large deployment this can be time consuming. Instead, we can clone the Interfaces DataSource and apply an Active Discovery filter that filters out all ports except eth0-eth4. Then we can adjust thresholds for the datapoints in this cloned DataSource and apply it to our one device group:
Alert Threshold History
Each time you edit an Alert Threshold, you will be prompted to add a note to explain why the change occurred. These notes are stored at the datapoint-level and displayed in a history log within the Manage Alert Thresholds dialog. This is a powerful feature that can contextualize your alert thresholds by documenting why a threshold was set at a certain value, who last edited the threshold, and what historical changes have been made to the threshold. This information can facilitate issue identification and remediation.