Troubleshooting Alert Delivery
The most common issues with alerts and alert delivery tend to be:
If you aren't receiving alerts and you think you should be
In general, if you don't receive an alert for something that is important to you, we recommend that you figure out why. Doing so will allow you to correct your alerting configuration such that you get notified for similar events in the future.
Are the alerts being generated?
First, it is important to distinguish whether your problem is with the alert generation or with the alert delivery. If you cannot find the alerts you think you should be receiving on the alerts tab of your account (make sure to select 'cleared' in the filter criteria to display all alerts from the past 7 days), then the alert probably isn't even being triggered. You need to adjust your alert thresholds such that alerts are triggered when you want them to be.
Does the alert match an alert rule?
If, on the other hand, you do see the alerts in your account but you aren't receiving alert notifications, then you need to determine whether you have an alert rule configured to route notifications for that type of alert. Remember that in order for alert notifications to be delivered to recipients, the particular service or device datapoint must match an alert rule, and this alert rule must reference an escalation chain that contains the recipients that you want to deliver notifications to. In most cases, the cause for an alert not delivering correctly is because it is being matched by an unexpected alert rule - we recommend testing your alert rules to ensure they are routing your alerts as intended.
Was the alert triggered during SDT?
Keep in mind that alerts that occur during periods of SDT will still display in your account, but will not be delivered anywhere. A group, device, datasource or instance in SDT will be displayed with the following icon in the device tree:
Is the escalation chain rate limited?
If rate limiting is enabled for the escalation chain, the number of alert notifications that can be sent to the escalation chain in a specified time period is limited. An escalation chain has rate limiting enabled if the value in the Rate Limiting column is anything other than 'Disabled':
Is the contact information for your user incorrect?
If the information for your user in an escalation chain or in a recipient group in an escalation chain is incorrect, alert notifications won't be delivered correctly. Please check your user settings and verify that your email and phone number are entered correctly.
Is your receiving email or SMS gateway refusing messages or queuing messages for delivery?
Alert notification messages could be refused or queued because of spam control, gateway misconfiguration, DNS issues, etc.
Was the alert notification marked as spam by your email client?
Please check your spam folder.
Is it a collector, service, eventsource, external alerting or integrations alert?
- Collector: check to make sure there is a valid escalation chain specified for your collector:
- Services: Make sure that the option to send individual alerts has not been disabled. Keep in mind that in-account alerts are still generated for individual alerts when this option is disabled:
- External alerting: Make sure that the referenced collector is online
- Integrations: Connectwise- Make sure that the various tickettype, companyid, or serviceboard properties have been set and are being inherited correctly on your device.
If none of the above apply...
One way that you can further troubleshoot alert routing issues is to attempt to generate a "fake" alert (try temporarily changing an alert threshold) and see if it routes or escalates up the stages in your escalation chain as intended. A support engineer can additionally help you with analyzing mail records during this process.
If you are receiving too many alerts
Receiving too many LogicMonitor alert notification emails can ultimately lead to people ignoring important alerts. The keys to avoiding this problem are:
- Tuning your alert thresholds to fit your environment
- Avoid routing all alerts to people within your organization - cut down the number of non-critical issues you're alerting people for. Some alerts, such as warning alerts, are better viewed regularly in LogicMonitor reports, or being posted to a ticketing system or HipChat room using custom alert delivery methods.