Settings

PagerDuty Integration

Introduction to PagerDuty Integration

PagerDuty's alerting platform allows users to centrally manage incidents with alerting, on-call scheduling, escalation policies, and incident tracking intended to increase the reliability and performance of your systems. Your LogicMonitor account comes pre-configured to integrate with your PagerDuty account. The integration enables LogicMonitor to open, update and close PagerDuty incidents and/or alerts based on LogicMonitor alerts. By sending alerts from LogicMonitor into PagerDuty, you can take advantage of PagerDuty’s alerting platform features to increase uptime of your apps, servers, websites, and databases. As with all integrations, LogicMonitor extends or provides functionality without impacting the other solutions you have implemented.

Note: As of v.116 (released in March of 2019), LogicMonitor's PagerDuty integration supports V2 of PagerDuty's Events and Webhook APIs. If you would like to upgrade an integration configured in an earlier release of LogicMonitor, see the Upgrading an Existing V1 PagerDuty Integration section of this support article for instructions.

Setting Up PagerDuty Integration

You can use the LogicMonitor PagerDuty integration to create, update and close PagerDuty incidents in response to alerts triggered in your account. You'll need to perform the following primary steps to set up PagerDuty integration.

  1. Obtain an integration key in your PagerDuty account that LogicMonitor can use to create, update and close incidents, as discussed next.
  2. Enable the PagerDuty integration in your LogicMonitor account, as discussed next.
  3. Add the PagerDuty integration as the contact method for a recipient in an escalation chain, as discussed in Escalation Chains.
  4. Add the escalation chain to an alert rule, as discussed in Alert Rules.

When a triggered alert matches the alert rule and is routed to the escalation chain with the PagerDuty integration, an incident will be created in your PagerDuty account. The ticket can then be updated and/or closed when the alert is acknowledged, escalated, de-escalated and/or cleared.

Obtaining a PagerDuty Integration Key

You will need to set up a new service in PagerDuty (Services | Add Service). As shown next, select "LogicMonitor" from the Integration Type field's dropdown menu.

The option found under the Incident Behavior heading can be set to either Create alerts and incidents or Create incidents. To use PagerDuty's event grouping, leave the default Create alerts and incidents option selected.

A new integration key will be generated once you add the service. You will need to save the key for later use when setting up the PagerDuty integration in your LogicMonitor account.

Enabling PagerDuty Integration in Your LogicMonitor Account

To begin configuring your PagerDuty integration in LogicMonitor, navigate to Settings | Integrations | Add | PagerDuty. As shown (and discussed) next, there are several settings that must be established in order to configure PagerDuty integration.

Integration Key

In the Integration Key field, enter the PagerDuty integration key retrieved in the previous section.

HTTP Delivery

The HTTP Delivery section controls how LogicMonitor formats and sends the HTTP Post requests to create, update and/or close incidents. By default, LogicMonitor will pre-populate four different HTTP requests, one for each of the following alert statuses:

  • New alerts (Active)
  • Acknowledged alerts (Acknowledged)
  • Cleared alerts (Cleared)
  • Escalated alerts (Escalated)

For each request, you can select which alert statuses should trigger the HTTP request. Requests will be sent for new alerts (status: Active), and can additionally be sent for alert acknowledgements (status: Acknowledged), clears (status: Cleared) and escalations/de-escalations (status: Escalated). Note that each alert status can only be associated with one request. Since LogicMonitor auto-populates a different request for each alert status by default, you'll have to delete a request in order to see the option to include that alert status in a different request.

You shouldn't need to edit anything in the HTTP Delivery section, but if you wish to create a customization of some kind, use the information in the following sections to guide you. If no customizations are required, save the integration and add it to an escalation chain, as discussed in Escalation Chains.

HTTP Method

The HTTP method for PagerDuty integrations is restricted to POST.

URL

The URL field displays the URL that the HTTP request should be made to. This field is auto-populated based on information you've provided.

Alert Data

The text block found under the Alert Data heading displays custom formatted alert data to be sent in the HTTP POST request (used to create, update and close PagerDuty incidents). This text will be auto-populated for you. If desired, you can customize the alert data field using tokens.

Include an ID provided in HTTP response when updating alert status

Check the Include an ID provided in HTTP response when updating alert status option if you'd like LogicMonitor to find the PagerDuty incident ID returned in the response to the HTTP request associated with a new alert, and use the ID in any subsequent requests for alert acknowledgements, clears and escalations/de-escalations. By default, this option will be selected.

HTTP Response Format

If LogicMonitor is to use the ID provided in the response, select the format the response will be in from the HTTP Response Format field's dropdown.

Test Alert Delivery

Clicking the Test Alert Delivery button sends a test alert and provides the response, allowing you to test whether you've configured the integration correctly.

Acknowledging LogicMonitor Alerts from PagerDuty

You can acknowledge LogicMonitor alerts directly from the PagerDuty platform. To set up this functionality, you will need to establish a LogicMonitor extension (i.e. webhook) within PagerDuty using the following steps.

  1. In your PagerDuty account, navigate to Configuration | Services and select your LogicMonitor integration from the table of established PagerDuty service integrations.

  2. Once in the LogicMonitor view, open the Integrations tab.
  3. Click the + New Extension button located in the bottom left corner.

  4. One of the fields that dynamically displays upon clicking the + New Extension button requires an API token from LogicMonitor. To generate this token:
    1. Open LogicMonitor and find your saved PagerDuty integration (Settings | Integrations).
    2. Click its corresponding gear icon in order to open the Manage Integration dialog.
    3. At the bottom of the Manage Integration dialog, click the Generate API Token button. Copy this token to your clipboard.

      Note: If you are upgrading from V1 to V2, as discussed in the Upgrading an Existing V1 PagerDuty Integration section of this support article, a prompt will warn you that you are overwriting the previous token. This is okay; click the Generate button to proceed with generating the token.

  5. Return to PagerDuty to complete the new extension configurations:
    1. From the Extension Type field's dropdown menu, select "LogicMonitor."
    2. In the Name field, assign a name to the extension that clearly describes its purpose (e.g. "LogicMonitor ACK").
    3. In the Details field, paste the URL value you just generated from within LogicMonitor.

  6. Click the Save button to complete the extension configuration and enable acknowledgement of LogicMonitor alerts directly from PagerDuty.

Upgrading an Existing V1 PagerDuty Integration

Update V1 Payloads to V2 Payloads

For your convenience, LogicMonitor has automated the process of updating payloads from PagerDuty's V1 APIs to V2. Simply open your PagerDuty integration in LogicMonitor and, from the Manage Integration dialog, click the Upgrade Payload to v2 button.

Update PagerDuty Service

Your previously configured PagerDuty service will continue to work without updates. However, if you would like to have the LogicMonitor integration use PagerDuty's Event Intelligence capabilities, follow these steps in PagerDuty to update the service:

  1. From your PagerDuty account, navigate to to Configuration | Services.
  2. Click the gear icon associated with the LogicMonitor service and select "Edit Service" from the dropdown.
  3. Under the Incident Behavior heading, select the Create alerts and incidents option.
  4. Click the Save Changes button.

Update PagerDuty Extension

If your PagerDuty service is currently using the "Generic Webhook V1" extension to acknowledge LogicMonitor alerts from PagerDuty, you'll need to update the extension to continue to successfully acknowledge events.

  1. In your PagerDuty account, navigate to Configuration | Services and select your LogicMonitor integration from the table of established PagerDuty service integrations.
  2. Once in the LogicMonitor view, open the Integrations tab.
  3. As shown next, delete the existing "Generic V1 Webhook" extension.

  4. Add a new extension following the steps provided in the Acknowledging LogicMonitor Alerts from PagerDuty section of this support article.

Tokens Available

The following tokens are available:

  • LogicModule-specific alert message tokens, as listed in Tokens Available in LogicModule Alert Messages.
  • ##ADMIN##. The user the alert was escalated to.
  • ##MESSAGE##. The rendered text of the alert message. This token will also pass all relevant acked information (e.g. the user that acknowledged the alert, ack comments, etc.).
  • ##ALERTTYPE##. The type of alert (i.e. alert, eventAlert, batchJobAlert, hostClusterAlert, websiteAlert, agentDownAlert, agentFailoverAlert, agentFailBackAlert, alertThrottledAlert).
  • ##EXTERNALTICKETID##. The PagerDuty incident ID.