Automate All the Things: Migrating Devices from your CMDB to LogicMonitor

What does LogicMonitor Professional Services do? We guide our customers through a full LogicMonitor deployment while solving some of their most complex integration challenges. We often leverage the LogicMonitor REST API to develop unique solutions to automate customer workflows and tasks.

In this post, I’ll walk through a specific use case for one of our major Enterprise customers, a well-known multinational CAD software maker, to illustrate the process we went through to come up with a custom solution to fit their needs.

2,000 CIs to Sync? No Sweat.

One of our Fortune 1000 customers informed us that they leverage their ServiceNow configuration management database (CMDB) to track their device inventory. LogicMonitor has a built-in integration with ServiceNow for two way Incident escalation and delivery – but not for configuration management. We identified this as an opportunity to automate the maintenance of their monitoring infrastructure. In this case, it meant that anytime a Configuration Item (CI) was created, updated, or removed in ServiceNow, this would be reflected within their LogicMonitor portal without any user intervention required.

LogicMonitor has a very robust REST API that helps automate the process of creating, reading, updating, and deleting (CRUD) devices in your portal with almost any programming language. Similarly we identified that the ServiceNow CMDB has its own REST API that we could use to query CIs.

The Nitty Gritty

One of the most important things to accomplish with this type of custom work is to gather all the necessary requirements at the outset. Below, you can see a list of the requirements questions we put together for this particular engagement, followed by their respective answers:

1. What types of devices (Storage, ESX, Linux, etc.) are we trying to sync?
2. What is the criteria for creating, updating, and deleting devices?
3. How do these CMDB CIs map to LogicMonitor devices?
4. How do we automatically assign discovered devices to Collectors?
   1. Will we have multiple Collectors to choose from?
   2. Is there a desire to load balance between a subset of Collectors?
5. What information needs to be passed along to custom alert integration?
6. When and where to execute the script from?

#1 and #2 Based on multiple working sessions we were able to extrapolate that all the data we wanted and needed for creating the LogicMonitor devices and grouping structure is available from the CMDB CI object itself. Below, I’ve listed the specific ServiceNow CMDB classes we chose to sync. This list can be expanded to any number of class types in the future.

# Classes to be synced from cmdb to LogicMonitor
  CMDB_SYNC_CLASSES = ['cmdb_ci_linux_server',
                       'cmdb_ci_msd',
                       'cmdb_ci_esx_server',
                       'cmdb_ci_solaris_server']

Additionally, we used the operational status of the CI to determine when to create, update, or delete devices.

#3 To map the CIs returned by the ServiceNow CMDB REST API we needed to create mapping tables that would map the properties from a ServiceNow CI to a LogicMonitor device. We also added the capability to specify whether the value is required:

# ServiceNow property names to LogicMonitor property name mapping and
# and whether they are required properties.
DIRECT_PROPERTIES = [['name', 'displayName', True],
                     ['ip_address', 'name', True]]

#4 In order to determine which Collector to assign the devices to, we created a mapping table (CSV file) that would choose from a subset of Collectors based on the CI’s name, location, and type. As an extra bonus, we also incorporated load balancing so that we would automatically assign the device to the Collector from the subset with the least amount of monitored devices. See an example of the CSV:

#5 In order to create tickets with their custom alert integration, the customer needed specific metadata that exists in the CI’s object. We were able to quickly meet this need by creating Custom Properties during the same process used for items 1 through 4. Their ticketing system was then able to successfully correlate the LogicMonitor device with the CI in their CMDB for tracking purposes.

# ServiceNow property names to LogicMonitor custom property name mapping
# and whether they are required properties.
CUSTOM_PROPERTIES = [['sys_class_name', 'sn.sys_class_name', True],
                     ['manufacturer', 'sn.manufacturer', False],
                     ['u_support_tier', 'sn.u_support_tier', True],
                     ['u_used_for', 'sn.u_used_for', True],
                     ['support_group', 'sn.support_group', True],
                     ['operational_status', 'sn.operational_status', True],
                     ['sys_id', 'sn.sys_id', True],
                     ['os', 'sn.os', False],
                     ['location', 'location', False]]

Below, you can see the code from a high-level bird’s-eye view. Included are the LogicMonitor resources and actions we used with their reference documentation:

def execute():
  logging.log('ServiceNow CMDB and LogicMonitor Portal sync started.')

  # Read in configuration settings such as credentials, mapping, load balancing, etc.
  configure_incremental_sync()
  configure_clients()

  # Fetch devices from ServiceNow CMDB.
  cmdb_devices = get_cmdb_devices()

  # If CMDB CIs are created, updated, or deleted make LogicMonitor reflect this.
  if len(cmdb_devices) > 0:
      # Create or update LogicMonitor device objects based on configuration criteria.
      lm_devices = map_cmdb_to_lm_devices(cmdb_devices)

      # Push LogicMonitor device objects to LogicMonitor portal.
      create_or_update_lm_devices(lm_devices)

  logging.log('ServiceNow CMDB and LogicMonitor Portal sync completed.')

#6 For code execution we wanted to think a bit outside the box. While the obvious answer would be to run the CMDB synchronization script on-premisess on a set schedule, we wanted to go a step further. In our previous talks with our customer, they mentioned that they were using LogicMonitor for their AWS monitoring, and were leveraging several AWS services in their infrastructure. This brought to mind AWS Lambda, which allows you to execute code without having to worry about provisioning any servers.

Server or Serverless?

While the script had been written to run on-premisess, AWS Lambda functions support a wide range of languages, including the language the script was written in. Lambda functions run based on a trigger event configured when creating the function. Using the lambda-canary trigger, we were able to execute the CMDB sync code on a set schedule. We also added some configuration flags to allow the script to either do a full or incremental sync. A full sync would bring all CIs from the CMDB, while an incremental sync would only attempt to bring in CIs that have changed since the last sync.

In the end, we were able to successfully automate the process of migrating devices from the customer’s CMDB into their LogicMonitor portal using a “serverless” code (AWS Lambda). Anytime a change is made to their CMDB for provisioning or decommissioning a device, this will automatically be reflected in their LogicMonitor portal once the script executes. This helps in identifying any downtime and performance issues with newly added CIs and also minimizes false alerts for decommissioned devices. Set it and forget it.