Linux SSH Monitoring

LogicMonitor offers monitoring for Linux systems that leverages the SSH protocol. SSH collects various metrics including CPU, memory and filesystem utilization, uptime, and throughput.

Configuring Linux instances for SSH involves the following:

• Adding your Linux host into monitoring.
  For more information on adding resources into monitoring, see Adding Resources.
• Configuring SSH on the Linux host for the DataSources to apply.
  You must authenticate the Collector’s access to the resource using an SSH public-private key pair instead of a password. 
• Generating the SSH key pair and copy the public key to the Collector’s host that is assigned to the resource in LogicMonitor and the resource.
  To use an SSH key pair, you need the following:
  • A valid home directory. The SSH server verifies with  ~/.ssh/authorized_keys when authenticating incoming connections.
  • An SSH key pair in classic OpenSSH format (.pem)
    For more information, see How to Use ssh-keygen to Generate a New SSH Key? from SSH.

Requirements for Monitoring Linux Resources Using SSH

To monitor Linux resources using SSH, do the following:

• Import the latest version of all Linux SSH LogicModules. DataSources for SSH modules automatically begin collecting data after installation.
  For more information, see Module Installation.
• Assign SSH credentials as properties on the Linux resource within LogicMonitor to authenticate on the Linux host.
  For more information, see Defining Authentication Credentials.
• Ensure the Linux kernel is updated to version 2.6.24 or newer, and using one of the following Linux distros:
  • CentOS
  • Debian
  • Oracle Linux
  • RHEL
  • Ubuntu

Linux SSH Control Groups and Service Status Instance DataSources

To monitor your Linux environment, you must add the control group and service status DataSources to LogicMonitor as resources. You can accomplish this in one of the following ways:

• Adding the instances manually—use the Resource Tree to add control groups and service instances.
• Using Active Discovery—Enable Active Discovery for control groups and service instance DataSources to detect and add them automatically. For more information, see Active Discovery.

Finding and Manually Adding Control Groups as Instances

1. On the monitored Linux host, open a terminal and run the following command: systemd-cgtop -n1 -b
   This command displays the control groups that are using the most resources. The -n1 flag (short for --iterations=1) denotes only one iteration of the command to execute. The -b flag (short for --batch) forces the command to run in batch mode.
2. From the command output, identify the control group you want to monitor and copy its name, excluding the parent container path.
   For example, if the output displays the following:
   system.slice/atd.service
   Then use the following command:
   atd.service


3. In LogicMonitor, navigate to the Resource Tree and select the Linux host.
4. Select “Add Monitored Instance” from the Manage dropdown menu.
   For more information on manually adding instances, see Adding Instances.
   1. In Add Monitored Instance, enter “Control Groups” in the DataSource field and enter “atd.service” in the Wildcard Value field.
   2. Enter a Name for your instance.
5. (Optional) Add a Description for your instance.
6. Select Save.
   If successful, the instance displays under the DataSource on the Resources page.
   

7. To verify data collection, navigate to the Raw Data tab for the instance and select Poll Now.

LogicMonitor collects data and updates the displayed results when polling completes.

Finding and Manually Adding Services as Instances

1. In a command line, run the following command: 
   systemctl list-units -a --type=service
   This command displays all units that systemd loaded or attempted to load, regardless of their current state on the system. The -a flag (shorthand for --all) ensures all units are listed including those which are inactive. The --type=service flag returns only service units.
2. From the resulting output, copy the name of the service you want to monitor from the UNIT column.
   

3. In LogicMonitor, navigate to the Resource Tree and select the Linux host.
4. Select “Add Monitored Instance” from the Manage dropdown menu.
   For more information on manually adding instances, see Adding Instances.
5. In Add Monitored Instance, enter “Service Status” in the DataSource field and the copied service name in the Wildcard Value field.
6. Enter a Name for your instance.
7. (Optional) Add a Description for your instance.
8. Select Save.
   If successful, the instance displays under the DataSource on the Resources page.
9. To verify data collection, navigate to the Raw Data tab for the instance and select Poll Now.

Active Discovery Script for the Linux_SSH_Cgroups DataSource Definition

Use the following script to enable Active Discovery in the Control Groups DataSource definition:

/*******************************************************************************
© 2007-2024 - LogicMonitor, Inc. All rights reserved.
 ******************************************************************************/
import com.santaba.agent.groovy.utils.GroovyScriptHelper as GSH
import com.logicmonitor.mod.Snippets
def azureHost = hostProps.get("auto.network.resolves") == "false" ? hostProps.get("system.azure.privateIpAddress") 
 : null
def loader = GSH.getInstance(GroovySystem.version).getScript("Snippets", Snippets.getLoader())
def remote = loader.load("lm.remote", "0").create(hostProps)
 .withHost(azureHost ?: hostProps.get("system.hostname"))
 .withKey(hostProps.get("ssh.pass") ? "" : hostProps.get("ssh.cert", "~/.ssh/id_rsa"))
 .withTimeout(15000)
// Expected output pattern capturing cgroup name, tasks, CPU, memory, input, output.
// Unless "CPUAccounting=1" and "MemoryAccounting=1" are enabled for the services in question, 
// no resource accounting will be available and the data shown by systemd-cgtop will be incomplete.
def line_pattern = ~/^\/?([^\/]+)\/(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s*$/
def command_output = remote.exec('systemd-cgtop -n1 -b --cpu=percentage')
command_output.eachLine { line ->
 def matcher = line_pattern.matcher(line) ?: [:]
 // Process lines that contain a match except cgroups in user.slice
 if (matcher.size() > 0 && matcher[0][1] != "user.slice" && matcher[0][1] != "docker") {
 // Replace invalid wildvalue characters with underscores
 def wildvalue = matcher[0][2].replaceAll(/[:|\\|\s|=]+/,"_")
 def cgroupPath = matcher[0][1].replaceAll(/[:|\\|\s|=]+/,"_")
 println "${wildvalue}##${wildvalue}######auto.cgroup.path=${cgroupPath}"
 }
}
return 0
Groovy

Active Discovery Script for the Linux_SSH_ServiceStatus DataSource Definition

The Service Status DataSource has default alerts that you may want to adjust when monitoring all services. Use the following script to enable Active Discovery in the Service Status DataSource definition:

/*******************************************************************************
© 2007-2024 - LogicMonitor, Inc. All rights reserved.
 ******************************************************************************/
import com.santaba.agent.groovy.utils.GroovyScriptHelper as GSH
import com.logicmonitor.mod.Snippets
def azureHost = hostProps.get("auto.network.resolves") == "false" ? hostProps.get("system.azure.privateIpAddress") 
 : null
def loader = GSH.getInstance(GroovySystem.version).getScript("Snippets", Snippets.getLoader())
def remote = loader.load("lm.remote", "0").create(hostProps)
 .withHost(azureHost ?: hostProps.get("system.hostname"))
 .withKey(hostProps.get("ssh.pass") ? "" : hostProps.get("ssh.cert", "~/.ssh/id_rsa"))
 .withTimeout(15000)
// Expected output pattern capturing cgroup name, tasks, CPU, memory, input, output.
// Unless "CPUAccounting=1" and "MemoryAccounting=1" are enabled for the services in question, 
// no resource accounting will be available and the data shown by systemd-cgtop will be incomplete.
def line_pattern = ~/^\/?([^\/]+)\/(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s*$/
def command_output = remote.exec('systemd-cgtop -n1 -b --cpu=percentage')
command_output.eachLine { line ->
 def matcher = line_pattern.matcher(line) ?: [:]
 // Process lines that contain a match except cgroups in user.slice
 if (matcher.size() > 0 && matcher[0][1] != "user.slice" && matcher[0][1] != "docker") {
 // Replace invalid wildvalue characters with underscores
 def wildvalue = matcher[0][2].replaceAll(/[:|\\|\s|=]+/,"_")
 def cgroupPath = matcher[0][1].replaceAll(/[:|\\|\s|=]+/,"_")
 println "${wildvalue}##${wildvalue}######auto.cgroup.path=${cgroupPath}"
 }
}
return 0
Groovy

LogicModules in Package

LogicMonitor’s package for Linux SSH Monitoring consists of the following LogicModules. For full coverage, ensure that all of these LogicModules are imported into your LogicMonitor platform:

When configuring static datapoint thresholds on the various metrics tracked by this package’s DataSources, LogicMonitor follows the technology owner’s best practice KPI recommendations.