Examples of JobMonitor Monitoring
Last updated on 19 August, 2024Use the following JobMonitor examples, Monitoring a Linux Cron Job and Monitoring a Windows Scheduler Task, to gain a better understanding of the job monitoring process. This process usually includes four steps:
- Creating a job monitor definition
- Wrapping up the job or task
- Setting the environment variables
- Checking the execution
Warning: Do not provide sensitive information, such as usernames or passwords, in the command line. This information should be handled inside the script, using any secrets management tool or environment variables. (Environment variables are preferred over plain text credentials in code.) LogicMonitor recommends leveraging a dedicated secrets management tool over using environmental variables.
Example 1: Monitor Linux Cron Job
Assume the monitored device in LogicMonitor is app1
, and the collector monitoring app1
runs on the device agenthost
. There is a cron job cleanlogs
running at 8:30, 12:30, and 16:30 everyday on app1
. LogicMonitor should monitor if:
- The task
cleanlogs
starts on time at 8:30, 12:30, and 16:30 everyday. - The task finished successfully.
Step 1: Create the JobMonitor Definition
The first step of monitoring a Linux cron task is to create a batch job definition.
- Log on to your LogicMonitor account.
- Go to Settings > LogicModules > JobMonitors.
- Click Add and then click JobMonitor.
- Edit the JobMonitor definition.
- Click Submit to save the JobMonitor definition.
The JobMonitor definition is associated with device app1
. It triggers an alert if the JobMonitor isn’t finished in ten minutes and triggers a warning alert if the job starts more than 15 minutes after its scheduled run times of 8:30, 12:30, and 16:30. The JobMonitor associated with the device app1
is displayed in the Devices page of your account.
Step 2: Wrap Up the Cron Job
The cron job cleanlogs
currently executes a script /usr/local/cleanlog.pl
at 8:30, 12:30, and 16:30 everyday:
- Copy lmbatchjobwrapper.sh (this file is stored in /usr/local/logicmonitor/agent/lib of
collectorhost
) toapp1
. - Run
crontab -e
(or equivalent) and update the command line to:
30 8,12,16 * * * /usr/local/lmcollector lmbatchjobwrapper.sh agenthost:7214 app1 "Clean log monitoring" /usr/local/cleanlog.pl
- Review the elements in the code sample:
agenthost:7214
is the name of the device on which the collector is running and the port on which the collector listen to accept batch job execution notification sent fromlmJobMonitorwrapper.js
.app1
is the name of the monitored device on which the batch job is running.Clean log monitoring
is the name given to this scheduled task in LogicMonitor.- /usr/local/cleanlog.pl tells the wrapper how to start the program to clean the logs.
Step 3: Set the Linux Environment Variables
Set the variables in /etc/environment
. These variables can be accessed by any user and persist through power-offs.
Note: Variable expansion is not supported in the /etc/environment.
- Open the terminal and enter the following command:
$ sudo -H vi /etc/environment
- Enter the password.
- Add the variables in the file that’s opened. (PASSWORD=pass1234)
- Save and close the file.
- Log out, and then log back in.
Note: Password can be used in the script with $PASSWORD. For more information, see Environment Variables with Linux/Unix.
Step 4: Check the Job Executions
You can check every execution of the scheduled task from the Devices page. Executions that didn’t start on time, or that finished abnormally, are shown with an error.
Example 2: Monitor Windows Scheduler Task
Assume the monitored device is app1
, and the collector monitoring app1
runs on the device agenthost
. There is a Windows scheduler task cleanlogs
running at 8:30, 12:30, and 16:30 everyday on app1
. LogicMonitor should monitor if:
- The task
cleanlogs
is started on time at 8:30, 12:30, and 16:30 everyday. - The task finished successfully.
Step 1: Create the JobMonitor Definition
The first step of monitoring a Windows scheduler task is to create a batch job definition.
- Log on to your LogicMonitor account.
- Go to Settings > LogicModules > JobMonitors.
- Click Add and then click JobMonitor.
- Edit the JobMonitor definition.
- Click Submit to save the JobMonitor definition
The JobMonitor definition is associated with device app1
. It triggers an alert if the JobMonitor isn’t finished in 10 minutes and triggers a warning alert if the job starts more than 15 minutes after its scheduled run times of 8:30, 12:30 and 16:30. The JobMonitor associated with the device app1
is displayed in the Devices page of your account.
Step 2: Wrap Up the Windows Scheduled Task
The scheduled task cleanlogs
executes a script C:\scripts\cleanlogs.bat
at 8:30, 12:30, and 16:30 everyday:
- Copy lmbatchjobwrapper.js (stored in <collectorInstallationDir>\Agent\lib of
collectorhost
) toapp1
. - Open the Properties dialog of the scheduled task
Cleanlogs
, and update the command line to:cscript.exe C:\lmcollector\lmbatchjobwrapper.js agenthost:7214 app1 "Clean log monitoring" "cmd.exe /C C:\script\cleanlogs.bat"
Where:
agenthost:7214
is the name of the device on which the collector is running and the port on which the collector listen to accept batch job execution notification sent from lmbatchjobwrapper.js.app1
is the name of the monitored device on which the batch job is running.Clean log monitoring
is the name given to this scheduled task in LogicMonitor.cmd.exe /C C:\script\cleanlogs.bat
tells the wrapper how to start the program to clean the logs.
Step 3: Set the Windows Environment Variables
- Open the Control Panel and select System.
- Select Advanced system settings.
- In the System Properties dialog box, under Advanced, select Environment Variables.
- Under user variables, select New and then enter the key and value. (PASS pass1234)
- Click Save.
Note: The variables can be provided in a PowerShell script as $PASSWORD=$env:PASS
.
Step 4: Check the Job Executions
You can check every execution of the scheduled task from the Devices page. Executions that didn’t start on time, or that finished abnormally, are shown with an error.
Troubleshooting Cron Jobmonitor
- Open the terminal and enter the following command:
$ crontab -e
- Enter the password.
<div style=”background-color: #eff6fd; margin-bottom: 30px; margin-top: 18px; padding: 20px 20px 2px 20px;”><strong>Note</strong>If you log in as a root user, you are not be required to enter the password.</div> - Update the Cron Jobomonitor by running the following command:54 11 * * *
/usr/local/logicmonitor/agent/lib/lmbatchjobwrapper.sh <agent ip>:7214 nginx.corp.exactasystems.co "CORP-NGINX GixyTest 6am Daily" /etc/scripts/gixytest.sh > /<path>/anyfile.log 2>&1
- To apply the changes, restart the Cron Jobmonitor by running the following command:
sudo systemctl restart cron
- To see if the Cron jobs are running correctly, run the following command:
tail -f filename.log
. - To enable the Cron Jobmonitor logging, open
/etc/rsyslog.d/50-default.conf
file by running the following command and uncommenting the code:# Uncomment the following line to enable cron logging to a separate file cron.* /var/log/cron.log cron.* /var/log/cron.log
- Restart the Syslog service to ensure the logging configuration is updated and functional by running the following command:
sudo systemctl restart rsyslog
- Once the changes are applied, you can see the Cron Jobmonitor details by navigating to Resource Tree > select the resource on which jobmonitor module is applied > data tab.