Collectors

Collector Caching

Collectors have the ability to cache Collector data to disk. This enables Collectors to store data in your environment during periods where your Collector is unable to deliver data to your LogicMonitor account (e.g. as a result of network issues). Once your Collector can reach your account again, the buffered data will be communicated to our servers, eliminating any gaps in data you would have otherwise seen. By default, Collector caching is enabled and configured to cache up to 30 minutes of data.

Note: LogicMonitor only evaluates the most recent five minutes of cached data for alerts. In other words, the connection to LogicMonitor must be reestablished within five minutes of an alert condition in order for an alert to occur for that condition.

 

How can I change the time limit from 30 minutes?

You can change how much time the Collector can cache data for by changing the value of the reporter.persistent.expire option in the Collector's agent.conf file.  This option is set to 30 by default, which corresponds to up to 30 minutes of cached data.  The option should not be set to any value larger than 1440, which corresponds to 24 hours (Collectors restart every 24 hours and caching cannot continue after a restart).  Note that setting the reporter.persistent.expire option to larger value will consume more disk space.  

 

How can I change the time limit from 30 minutes?

How much disk space is needed?

The amount of disk space necessary on the Collector server is dependent on how heavily loaded the Collector is, as well as how long data is cached for.  The following are estimates of disk space usage for various Collectors, assuming 50 instances per device, an average collection interval of 2 minutes & 30 minutes of cached data:

Collector Load

Disk Space Usage

Light Load


Devices: 50

75MB

Medium Load


Devices: 200

150MB

High Load


Devices: 1000

400MB

 

How can I disable caching?

You can disable/enable Collector caching by setting the reporter.persistent.enable option in the Collector's agent.conf file to false/true, respectively. Note that this option is set to true by default.

 

How can I disable caching?

Where is the cached data stored?

The cached data is stored in the /usr/local/logicmonitor/agent/bin/queues/data directory.  (This path may change if you didn't install the Collector in the default /usr/local/logicmonitor directory).

 

How is data discarded when the time limit is reached?

If the Collector continues to cache data after the limit configured in reporter.persistent.expire (30 mins by default), the oldest data will be discarded.