Settings

Collector Capacity

The amount of data that can be monitored by one Collector depends on the Collector's configuration and resources.  You can monitor the load and capacity of your Collector using the Collector monitoring facilities of LogicMonitor.  If you have a large environment, and are experiencing alerts on the Unavailable Task Rate datasource of your Collectors, you may need to tune your Collector to increase its monitoring capacity.

Device Capacity Limits 

The following are representative measures of the capacity of differently sized Collectors of version GD 24.000 or later, measured in terms of a "standard device."  The "standard device" assumes there are 50 instances collected by each protocol, checked with a 2 minute frequency. Additionally,  the capacity is dependent on the number of instances that need to be discovered for each monitored device. For instance, if each device is a load balancer with 10,000 instances, the Collector capacity will be lower. Similarly, if each device is a switch with hundreds of interfaces, the Collector capacity may be lower because it is limited by discovery.

Notes:

  • If your Collector will be running on an Amazon EC2 instance, we recommend using a fixed performance instance type (e.g. M3, C3) instead of a credit based instance type (e.g. T2).
  • While we strive to ensure that these measures are as accurate as possible, these are estimates and they may not be accurate for every production environment.  These measures are subject to change.
  • The Nano Collector size is not included in the following tables. This is due to the fact that the Nano Collector is intended for testing purposes. As such, no recommended device-count capacity has been assigned to it.
 

Protocol

Small Collector


CPU:1 Intel Xeon Family

System Memory: 2GB

JVM maximum memory size: 1GB

Medium Collector


CPU:2 Intel Xeon E5-2680v2 2.8GHz

System Memory: 3.75GB

JVM maximum memory size: 2GB

Large Collector


CPU:4 Intel Xeon E5-2680v2 2.8GHz

System Memory: 7.5GB

JVM maximum memory size: 4GB
SNMP 400 requests per second (1000 standard devices) 830 requests per second (2000 standard devices) 1040 requests per second (2500 standard devices)
HTTP 350 requests per second (840 standard devices) 880 requests per second (2100 standard devices) 1175 requests per second (2800 standard devices)
WMI 40 requests per second (100 standard devices) 210 requests per second (500 standard devices) 330 requests per second (800 standard devices)
Perfmon 205 requests per second (490 standard devices) 410 requests per second (980 standard devices) 530 requests per second (1260 standard devices)
JMX 65 requests per second (150 standard devices) 300 requests per second (500 standard devices) 700 requests per second (1500 standard devices)


Netflow Capacity Limits

The following are representative measures of the flows per second that can be comfortably monitored by a Collector running in different sample environments:
 

Scenario

Supported Flows/second

System: Linux 32bit
Memory: 2G
CPU: 1 core
Disk: Sata
Collector Size: Small
6000 flows/s
System: Linux 64bit
Memory: 4G
CPU: 8 cores
Disk: SSD
Collector Size: Small
9000 flows/s
System: Windows 64bit
Memory: 4G
CPU: 4 cores
Disk: SSD
Collector Size: Medium
15000 flows/s


Tuning Collector Size

 You can adjust the Collector size directly from the LogicMonitor UI. This is particularly useful for performance tuning and increasing the capacity of your Collector after you have already installed it.

From Manage Collector | Support| Collector Configuration, select the desired Collector size from the dropdown menu. As soon as you select "Save and Restart," we will auto-verify that your host has enough available memory to support the new Collector size. 

Please note the following

  1. Older Collectors will display their current size as "Custom (xGB)" in the dropdown, even if no parameters have been modified since installing. This is because our definition of size has changed since the Collector was installed. If you want to ensure the Collector configuration is up to date, simply select the size you want (or had installed originally) and select Save and Restart.
  2. Changing a Collector's size has no effect on parameters unrelated to its size. The parameters listed in the section below, Configuration Details, are the only ones impacted by a change in the Collector's size. 
  3. If your Collector version is below EA 24.126, you will need to manually change the configuration files (wrapper, sbproxy, and agent) to match the specifications listed below for the size you desire. You can do so by selecting "Edit config manually" and finding the necessary parameters in the text.

If you are manually changing the Collector's config parameters, we will run a validity check after you select "Save and Restart" to ensure that no errors were made in the new configuration. If errors are detected, we will display which lines are missing/duplicated so they can be corrected.



 

Configuration Details 

Small Collector
 

Config File Parameters Description
wrapper.conf wrapper.java.initmemory=128 Minimum Java Heap Size(MB) for Collector
wrapper.java.maxmemory=1024 Maximum Java Heap Size(MB) for Collector
sbproxy.conf wmi.stage.threadpool.maxsize=100 The maximum size of threads to handle WMI query/fetch data in sbwinproxy.exe
wmi.connection.threadpool.maxsize=50 The maximum size of threads for WMI to connect to remote machine in sbwinproxy.exe
agent.conf sbproxy.connector.capacity=8192 The maximum number of requests that the Collector can send in parallel to sbwinproxy and sblinuxproxy
discover.workers=10 Allocates resources to Active Discovery iterations
autoprops.workers=10 The thread pool size for AP
reporter.persistent.queue.consume.rate=10 The max count of data entries that will be reported for each API call.
reporter.persistent.queue.consumer=10 The thread count used to read from buffer and execute reporting.
collector.script.threadpool=100 The max thread count to run script tasks.
watchdog.conf  N/A N/A


Medium Collector
 

Config File Parameters Description
wrapper.conf wrapper.java.initmemory=512 Minimum Java Heap Size(MB) for Collector
wrapper.java.maxmemory=2048 Maximum Java Heap Size(MB) for Collector
sbproxy.conf wmi.stage.threadpool.maxsize=200 The maximum size of threads to handle WMI query/fetch data in sbwinproxy.exe
wmi.connection.threadpool.maxsize=100 The maximum size of threads for WMI to connect to remote machine in sbwinproxy.exe
agent.conf sbproxy.connector.capacity=8192 The maximum number of requests that the Collector can send in parallel to sbwinproxy and sblinuxproxy
discover.workers=40 Allocates resources to Active Discovery iterations
autoprops.workers=10 The thread pool size for AP
reporter.persistent.queue.consume.rate=12 The max count of data entries that will be reported for each API call.
reporter.persistent.queue.consumer=10 The thread count used to read from buffer and execute reporting.
collector.script.threadpool=200 The max thread count to run script tasks.
watchdog.conf N/A N/A


Large Collector
 

Config File Parameters Description
wrapper.conf wrapper.java.initmemory=1024 Minimum Java Heap Size(MB) for Collector
wrapper.java.maxmemory=4096 Maximum Java Heap Size(MB) for Collector
sbproxy.conf wmi.stage.threadpool.maxsize=400 The maximum size of threads to handle WMI query/fetch data in sbwinproxy.exe
wmi.connection.threadpool.maxsize=200 The maximum size of threads for WMI to connect to remote machine in sbwinproxy.exe
agent.conf sbproxy.connector.capacity=16384 The maximum number of requests that the Collector can send in parallel to sbwinproxy and sblinuxproxy
discover.workers=80 Allocates resources to Active Discovery iterations
autoprops.workers=15 The thread pool size for AP
reporter.persistent.queue.consume.rate=12 The max count of data entries that will be reported for each API call.
reporter.persistent.queue.consumer=15 The thread count used to read from buffer and execute reporting.
collector.script.threadpool=300 The max thread count to run script tasks.
watchdog.conf N/A N/A


 Minimum Recommended Disk Space for Collectors 

Although the Collector operates in memory, certain operations- particularly caching- do require a small amount of available disk space on its host. The exact amount of required storage is highly variable and dependent of several factors such as Collector size, configuration, Netflow usage, number of Collector logs, etc. Nevertheless, here are a few examples of required disk space based on these factors: 

  1. A brand new install Collector will use about 500MB.
  2. At most, Collector logs will use 800MB.
  3. Temporary files (ie. upgrade files) will use less than 1500MB.
  4. Report cache data will use less than 500MB by default (this figure represents 30 minutes of cached data for a Large Collector)
  5. If using Netflow the disk usage is less than 30GB. 

In total, this means Collector disk usage will be, at most, less than 3.5GB without Netflow and less than 33.5GB with Netflow enabled.