General Deployment Collector (optional) - 21.0

General Deployment Collector 21.0 is an optional collector update, available the week of March 14th.   It comprises of all of the features and fixes added after required release GD 19, including everything in optional GD 20 and early release collector versions 20.059, 20.074 and 20.083.  GD 21 provides most of what is going to be included our upcoming mandatory collector version.  

NOTE:

• RC4 related ciphers are disabled in this collector version, which may cause data collection issues for servers which utilize RC4 connections.  
• This collector version may cause SNMPv3 data collection task timeouts in some cases, due to engine time and engine boots not being synchronized correctly.
• Note that this Collector version requires that you enable TLS support for monitored 7-mode NetApp devices. You can enable TLS support on your NetApp devices using the command  “options tls.enable on”.

We will be releasing a new collector version with RC4 ciphers re-enabled & a fix for the SNMPv3 timeouts.

Here’s a summary of what is included in GD 21:

New Features

• A new BatchScript collection method is available for Datasources.  Our existing Datasource Script collection method polls data per discovered instance.  The new BatchScript collection method allows your collector to poll data for multiple instances at once, which is useful for script datasources that are collecting data across a large number of instances.  
• SNMP v3 data collection now supports the contextName and contextEngineId fields, which are used to distinguish multiple SNMP applications within the same SNMP engine. 
• Eventsource collection can now happen via script, eliminating the need for IPMI collection methods.  
• The collector will now determine if a monitored device has a collector installed on it.  If so, the collector datasources will automatically apply.
• Windows collectors now support embedded PowerShell scripting in datasources. The embedded PowerShell feature behaves just like embedded groovy scripting, in that PowerShell can be used both in Active Discovery and datapoint collection scripts. In the collector debug window there’s now a !posh command for testing PowerShell scripts that’s analogous to the existing !groovy command. Existing datasources using external PowerShell scripts (using the upload functionality) will continue to behave as always.
• This collector version supports metrics for three new collector datasources, Data Collecting Task, Event Collecting Task, and Collector Status.  These datasources provide metrics regarding how many instances a collector is monitoring, it’s current version, uptime, and task counts. 

Other Improvements

• Added more accurate status information when data collection is initializing for Perfmon datasources.
• We now utilize TLS 1.2 for reporting data back to LogicMonitor as well as for webpage and CIM collection tasks.
• Added validation for EventIDs in event datasources, including a maximum value of 65536.
• Added support for collector installations on newer versions of Ubuntu, RedHat, Centos and other operating systems using systemd.
• Added timeouts to collector initialization operations to avoid collector watchdog tasks getting stuck in a starting state.
• Added additional Netapp configurations (netapp.connection.concurrency# and netapp.connection.threadPoolSize)  to manage the number of concurrent requests allowed to a netapp host.
• Improved Netflow to handle empty flows and delays in data refreshing.
• Improved how WMI reports no data, better distinguishing between the device not returning data and the collector having issues reporting data
• Optimized SNMP, Ping and Windows events to support 1000 devices.
• Optimized collection for large Netapp clusters to avoid collection tasks getting stuck in a waiting state.
• Enhanced ElastiCache Active Discovery.
• Restructured AWS S3 billing data collection to use CloudWatch data.
• Updated the ESX active discovery script to ignore IDs, allowing for discovery of all VMWare systems.
• Added the option to ignore NetFlow timestamps, commonly needed by SonicWall devices that do not report the current time in a common format.
• Optimized SNMPv3 to fail over to SNMPv2 and SNMPv1.
• Improved the webpage collector to avoid connections remaining open.

Upgrades

• The Collector’s JRE has been upgraded to JDK 8.  This addresses the security issue identified in CVE-2014-6593 and supports the GCM cipher suite.  Please see this page for the operating systems and configurations officially supported for JRE8.
• The Java Service Wrapper under which the collector runs has been replaced with YAJSW.  This change addresses both a potential memory leak situation and allows for better collector troubleshooting. Additionally, the YAJSW wrapper provides more flexibility and support for future collector enhancements.
• The MySQL JAR has been upgraded to version 5.1.38.
• The Joda Time JAR has been upgraded to version 2.9.1
• The Jsch JAR has been upgraded to version 0.1.53 
• The Mongo-Java-Driver JAR has been updated to version 2.13.1

Bug Fixes

• ESX data collection sometimes resulted in HostStatus alerts not being triggered for dead devices. This has been corrected.
• sFlow ports were previously being reported incorrectly in some cases.
• Fixed an issue with SNMP v3 data collection related to task timeouts.
• TCP data collection tasks no longer fail when ‘Connection reset’ is encountered.
• An empty auto discovery parameter was causing auto discovery to fail.
• We previously weren’t always correctly processing NaN in complex datapoints. We’ve corrected this by enabling conversion of NaN to 0.
• Glob was not accurately supported for windows file names in Logfile EventSources.
• Fixed an issue with the event collector to support batch job alerts that were receiving access not allowed errors.
• Any string returned via perfmon would cause all datapoints to be reported as NaN.
• If netflow collection initialized during other collector processes, it was possible the script could be terminated.  This potential conflict has been removed.
• Added the ability to manually disable event collectors.  Previously this change could be made in the collector but did not take effect.
• If a network error occurs during a script download for a datasource, the previous version will persist.
• A previous JDK upgrade caused an issue with AWS EC2 Netscan policies, the Netscan processing has been adjusted to account for the conflict. 
• Corrected async SNMP collection timeouts.
• We’ve added validation to avoid blank device or group properties interfering with data collection.
• The collector was reporting an unknown mongo status error in place of the correct timeout status.