Troubleshooting Perfmon Access
Chances are that if you are an avid Windows user, you have probably come across the Perfmon utility at least once in your exploration of system & network monitoring.
Most of LogicMonitor's Windows data collection uses WMI queries, but we do utilize Perfmon counters for our Windows SQL Server, Exchange, SMTP, & Terminal Services monitoring datasources. If you are having gaps of No Data for these datasources in particular, but the rest of your data collection (CPU, Disk, Memory, Ping, DNS metrics) is stable, then you most likely have an issue with Perfmon.
Because Perfmon accesses performance counters from remote hosts, it is necessary that your collector services have Local Administrative privileges to access counters on your remote host if you are using a domain. If you are using a workgroup, your collector will need to be running under a user that has Local Administrative rights on the remote workgroup host that you are attempting to monitor.
NOTE: If the WMI credentials set for your device include a domain\user, but the remote computer is in a different domain, and the user is local, you may need to define pdh.user and pdh.pass properties to access Perfmon data. If pdh.user and pdh.pass properties are defined on your device, they will be used over any wmi username and password properties defined for collecting Perfmon data.
Perfmon Connection Timeouts & Latency
The most common symptoms seen when troubleshooting Perfmon instability are issues where Perfmon is having difficulty initiating a connection to a remote host. This must succeed between the collector and host in order for the LogicMonitor ActiveDiscovery mechanism to detect which Perfmon performance counters are available on the remote host and to read data from the host.
After datasource instances have been added to the host, Perfmon connectivity must continue to function in order for data collection to operate in a stable manner. If connectivity stops after datasource instances have been matched to the host, you will end up with blank or spotty graphs that are returning No Data.
To troubleshoot these connectivity issues:
- Remote into the collector machine under the user that your collector services are running under, and open a Run prompt > "perfmon.exe" to access the Perfmon GUI.
- Use the green "+" option to "Add" a new count.
- Specify the UNC path of your remote machine. When you are finished adding the path, Perfmon will pause while it automatically attempts to make a connection
- If the connection is successful, you should be returned with a listing of all of the available Perfmon performance counters for your remote machine.
- If the connection fails, please attempt the same procedure from the local collector to itself, and from the local host to itself, to isolate the issue down to the host machine or the collector machine.
If you are unable to make a connection using the perfmon.exe utility on your collector to your remote host, the collector services will be unable to do so as well. This indicates an issue not with LogicMonitor, but with your Windows configuration or authentication.
Services & Dependencies
If WMI-based data collection succeeds, but Perfmon datasources fail & you cannot get a connection established via the Perfmon utility, the issue may be that certain services need to be running on the host being monitored for Perfmon to respond to RPC queries.
The following services need to be set to an Automatic startup type:
- Remote Procedure Call (RPC)
- Remote Registry
And the following services need be set to Manual or greater startup type:
- WMI Performance Adapter
- Performance Counter DLL Host
- Performance Logs and Alerts
- Remote Procedure Call (RPC) Locator
Perfmon relies on inbound RPC port 135 TCP and Windows SMB port 445 TCP on the host. When troubleshooting Perfmon connectivity issues, please ensure that these ports are unrestricted in your firewall configuration.
If that doesn't help and you suspect permission issues are to blame, you can work around this via regedit:
- Open regedit on the machine to which you are trying to connect to perfmon.
- Browse to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
- Right click on Perflib key and select permissions
- Click Add and add Local Service with full control
- Save and exit
- Restart the Remote Registry Service
Rebuilding Perfmon Performance Counters
If you happen to get a response via Perfmon utility & from the collector debugger, but you get an error similar to "Object Not Found" when you attempt to view a counter, please attempt the below counter repairs:
- cd c:\windows\system32
- lodctr /R
- cd c:\windows\sysWOW64
- lodctr /R
- WINMGMT.EXE /RESYNCPERF
- A restart is advised, but if not possible:
- Stop and restart the Windows Management Instrumentation service (set to automatic if not already).
- Stop and restart the Performance Logs and Alerts service (set to automatic if not already).
Also, see SQL Server monitoring
For additional help with troubleshooting your Windows Perfmon data collection, please submit a support ticket or use the chat with engineer link.