Configuring WinRM for Windows Collector

Security of customer data and our portal is of paramount importance to LogicMonitor. We ensure that vulnerabilities are preempted and handled quickly. Additionally, many customers requested for reducing the privileges to run Windows collectors. As part of our initiative to reduce attacks by malicious entities, we have adopted a principle of least privileges. To implement this initiative, we have developed a Windows Remote Management (WinRM) based solution where Windows collectors are installed without administrator privilege.

Along with making LogicMonitor more secure, we are also moving away from a WMI based solution to a WinRM based solution. WMI uses DCOM which requires multiple ports, comparatively less secure, and less Network Address Translation (NAT) friendly. On the other hand, WinRM uses WS-Management, a Simple Object Access Protocol (SOAP) based protocol for the management of servers, devices, applications, and various web services. It can be easily meted out via Active Directory, by default it is encrypted, and uses HTTPS and one port. This makes it more secure, firewall friendly, and easy to deploy Windows collectors.

Advantages of Using WinRM

WinRM is Microsoft’s implementation of WS-Management Protocol, a standard SOAP-based firewall friendly protocol that allows hardware and operating systems from different vendors to interoperate. Thus, WinRM provides a way to get management data from a local computer or a remote computer using WS-Management protocol.

If a monitoring device is Windows-based, then its management data can be read through WMI. In addition to Windows, WinRM also monitors devices that support WS-Management protocol. Currently, collector gets this management data from local device or remote device using WMI via DCOM/WBEM. The WS-Management protocol returns data in XML rather than objects such as DCOM/WBEM. Thus, due to extensibility and interoperability of WS-Management protocol compared to DCOM/WBEM, using WinRM for data collection is highly beneficial.

Prerequisites

You must meet the following prerequisites to configure and use WinRM to query data.

• You must install EA Collector 34.100 or a later version.
• You must setup Domain Controller (DC). Machines that you want to monitor via WinRM must be part of the domain. So, a proper Domain Control forest setup should be present. To check if a machine is part of the domain, go to About PC > Machine name.
• You must create a domain user who is not a member of the Administrators group. See, Creating Domain User.
• You must create and enable an HTTPS listener on the Windows devices. See, Creating an HTTPS Listener (port 5986).

Creating Domain User

You can create a new domain user or use an existing domain user. However, ensure that the domain user is not a member of the Administrators group. If you are going to create a new domain user, it is recommended that you first create a domain user and then install the collector using that domain user.

1. Navigate to Server Manager > Tools > Active Directory Users and Computers. 
2. Go to the domain that you want to use for WinRM and right-click Users and select New.
3. Enter the new user details and select Next.
   
4. Enter a password and confirm it, and then select Next.
   
5. Check details and select Finish.
   

Creating an HTTPS Listener (port 5986)

An HTTPS listener must be created and/or enabled on the Windows devices. If you do not have an HTTPS listener created for WinRM, follow the steps given below:

1. Run the command WinRM e winrm/config/listener in cmd_prompt to check if port 5986 is already enabled on WinRM service.
2. Use an existing signed certificate (signed by a valid CA server).

3. Copy the thumb print of the existing certificate and run the following command in cmd_prompt.
   winrm create winrm/config/Listener?Address=*+Transport=HTTPS '@{Hostname="<YOUR_DNS_NAME>"; CertificateThumbprint="<COPIED_CERTIFICATE_THUMBPRINT>"}‘
4. To verify if the listener is created, run the command WinRM e winrm/config/listener in cmd_prompt to print details of port 5985 and 5986.
5. To add a new firewall inbound rule to allow all connections for port 5986 (TCP), follow these steps:
   1. Search and select the Windows Firewall with Advanced Security option. The Windows Firewall with Advanced Security window is displayed.
   2. In the left navigation, right-click Inbound Rules and select New Rule. The New Inbound Rule Wizard window is displayed.
   3. Select the Port radio button and select Next.
   4. Select the TCP radio button.
   5. Select the Specific local ports radio button and enter 5986 in the blank field. Select Next. 
   6. Select the Allow the connection radio button and select Next.
   7. Select the Domain, Private, and Public checkboxes and select Next.
   8. Enter a rule name in the Name field and select Finish.

Setting Up WinRM Enabled Collector

To setup WinRM enabled collector, follow these steps:

1. Install Windows collector
2. Configure WinRM properties in agent.conf
3. Migrate automated WinRM scripts
4. Create GPO policy
5. Add WinRM tasks to the GPO policy
6. Run the policy in PowerShell

Installing Windows Collector

Follow the instructions given in Installing Collectors to install Windows Collector. Ensure that you install EA Collector 34.100 or a later version.

Configuring WinRM Properties in agent.conf

To use the WinRM data collection feature using HTTP (port 5985), you must configure the following properties in the agent.conf file.

Restart the collector after you update the agent.conf file with the configuration properties. 

Migrating Automated WinRM Scripts

We have automated the WinRM configuration process by adding two scripts: WinRM_Config_02.ps1 and WinRM_Config_01.ps1. You must consider the following points about the scripts:

• The scripts are bundled with the Windows collector installer. 
• After installing collector, go to \LogicMonitor\Agent\bin and move the scripts to a shared folder. 
• All the machines in the domain should have access to the shared folder in order to access these scripts when executed through Group Policy (GPO).

Purpose of running the scripts:

• WinRM_Config_01.ps1 – To monitor WMI classes, especially Win32_Service, users must have certain permissions. WinRM_Config_01.ps1 gives the following read only permissions to users: LC – List Content, RP – Read Properties, and RC – Read Permissions.
• WinRM_Config_02.ps1 – It makes necessary changes by adding users to the 3 user groups: Performance Monitor Users, Event Log Readers, and Remote Management Users. It also provides correct WMI namespace security rights: Enable, MethodExecute, ReadSecurity, and RemoteAccess.

Creating GPO Policy

1. Navigate to Server Manager > Tools > Group Policy Management.
2. Right-click the domain you want to use and select Create a GPO in this domain, and Link it here…
   
3. Enter a name for the new GPO policy (for example, WinRM Configuration) and then select OK. Note that the Source Starter GPO will remain (none).
   
4. Right-click the newly created GPO policy and select Edit to add WinRM tasks.
   

Adding WinRM Tasks to GPO Policy

1. In the Group Policy Management Editor, navigate to Computer Configuration > Policies > Windows Settings > Scripts (Startup / Shutdown) and select Startup.
   
   A Startup Properties dialogue box is displayed.
2. Under the Scripts tab select Add.
   
   An Add a Script dialogue is displayed.
3. On the Add a Script dialogue, enter the following details.
   1. In the Script Name field browse for the PowerShell.exe path. Typically, the path is C:\Windows\System32\WindowsPowerShell\v1.0\Powershell.exe
Note: If the path differs, browse for the specific folder where you have placed the Powershell.exe.
   2. In the Script Parameters field enter the following parameters for the WinRM_Config_01.ps1 script.
      -Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_01.ps1 -add -UserName logicmonitor\DomainUser01 -log True
      
4. Select OK and then select Apply.
5. Navigate to Computer Configuration > Preferences > Control Panel Settings > Scheduled Tasks.
6. Right-click Scheduled Tasks and select New > Immediate task (At least Windows 7).
   
   A New Task (At least Windows 7) Properties dialogue box is displayed.
   1. Enter the name and description of the user.
   2. Select Change User or Group. A Select User or Group dialogue box is displayed.
   3. Type Administrator in the Enter the object name to select text area and select Check Names.
      
      Multiple Names Found dialogue is displayed.
   4. Select Administrator in folder (<Domain>/Users) from the matching names list.
      
   5. Select OK to close the Multiple Names Found and Select User or Group dialog boxes.
   6. Select the Run with highest privileges checkbox.
   7. In the Configure for drop down, select Windows 7, Windows Server 2008R2
      
7. In the Actions tab, select New… A New Action dialog box is displayed. Enter the following details to specify the action this task is going to perform.
   1. Select Start a program in the Action drop down.
   2. In the Program/script field browse for the location of the PowerShell.exe. It is located at – C:\Windows\System32\WindowsPowerShell\v1.0\Powershell.exe
   3. In the Add Arguments (optional) field enter the following path and parameters for WinRM_Config_02.ps1 script.
      -Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_02.ps1 -add -UserName logicmonitor\DomainUser01 -log True
      
   4. Select OK.
8. Go to the Common tab and select the Apply Once and do not reapply checkbox. 
9. Select the Item-level targeting checkbox and then select Targeting…. Targeting Editor dialog box is displayed.
10. Select New Item from the drop down.
     
11. In the Computer name field, enter the Domain Controller computer’s name (DC1 in our example).
    
12. Select OK and then Apply.

Running Policy in PowerShell

1. Open PowerShell terminal and run gpupdate/force.
   
2. Once the user policy is successfully updated, restart the domain machines.

Logging Script Result

The flag -log True indicates if the execution of the scripts is logged or not. If the flag is set, the result is logged; else it is not logged. For example:

-Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_01.ps1 -add -UserName logicmonitor\DomainUser01 -log True

-Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_01.ps1 -add -UserName logicmonitor\DomainUser01

Log files are created in the %appdata%\LogicMonitor\Logs folder.

Using the Debug Command to Verify Configuration

Once the configuration is complete, you can verify it by running the !testwinrmconfig debug command. Depending on the setup you can see message in form of 3 tiers. If you face issue with WinRM monitoring, run the following command to verify the configuration.

!testwinrmconfig host=<Computer Name> [auth=<Auth Type>] [useSSL=true|false] [username=foo] [password=var]

Where,

host – (Mandatory) Name of the remote domain computer which you want to monitor. 
auth – Supported authentication types – Kerberos and Default. The default value is Default.
useSSL – WinRM data collection over HTTPS. The default value is true.
username and password – Domain user credentials. Will use wmi.user / wmi.pass if not passed.

!testwinrmconfig host=EC2AMAZ-93376C4.logicmonitor.com auth=Kerberos username=LOGICMONITOR\Aditya password=Aditya@123

You can see the following result:

Using Validation Flag

You can use the -validate True flag to check if the WinRM_Config_02.ps1 script is applied or not. The flag will check if the passed user -UserName is added to the 3 WinRM related groups on all the domain devices including the domain controller.

Although this step is optional, if you run the script as a scheduled task (instead of one time immediate task) this flag will save repeated processing and result in fewer PowerShell events.

Some combinations with and without this flag are as follows:

-Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_02.ps1 -add -UserName logicmonitor\NewUser04 -validate True

-Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_02.ps1 -add -UserName logicmonitor\NewUser04

-Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_02.ps1 -add -UserName logicmonitor\NewUser04 -log True -validate True

Rolling Back WinRM Configuration

You can revert the changes done by WinRM_Config_01.ps1 and WinRM_Config_02.ps1 scripts. Refer the following examples:

• -Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_01.ps1 -remove -UserName logicmonitor\ScriptTestUser01 -log True
• -Noninteractive -ExecutionPolicy Bypass -Noprofile -file \\10.55.21.12\To\WinRM_Config_02.ps1 -remove -UserName logicmonitor\ScriptTestUser02 -log True

Follow the steps given in Adding WinRM Tasks to GPO Policy section but run the scripts with remove flag to roll back WinRM configuration.

Limitation

Before introducing WinRM, WMI data was collected via SbProxy. With the new WinRM initiative, WMI data will be collected by PowerShell script via SPSE (script engine for executing PowerShell scripts). Because of this you may observe increased load on SPSE, we are working to improve the scalability of SPSE.

Troubleshooting

WQL filter issue for groovy scripts modules containing wmi.queryfirst() and wmi.queryAll()

WQL filter supports the following 2 formats:

• Format 1 – “select * from win32_bios where NAME LIKE ‘%’” – Double quotes surround the WQL and the clause value is enclosed with single quotes.
• Format 2 – ’select * from win32_bios where NAME LIKE “%” ’- Single quotes surround the WQL and the clause value is enclosed with double quotes.

WinRM (when you set wmi.winrm.enable= true) supports only the format 2. If you have enabled WinRM and if for custom datasource you use the format 1, then you will see errors on calling wmi.queryFirst() / wmi.queryAll() in the groovy script. To avoid the errors, we recommend you to use the format 2.

Intermittent issue – Network path not found

Due to issues with network or domain you may get the “Network path not found” message.

Error Message : Exception calling "Invoke" with "2" argument(s): "The network path was not found.".Exception.Messages

When you find this error message in the logs, to troubleshoot it, reapply the scripts as a fresh task.