Support Center Home


Setting up Azure Logs Ingestion

The Microsoft Azure integration for LM Logs is implemented as an Azure function that consumes logs from an Event Hub and forwards the logs to the LogicMonitor logs ingestion API.

Requirements

  • An Azure Cloud Account created in your LogicMonitor portal.
  • LogicMonitor API tokens to authenticate all requests to the log ingestion API.
  • The Azure CLI tools installed on the machines that will forward logs.
  • A “User Administrator” role in Azure to create the managed identity which will access the Azure resources and logs.
  • Azure devices can only send logs to the Event Hubs within the same region. Each Azure region requires a separate Azure Function deployment.

Configure using templates

We provide Microsoft Templates for you to configure and deploy the Azure Function, create a managed identity to access the Azure resources and logs, and forward the logs to LogicMonitor.

Deploy the Azure function

Click the button below to open the Microsoft template that will deploy the Azure function and create the Event Hub.



When deploying, you will need to provide the following details in the template:

Parameter Description
Region (Required) Select the location where the resource group stores metadata about the resources.

For a list of Azure regions by their display names, see Microsoft’s Azure geographies overview.
resource_group_region (Required) Type in the region where you want to create the resource group and deploy the resources, such as the Event Hub, Function app, and so on.

For a list of the Azure regions in plain text, you can run the following command from PowerShell with the Azure CLI tools installed:

az account list-locations -o table
LM_Company_name (Required) Your LogicMonitor company or account name in the target URL. This is not the fully qualified domain name (FQDN), just the <account> value: https://<account>.logicmonitor.com
LM_Access_Id (Required) The LM API tokens access ID. We recommend an API only user to be used with this integration.
LM_Access_Key (Required) The LM API tokens access key.
Azure_Client_Id (Required) The Application (client) ID that is used while creating the Azure Cloud Account in your LogicMonitor portal.

Note: This ID should have been created when you connected the Azure Cloud Account and can be found in the Azure Active Directory under App Registrations.
Enable Activity Logs (Optional) Specify whether or not to send Activity Logs to the Event Hub that is created with the Azure Function.

Allowed values are Yes or No. Default is Yes.

If the deployment is successful and you Enabled Activity Logs in your configuration, you should start seeing the forwarded logs appear in the LM Logs page. These logs will be mapped to the Azure Cloud Account you created in your LogicMonitor portal.

If you didn’t enable activity logs, then you will need to configure logs to forward to the Event Hub. We’ve provided a template for this process as well. See Forwarding Azure logs to the Event Hub.

Create a managed identity

Note: This is only required if you are forwarding logs with the template provided. Without the managed identity, you can still manually configure Azure resources to forward their logs to the Event Hub.

Click the button below to open the Microsoft Template which will create the Managed Identity with the User Administrator role.



When creating the managed identity, you will need to provide the following details in the template:

Parameter Description
resource_group_region (Required) The region where you want to create the resource group and deploy the resources, such as the Event Hub, Function app, and so on.

For a list of the Azure regions in plain text, you can run the following command from PowerShell with the Azure CLI tools installed:

az account list-locations -o table

Note: The resource group and the resources within it must be in the same region as that of the Event Hub created when you deployed the Azure Function.
LM_Company_name (Required) Your LogicMonitor company or account name in the target URL. This is not the FQDN, just the <account> value: https://<account>.logicmonitor.com

Forward logs using a template

Click the button below to open the Microsoft Template where you can configure log forwarding to the Event Hub. This template forwards the diagnostic settings of selected resources.



When configuring logs forwarding, you will need to provide the following details in the template:

Parameter Description
Resource Group (Required) The resource group from where you want to forward logs to the Event Hub.

For a list of Azure regions by their display names, see Microsoft’s Azure geographies overview.
Subscription ID (Required) The ID for the subscription which consists of all the resource groups.
LM_Company_name (Required) Your LogicMonitor company or account name in the target URL. This is not the FQDN, just the <account> value: https://<account>.logicmonitor.com
Force Update Tag (Optional) Changing this value between template deployments forces the deployment script to re-execute.
Deployment Location (Required) Select the region where this deployment is configured.

Note: While this deployment is running, you can view the deployment logs in the script that gets created in the resource group, for example “lm-logs-<LM-Company-name>-<region-group>”.

Forward Azure logs to the Event Hub

Once the Azure Function is deployed, it listens for logs from the Event Hub. If the Event Hub isn’t receiving any logs, you will need to configure your resources and resource groups to send their logs to the Event Hub. For most Azure resources, this can be done by creating diagnostic settings.

To forward system and application logs from virtual machines you will need to install and configure diagnostic extensions on the virtual machines. See the following instructions for Linux and Windows VM configurations.

Note: These steps are not necessary if you are already using an LM Collector to ingest logs from the VM.

Sending Linux Virtual Machine logs

To configure Linux VMs to forward their system and application logs:

1. Install a diagnostic extension on the VM.

2. Install the Azure CLI

3. Sign in to Azure using the Azure CLI: az login

4. Download the configuration script:

wget https://raw.githubusercontent.com/logicmonitor/lm-logs-azure/master/vm-config/configure-lad.sh

5. Run the script to create the storage account and configuration files needed by the diagnostic extension:

./configure-lad.sh <LM company name>

6. Update lad_public_settings.json to configure types of system logs and their levels (syslogEvents) and application logs (filelogs) to collect.

7. Run the following command to configure the extension:

az vm extension set --publisher Microsoft.Azure.Diagnostics --name LinuxDiagnostic --version 3.0 --resource-group <your VM's Resource Group name> --vm-name <your VM name> --protected-settings lad_protected_settings.json --settings lad_public_settings.json

(The exact command will be printed by the configure-lad.sh script.)

Sending Windows Virtual Machines logs

To configure Windows VMs to forward their system and application logs:

1. Install a diagnostic extension on the VM.

2. Install the Azure CLI using PowerShell: 

Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'; rm .\AzureCLI.msi

3. Sign in to Azure using the Azure CLI: az login

4. Download the configuration script with the following command:

Invoke-WebRequest -Uri https://raw.githubusercontent.com/logicmonitor/lm-logs-azure/master/vm-config/configure-wad.ps1 -OutFile .\configure-wad.ps1

5. Run the configuration script to create the storage account needed by the extension and the configuration files:

.\configure-wad.ps1 -lm_company_name <LM company name>

6. Update wad_public_settings.json to configure types of event logs (Application, System, Setup, Security, and so on) and their levels (Info, Warning, Critical) to collect.

7. Run the following command to configure the extension:

az vm extension set --publisher Microsoft.Azure.Diagnostics --name IaaSDiagnostics --version 1.18 --resource-group <your VM's Resource Group name> --vm-name <your VM name> --protected-settings wad_protected_settings.json --settings wad_public_settings.json

(The exact command was printed by the configure-wad.ps1 script.)

Troubleshooting

Follow the steps below to troubleshoot issues with your Azure logs integration.

1. Confirm that the install process provisioned all the required resources: an Event Hub, a resource group, a storage account, and an Azure Function.

2. Confirm that logs are being sent to the Event Hub.

Navigate to your Event Hub in the Azure portal and check that the incoming messages count is greater than 0.

You can also check this for specific agents or applications by looking in their Azure Logs folder. For example, if you are running a Windows VM with a IaaSDiagnostics extension, its logs will be in the following Azure Logs directory (with version and wadid specified):

C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.Diagnostics.IaaSDiagnostics<VERSION><WADID>\Configuration

3. Confirm that the Azure Function is running and forwarding logs to LogicMonitor. See Enable debug logging.

If the function app is running and receiving logs, but you are not seeing the logs in LogicMonitor, confirm that the LM_Access_Key or LM_Access_Id provided is correct.

If the function app is not being executed, but logs are sent to the Event Hub, try to run the Azure function locally and check if it receives any log messages:

  • If the local function receives logs, stop and run the function on the Azure Portal. (You can check its logs using the Azure CLI.)
  • If the local function does not receive logs, you may want to check its connection string and the shared access policy of the Event Hub.

4. You can use PowerShell to send a test event from the log-enabled VM. On the configured device, enter the PowerShell prompt and run the following command:

eventcreate /Id 500 /D "test error event for windows 222222" /T ERROR /L Application

After the command runs, you will see the event show up in the LM Logs page.

Enable debug logging

For logs forwarded from Microsoft Azure, you can enable Application Insights in the Function App to check whether it is receiving logs. Refer to Microsoft’s documentation about Streaming Logs.

You can configure the application logging type and level using Azure CLI webapp log config command, for example:

az webapp log config --resource-group <Azure Function's Resource Group name> --name <Azure Function name> --application-logging true --level verbose --detailed-error-messages true

After configuring application logging, you can review the logs using Azure CLI webapp log tail:

az webapp log tail --resource-group <Azure Function's Resource Group name> --name <Azure Function name>

How to remove Azure functions

The Azure templates you ran to set up log ingestion create several resources, including the Event Hub, which sends the logs data to LM Logs.

To remove the LM Logs integration and stop the flow of data and any associated costs, follow these steps:

1. In your Azure portal, navigate to the monitored VM > Activity log > Diagnostic settings > Edit setting (for the Logs Event Hub) and click Delete.

2. Delete the Event Hub which has the name and region name that you created during setup. This will now cut off the logs flow from Azure to LM Logs.

3. (Optional) You can remove all other resources, such as the Function App, Managed Identity, App Insight, and Storage account. The names of these will following the Event Hub naming convention from the template. You can remove each item individually, or if they are in a resource group you can remove the entire group.

Note: Before removing the resource group, insure that you have not added other non-LM Logs items into the group.

In This Article