Support Center Home


Setting up Azure Logs Ingestion

The Microsoft Azure integration for LM Logs is implemented as an Azure function that consumes logs from an Event Hub and forwards the logs to the LogicMonitor logs ingestion API. You can deploy the Azure Function using Terraform in the Azure CLI or using a Microsoft Template that we provide.

Prerequisites

  • LogicMonitor API tokens to authenticate all requests to the log ingestion API.
  • The Azure CLI installed.
  • A “User Administrator” role in Azure to create the managed identity.
  • Azure devices can only send logs to the Event Hubs within the same region. Each Azure region requires a separate Azure Function deployment.

Deploying the Azure Function

You have two options for deploying the Azure Function: deploy from the Azure CLI using Terraform and deploy using templates that we provide.

Deploy using Terraform

Terraform is included in the Azure CLI. To deploy the Azure Function using Terraform:

1. Sign in to Azure using the Azure CLI: az login

2. Download the file: deploy.tf

This script creates the Event Hub, resource group, and storage account needed in your Azure environment to run the Azure function.

3. Start Terraform: terraform init

4. Run the following command, filling in the necessary variables:

terraform apply --var 'lm_company_name=<ACCOUNT>' --var 'lm_access_id=<ID>' --var 'lm_access_key=<KEY>' --var 'azure_region=<REGION>'

If there are no errors, the Azure function will deploy and start. If the Azure function doesn’t start, you may need to restart the function app on the Azure Portal. See this issue report for more details.

5. After the Azure function is deployed, the next step is to send logs to the Event Hub. In this Terraform configuration, the logs should be sent to an Event Hub named log-hub in namespace lm-logs-<LM company name>-<Azure region>. For most Azure resources, this can be done by creating diagnostic settings in your Azure portal. You can also send system and application logs from virtual machines to the Event Hub.

Once this is complete, you should start seeing the forwarded logs appear in the LM Logs page.

Deploy using a template

As an alternative way to set up Azure logs ingestion, we’ve provided a Microsoft template that will deploy the Azure function and create the Event Hub. Click the button below to start:



When deploying, you will need to provide the following details in the template:

Parameter Description
Region (Required) This is the location where the resource group stores metadata about the resources.
resource_group_region (Required) The region where you want to create the resource group and deploy the resources, such as the Event Hub, Function app, and so on.
LM_Company_name (Required) Your LogicMonitor company or account name in the target URL: https://<account>.logicmonitor.com
LM_Access_Id (Required) The LM API tokens access ID. We recommend an API only user to be used with this integration.
LM_Access_Key (Required) The LM API tokens access key.
Azure_Client_Id (Required) The Application (client) ID that is used while creating the Azure Cloud Account in your LogicMonitor portal.
Enable Activity Logs (Optional) Specify whether or not to send Activity Logs to the Event Hub that is created with the Azure Function. Allowed values are Yes or No. Default is Yes.

If the deployment is successful and you Enabled Activity Logs in your configuration, you should start seeing the forwarded logs appear in the LM Logs page. These logs will be mapped to the Azure Cloud Account you created in your LogicMonitor portal.

If you didn’t enable activity logs, then you will need to configure logs to forward to the Event Hub. We’ve provided a templates for this process as well. See Forwarding Azure logs to the Event Hub.

Forwarding Azure logs to the Event Hub

Once the Azure Function is deployed, it listens for logs from the Event Hub. If the Event Hub isn’t receiving any logs, you will need to configure your resources and resource groups to send their logs to the Event Hub. For most Azure resources, this can be done by creating diagnostic settings. You can also send system and application logs from virtual machines to the Event Hub.

We provide a template that will automatically configure Azure resources to send their logs to the Event Hub. To use that template, you first need to create a managed identity with the permissions required to access the Azure resources and logs. (We also provide a template that will create the managed identity for you.)

Create a managed identity

Note: This is only required if you are forwarding logs with the template provided. Without the managed identity, you can still manually configure Azure resources to forward their logs to the Event Hub.

Click the button below to open the Microsoft Template which will create the Managed Identity with User Administrator role.



When creating the managed identity, you will need to provide the following details in the template:

Parameter Description
resource_group_region (Required) The region where you want to create the resource group and deploy the resources, such as the Event Hub, Function app, and so on.

Note: The resource group and the resources within it must be in the same region as that of the Event Hub created when you deployed the Azure Function.
LM_Company_name (Required) Your LogicMonitor company or account name in the target URL: https://<account>.logicmonitor.com

Forward logs using a template

Click the button below to open the Microsoft Template where you can configure log forwarding to the Event Hub. This template forwards the diagnostic settings of selected resources.



When configuring logs forwarding, you will need to provide the following details in the template:

Parameter Description
Resource Group (Required) The resource group from where you want to forward logs to the Event Hub.
Subscription ID (Required) The ID for the subscription which consists of all the resource groups.
LM_Company_name (Required) Your LogicMonitor company or account name in the target URL: https://<account>.logicmonitor.com
Force Update Tag (Optional) Changing this value between template deployments forces the deployment script to re-execute.

Note: While this deployment is running, you can view the deployment logs in the script that gets created in the resource group, for example “lm-logs-<LM-Company-name>-<region-group>”.

Forwarding Virtual Machine logs to the Event Hub

Forwarding system and application logs from virtual machines requires installing and configuring diagnostic extensions on the machines. See the following instructions for Linux and Windows VM configurations.

Forward Linux Virtual Machine logs

To configure Linux VMs to forward their system and application logs:

1. Install a diagnostic extension on the VM.

2. Sign in to Azure using the Azure CLI: az login

3. Install wget: sudo apt-get install wget

4. Download the configuration script:

wget https://raw.githubusercontent.com/logicmonitor/lm-logs-azure/master/vm-config/configure-lad.sh

5. Run the script to create the storage account and configuration files needed by the diagnostic extension:

./configure-lad.sh <LM company name>

6. Update lad_public_settings.json to configure types of system logs and their levels (syslogEvents) and application logs (filelogs) to collect.

7. Run the following command to configure the extension:

az vm extension set --publisher Microsoft.Azure.Diagnostics --name LinuxDiagnostic --version 3.0 --resource-group <your VM's Resource Group name> --vm-name <your VM name> --protected-settings lad_protected_settings.json --settings lad_public_settings.json

(The exact command was printed by the configure-lad.sh script.)

Forwarding Windows Virtual Machines

To configure Windows VMs to forward their system and application logs:

1. Install a diagnostic extension on the VM.

2. Install Azure CLI via PowerShell: 

Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'; rm .AzureCLI.msi

3. Sign in to Azure using the Azure CLI: az login

4. Download the configuration script:

https://raw.githubusercontent.com/logicmonitor/lm-logs-azure/master/vm-config/configure-wad.ps1 -OutFile .configure-wad.ps1

5. Run the configuration script to create the storage account needed by the extension and the configuration files:

.configure-wad.ps1 -lm_company_name <LM company name>

6. Update wad_public_settings.json to configure types of event logs (Application, System, Setup, Security, and so on) and their levels (Info, Warning, Critical) to collect.

7. Run the following command to configure the extension:

az vm extension set --publisher Microsoft.Azure.Diagnostics --name IaaSDiagnostics --version 1.18 --resource-group <your VM's Resource Group name> --vm-name <your VM name> --protected-settings wad_protected_settings.json --settings wad_public_settings.json

(The exact command was printed by the configure-wad.sh script.)

Troubleshooting

Follow the steps below to troubleshoot issues with your Azure logs integration.

1. Confirm that the install process provisioned all the required resources: an Event Hub, a resource group, a storage account, and an Azure Function.

2. Confirm that logs are being sent to the Event Hub.

Navigate to your Event Hub in the Azure portal and check that the incoming messages count is greater than 0.

You can also check this for specific agents or applications by looking in their Azure Logs folder. For example, if you are running a Windows VM with a IaaSDiagnostics extension, its logs will be in the following Azure Logs directory (with version and wadid specified):

C:WindowsAzureLogsPluginsMicrosoft.Azure.Diagnostics.IaaSDiagnostics<VERSION><WADID>Configuration

3. Confirm that the Azure Function is running and forwarding logs to LogicMonitor. See Enable debug logging.

If the function app is running and receiving logs, but you are not seeing the logs in LogicMonitor, confirm that the LM_Access_Key or LM_Access_Id provided is correct.

If the function app is not being executed, but logs are sent to the Event Hub, try to run the Azure function locally and check if it receives any log messages:

a. If the local function receives logs, stop and run the function on the Azure Portal. (You can check its logs using the Azure CLI.)

b. If the local function does not receive logs, you may want to check its connection string and the shared access policy of the Event Hub.

Enable debug logging

For logs forwarded from Microsoft Azure, you can enable Application Insights in the Function App to check whether it is receiving logs. Refer to Microsoft’s documentation about Streaming Logs.

You can configure application logging type and level using Azure CLI webapp log config command, for example:

az webapp log config --resource-group <Azure Function's Resource Group name> --name <Azure Function name> --application-logging true --level verbose --detailed-error-messages true

After configuring application logging, you can review the logs using Azure CLI webapp log tail:

az webapp log tail --resource-group <Azure Function's Resource Group name> --name <Azure Function name>

In This Article