Lambda Function Deployment for AWS Logs Ingestion

Lambda function deployment is the process of packaging and uploading code to Amazon Web Services (AWS) Lambda to run deployment packages, or the bundled resources required to run your code such as dependencies, libraries, and so on. Lambda supports the following two types of deployment packages:

• Container images—Used to configure a function 
• .zip file archives—Used to create a function using the Lambda console or an API

During deployment, the Lambda function can gather log information from various applications by sending it to a centralized logging service, such as Amazon CloudWatch. The Amazon Web Services integration for LM Logs sends AWS CloudWatch logs to LogicMonitor using a Lambda function configured to forward the log events. For more information, see the AWS integration for LM Logs repository in GitHub.

There are two methods to automate the process of forwarding the logs to LogicMonitor: 

• AWS CloudFormation Stack templates—Uses scripts written in JSON or YAML that automatically set up and manage AWS resources as a single, repeatable unit. You can leverage a template in LogicMonitor’s GitHub repository by configuring parameters that are mapped to a specific CloudWatch logs group for LogicMonitor. For more information, see the AWS integration for LM Logs repository in GitHub.
• Terraform configuration—Uses scripts written in HCL (HashiCorp Configuration Language) that automate the setup and management of cloud infrastructure across various providers.

Requirements for Deploying a Lambda Function

To deploy Lambda Function using CloudFormation, you need the following:

• LogicMonitor API tokens—The log ingestion API uses the API token to authenticate requests. For more information, see Adding an API Token.
• CloudFormation stack template—AWS uses a CloudFormation stack template to deploy CloudFormation stacks with the CloudFormation console. For more information, see Create a stack from the CloudFormation console from Amazon.

To deploy Lambda Function using Terraform, you need the following:

• LogicMonitor API tokens—The log ingestion API uses the API token to authenticate requests.

CloudFormation Stack Options

The following table lists the parameters you define when configuring your CloudFormation stack:

CloudFormation Permissions

Deploying the CloudFormation Stack with default options requires permissions to save a secret, create an S3 bucket for the Forwarder code, and create Lambda functions with roles and logs. The following is an example of the required permissions:

{
           "Effect": "Allow",
           "Action": [
               "cloudformation:*",
               "secretsmanager:CreateSecret",
               "secretsmanager:TagResource",
               "secretsmanager:DeleteSecret",
               "s3:CreateBucket",
               "s3:GetObject",
               "s3:PutEncryptionConfiguration",
               "s3:PutBucketPublicAccessBlock",
               "s3:DeleteBucket",
               "iam:CreateRole",
               "iam:GetRole",
               "iam:PassRole",
               "iam:PutRolePolicy",
               "iam:AttachRolePolicy",
               "iam:DetachRolePolicy",
               "iam:DeleteRolePolicy",
               "iam:DeleteRole",
               "iam:TagRole",
               "lambda:TagResource",
               "lambda:CreateFunction",
               "lambda:GetFunction",
               "lambda:GetFunctionConfiguration",
               "lambda:GetLayerVersion",
               "lambda:InvokeFunction",
               "lambda:PutFunctionConcurrency",
               "lambda:AddPermission",
               "lambda:RemovePermission",
               "lambda:DeleteFunction",
               "logs:CreateLogGroup",
               "logs:DescribeLogGroups",
               "logs:PutRetentionPolicy",
               "logs:PutSubscriptionFilter",
               "logs:DeleteSubscriptionFilter"
           ],
           "Resource": "*"
}

Deploying the Lambda Function Using AWS CloudFormation

You can deploy the Lambda function using the LogicMonitor GitHub repository. See the AWS integration for LM Logs repository in GitHub.

1. In the AWS integration for LM Logs repository, navigate to the README and select Launch Stack. 
2. Configure stack options in the template. For more information, see the CloudFormation stack options table.

After you create the stack, a Lambda function is deployed and subscribed to the specific CloudWatch Logs group to forward logs to LogicMonitor.

If your CloudWatch log group does not already include the logs you want to forward, see AWS Service Configuration for Log Ingestion.

If logs are sent to the correct CloudWatch Logs group, the Lambda function will automatically forward them to the log ingestion API.  The logs and log anomalies will display in the UI on the Logs and Alerts pages. 

Deploying the Lambda Function Using Terraform

Open the command line and run the following Terraform command in the directory that the Terraform configuration is located (filling in the necessary variables):

# terraform apply --var 'lm_access_id=<lm_access_id>' --var 'lm_access_key=<lm_access_key>' --var 'lm_company_name=<lm_company_name>'--var 'lm_company_domain=<lm_company_domain>'`

For more information, see the Sample Configuration for the LM Logs Forwarder in GitHub.

Troubleshooting

To help troubleshoot logs forwarded from Amazon CloudWatch, enable debug logging in your Lambda logs:

1. In the AWS console, go to AWS Lambda > Functions and select “LMLogsForwarder” (or whatever you named the log forwarding Lambda function during setup).

2. Add an environment variable with the key DEBUG and value true.

Migrating AWS Lambda runtime from Go1.x to Amazon Linux 2

Lambda function forwarder is written in Golang. As AWS is deprecating Go1.x runtime, it is encouraging its customers to upgrade to Amazon Linux 2 runtime.

To migrate AWS Lambda runtime from Go1.x to Amazon Linux 2 runtime, do the following:

1. Download the latest lambda.zip from the s3 URL.
2. Go to Amazon console and search for CloudFormation.
3. On the CloudFormation page, select Stacks > lm-forwarder.
4. On the Resources tab, select the lambda forwarder.
5. On the Lambda Function page, select the Code tab, and then select Upload from > .zip file.
6. Upload the latest lambda.zip file downloaded earlier, refer to step 1.
7. Go to the Runtime settings section, select Edit.
8. In the Runtime field, select Custom runtime on Amazon Linux 2.
9. In the Handler field, type bootstrap.
10. Select Save.