LM Cloud

AWS Billing Monitoring - Cost & Usage Report

LogicMonitor currently supports monitoring of billing data via AWS Cost and Usage Reports

What data is monitored?

LogicMonitor can monitor the cost of any line items in your Cost & Usage Report. We have five out-of-the-box DataSources (listed below), but custom DataSources can be used to extend what is monitored. Pre-configured DataSources include:

  1. Cost by Account (AWS_Billing_CostByAccount) - automatically monitors cost by Account Ids listed in the Cost & Usage Report
  2. Cost by Region (AWS_Billing_CostByRegion) - automatically monitors cost by Region per the Cost & Usage Report
  3. Cost by Service (AWS_Billing_CostByService) - automatically monitors cost by AWS Products listed in the Cost & Usage Report
  4. Cost by Operation (AWS_Billing_CostByOperation) - automatically monitors cost by operations listed in the Cost & Usage Report
  5. Cost by Tag (AWS_Billing_CostByTag) - monitors cost by cost allocation tags added as instances

How do I get started?

To get started, you’ll need to perform the following steps:

    1. Create and configure a Cost & Usage Report to be stored in an S3 bucket

From your AWS console, navigate to your username in the upper right hand corner and select ‘My Billing Dashboard’.  Under the Reports section, create a report with the following configuration:

Make sure you enable support for Redshift - this does not actually require that you run any Redshift instances, but will include a script with the saved billing file that LogicMonitor uses to format and process the data.


The resulting configuration should look like this:



You’ll also need to configure an S3 bucket on the next page, and specify a report path.  Remember the path and bucket specified, as you’ll need to provide them in LogicMonitor later.  Note that the S3 bucket you specify will need to include permission for the Cost & Usage Reports to be saved.  You can use the sample policy provided by AWS (confirm bucket name) and the policy can be added to your bucket on the Permissions tab for the bucket.

    1. Provide LogicMonitor with permissions sufficient for accessing the billing files in the S3 bucket

Your LogicMonitor role needs to have sufficient permission for accessing the reports configured in step 1.  This is as simple as attaching the following inline policy to the role:

{
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:GetObjectVersion",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::S3BUCKETNAME",
"arn:aws:s3:::S3BUCKETNAME/*"
]
}
],
"Version": "2012-10-17"
}

Where S3BUCKETNAME should be replaced with the name of the bucket you specified in step 1.

      1. Enable the ‘Billing (Cost & Usage Report)’ service in LogicMonitor

From the manage dialog for your AWS account in LogicMonitor, on the billing tab, enable billing monitoring.  You’ll need to provide the bucket name and path you specified in step 1:


        1. Import the new AWS Billing DataSources

    Once these steps are complete, you should see Cost by service, region, account and operation DataSources appear under the account level device, like this:


    Products, regions, account ids, and operations will be discovered dynamically as instances via custom Active Discovery methods.

    The Cost by tag DataSource is multi-instance, but does not have Active Discovery (AD) enabled by default.  As such, you have two options: (1) manually add instances that correspond to the tags you’d like to track cost for and (2) enable AD such that tag values are auto-discovered and cost is tracked for each.  The only requirement for monitoring cost associated with tags is that you have the tags enabled as cost allocation tags included your Cost & Usage report.

    You can enable AD and auto-discovery of tag values in the DataSource definition, like this:

    We suggest that you start from the Cost By Tag DataSource, clone it, and edit the AD section. In the AWS Billing Report Attribute field, you should enter your tag key prefixed with 'user:' (AWS prepends this in the Cost & Usage report as it's a user defined tag). All values for the specified key will be discovered, and cost will be monitored for each. So you'll need a different DataSource for each tag key you want to track costs for.

    If you prefer to manually add instances, you can do at the account level device like this:

    DataSource - should be the ‘Cost by Tag’ DataSource

    Name - should be the Tag value you’d like to monitor cost for

    Wildvalue - should be the name of the Tag as it appears in the Cost & Usage report, which will usually be preceded by ‘user:’ to denote a user specified tag

    Once complete, data should populate for your new instance within a few minutes.

    Create a Billing Dashboard

    Once you've configured LogicMonitor's AWS Billing monitoring and data collection has begun, you can visualize this data in a dashboard. Download this JSON template and import it into you account, or create your own custom Dashboard.