Setting up Logstash Logs Ingestion

This output plugin sends Logstash events to the LogicMonitor logs ingestion API. You can also install the Logstash Monitoring LogicModules for added visibility into your Logstash metrics alongside the logs.

Prerequisites

LogicMonitor account name.
LogicMonitor API access tokens.

Installation

The Logstash plugin can be installed through Ruby Gems. Run the following command on your Logstash instance:

logstash-plugin install logstash-output-lmlogs

Configuration

The following is an example of the minimum configuration needed for the Logstash plugin. You can append more settings into the configuration file. See the parameters tables below.

output {
  lmlogs {
    access_id => "access_id"
    access_key => "access_key"
    portal_name => "account-name"
    property_key => "hostname"
    lm_property => "system.sysname"
  }
}

Required Parameters

Name	Description	Default
access_id	Username to use for HTTP authentication.	N/A
access_key	Password to use for HTTP authentication.	N/A
portal_name	The LogicMonitor portal account name.	N/A

Optional Parameters

Name	Description	Default
batch_size	The number of events to send to LM Logs at one time. Increasing the batch size can increase throughput by reducing HTTP overhead.	100
keep_timestamp	If false, LM Logs will use the ingestion timestamp as the even timestamp.	true
lm_property	Specify the key that will be used by LogicMonitor to match a resource based on property.	system.hostname
message_key	The key in the Logstash event that will be used as the logs message.	message
property_key	They key in Logstash to find the hostname value, that will be used map to lm_property.	hostname
timestamp_is_key	If true, LM Logs will use a specified key as the event timestamp value.	false
timestamp_key	If timestamp_is_key=true, LM Logs will use this key in the event as the timestamp. Valid timestamp formats are ISO8601 strings or epoch in seconds, milliseconds, and nanoseconds.	logtimestamp

Note: The syntax for message_key and property_key values are available in the Logstash Event API Documentation.

Development

The Logstash plugin can be built in Docker by running the following command:

docker-compose run jruby gem build logstash-output-lmlogs.gemspec

Troubleshooting

If you are not seeing logs in LM Logs:

Ensure that the device from which the logs are expected is being monitored.
If the device exists, check that the lm_property used for mapping the device is unique. Log ingestion will not work if lm_property is used for more than one device.

Product Documentation