About LogSources

LogSources is a LogicModule that provide templates to help you enable logs and configure log data collection and forwarding. LogSources contain details about what logs to get and where to get them, and which fields should be considered for parsing, for example timestamp, resource, and message. LogSources is available for Syslog, Windows Events, Kubernetes Events, and other common sources of log data.

LogSources Data Collection

LogSources for Syslog, Windows Events, and Kubernetes sources use the LM Collector to collect and forward log data. For information about required collector versions, see Creating LogSources.

The Log Files LogSources type forwards traces from your instrumented applications to the LogicMonitor platform. This logsource type uses the LM OTEL Collector to collect and forward log data.

For an overview of available methods for collecting and forwarding log data, see About Log Ingestion.

Supported Logsource Types

LogSources are available for the following types of log data sources:

• Syslog
• Windows Event Logging
• Kubernetes Event Logging
• Kubernetes Pods
• Log Files
• API Script

Configuration Properties

LogSources have the same set of configuration properties as in the LM Logs Collector configuration with system properties to map log ingestion, filters, and so on. 

Instead of a single log collection configuration per collector, LogSources allows for multi-log collection configurations on a single collector.

Processing Pipeline

The LogSources data collection processing pipeline consists of the following steps:

1. Filtering to include or exclude data to reduce the log volume.
2. Resource mapping and data enrichment (for some logsource types).

Logs are filtered on the collector side using critera based on standard comparison operators like “Equal”, “Contain”, and “RegexMatch”. Available operators vary depending on the logsource type.

The incoming log is parsed to populate and map the resource information. This can be for example a timestamp, resource details, or some extra tags which can be used later for searching. Standard regular expression is used to get this information.

LogSource Types and Configuration Options

Similar to collector attributes, LogSources have additional attributes that you can configure to enrich the collected data and add tags for the logs. Examples of configurable attributes are applies to, log file path, timestamp, mapping from log, the LM property to match to, and other configuration items added through agent.conf for the collector.

You can also add the following attributes:

• Static attributes, for example “cust.name = DataCenter”.
• Token-based attributes, for example “cust.name = ##some.properties##”.

LogSources configuration in the LogicMonitor portal:

• Applies To: The resources to which the logsource is applied. 
• Type: Select a supported logsource type.
• Group: The LogSource group (optional).
• Log Attributes: Varies based on logsource type.
• Filters: Options to include/exclude sources.
• Log Fields/Tags: Include custom metadata to be sent with the logs, either dynamic value parsed from the log, or a static value.
• Resource Mapping: The resource the logs should map to and mapping method.
• Other Options: For example alternative timestamp to use.

Syslog

For a Syslog logsource type, you can define:

• Include filters
• Log fields/tags
• Resource mapping
• Use received time stamp instead of log timestamp.

Windows Event Logging

For a Windows Event logsource type, you can define:

• Exclude/include filters
• Log fields/tags
• Resource mapping

Kubernetes Event Logging

For a Kubernetes Event logsource type, you can define:

• Include filters
• Log fields/tags

Kubernetes Pods

For a Kubernetes Pods logsource type, you can define:

• Include filters
• Log fields/tags

Log Files

For a LM Logs logsource type, you can define:

• Include filters
• Log fields/tags

API Script

For an API Script logsource type, you can define:

• Attributes
• Include filters
• Log fields/tags
• Resource mapping