Forrester Total Economic Impact™ study finds Edwin AI delivered a 313% ROI for composite organization.

Read more

The LogFiles LogSource enables LM Logs to collect logs that are written to files on disk by using a deployed and registered LogicMonitor OpenTelemetry (OTel) Collector. After the Collector is configured, you can create a LogFiles LogSource and apply it to the monitored resource that generates the log files.
For more information on configuring the OTel Collector, see OpenTelemetry Installation.

Note: Starting with OpenTelemetry Collector version 5.2.0.0, LogicMonitor uses improved log-to-resource mapping to associate logs with the correct cloud resource instead of defaulting them to the Collector host. If logs are not mapped to a resource, they are classified as “deviceless”. To manually associate logs with a resource, set the LM_DEVICE_ATTRIBUTES environment variable using key-value pairs (for example, LM_DEVICE_ATTRIBUTES="key1=value1".)

Requirements for Configuring Log Files LogSource

To configure Log Files LogSources, you must have the following:

Note: This LogSource requires a LogicMonitor OpenTelemetry (LM OTEL) Collector to be installed, registered in LogicMonitor, and in a Running state. Log collection does not occur unless an active LM OTEL Collector is available to collect and forward the log data.

Configuration Options

The following describes configuration details specific to the Log Files type of LogSource. For more information on how to add a LogSource, see Configuring a LogSource.

Basic Information

Provide the path name for the log file in the Log File Path field.

The following image displays an example of single log file path configured for ingestion.

single log file path

The following image displays an example of log file path configured with wildcards to capture multiple log files and rollover patterns across directories.

Multipath log file

AppliesTo

The AppliesTo Filter (ATF) determines which log data this LogSource applies to.

For Log Files collected through OpenTelemetry, the AppliesTo expression must reference the OpenTelemetry Collector identifier. If the AppliesTo expression does not match the OpenTelemetry Collector name, the LogSource does not process incoming logs.

Note: Configure the AppliesTo expression to match the LM OpenTelemetry (OTel) Collector resource, not the monitored resource that generates the log files. If the AppliesTo expression matches the monitored resource instead of the LM OpenTelemetry Collector resource, the LogSource will not collect logs.

Filtering Using AND or OR Operator

After the AppliesTo Filter matches, the system evaluates include and exclude filters to determine whether a log is ingested or dropped. You can toggle the AND or OR switch to control how this evaluation occurs. By default, these switches are disabled.

Include Filters

You can add include filters to specify which resources are eligible for log ingestion, such as a specific resource type or application. Logs that match the include filter criteria are forwarded to the log ingestion process.

Exclude Filters

You can add exclude filters to prevent logs from specific resources, resource types, or attributes from being ingested. Logs that match an exclude filter are dropped and are not forwarded to the log ingestion process.

Available parameters

AttributesComparison operatorValue example
MessageRegexMatch, RegexNotMatch.Regular expression

Log Fields

You can configure Log Fields (tags) to send additional metadata with the logs.

Available parameters

MethodKey exampleValue exampleDescription
Static“Customer”“Customer_XYZ”
Dynamic(REGEX)“Host”“host=*”The query will run on the message field.
LM Property(Token)“Device”“##system.deviceId##”

Note: You can use the optional resource_type property to define the resource type applied to all ingested logs.

The resource type value that you specify is used for all logs. If the value is not specified, the system assigns a resource type based on the value of the predef.externalResourceType field in the resource settings.

Sensitive Data Masking

Sensitive data masking enables you to mask sensitive information before logs are ingested into LM Logs. This prevents exposure of sensitive data, such as personally identifiable information (PII), while preserving log usability for analysis and alerting.
For more information on Default Masking Rules, see Default Sensitive Data Masking Rules.

Configuring a Log Files LogSource

Configuring Log Files LogSource

  1. In LogicMonitor, navigate to more options MoreModules.
  2. From My Module Toolbox, select Add Datasource Add.
  3. In the Add window, select LogSource. The Add New LogSource window displays.
  4. In the Info section, do the following:
    1. In the Name field, enter a descriptive name. This displays in the list of LogSources.
    2. In the Description field, provide additional context to explain the purpose, function, or configuration details of the log source.
    3. In the Tags field, search for tags you want to assign to your log source. 
      Note: Tags are metadata fields used for filtering, grouping, and automation.
    4. In the Group field, select one or more groups that best categorize the log source. Groups typically reflect the function, location, or environment of the log origin. 
    5. Use the Technical Notes field to provide detailed reference information that may assist with troubleshooting or maintenance.
    6. Toggle the Show preview of rendered technical notes switch to view how markdown-formatted content display when rendered.
    7. Select “LM Logs: Log Files“ in the Type dropdown menu.
    8. Provide the path name for the log file in the Log File Path field
      Add New Logsource Info section
  5. Assign Access Groups to enable who can view, manage, and interact with the logs.
    For more information, see Access Groups.
  6. In the AppliesTo field, enter an AppliesTo expression that matches the OpenTelemetry Collector name deployed on the resource.
    For more information, see AppliesTo Function Scripting Overview.
  7. In the Filters section, do the following:
    1. Select AND or OR operator based on the filtering logic you want to apply.
    2. Select Add icon blue Add Filters.
    3. Select “Message” from the Attribute dropdown.
    4. Select Filter Type, for example “RegexMatch” or “RegexNotMatch”.
    5. Select a Comparison Operator, for example “RegexMatch” or “RegexNotMatch”.
    6. In the Value field, enter the pattern or text to match. This value works with the selected attribute and operator to filter log entries.
    7. Add an optional Comment.
    8. Select save Save. The filter is added.
      Add Filters Modal for LogFiles LogSource
  8. To configure Log Fields (tags) to send additional metadata with the logs, do the following:
    1. Select add Add Log Fields.
    2. Select an option from the Method dropdown menu for collecting the metadata.
    3. In the Key field, enter the log field name. This identifies the metadata attribute used for filtering and querying logs.
    4. In the Value field, enter the value to associate with the log field. This defines the data captured and used for grouping or analysis.
    5. Add an optional Comment to document the purpose of the log field.
    6. Select save Save. The log field is added.
      Add Log Fields Modal for LogFiles LogSource
  9. In the Sensitive Data Masking section, you can either select Load Default Data Masking Rules to use the predefined masking rules provided by LogicMonitor or create custom masking rules to define how sensitive data is detected and masked.
    To create a custom masking rule, do the following:
    1. Select Add icon blue Add Sensitive Data Masking Rules.
    2. Select an option from the Type dropdown menu. For example, Regex.
    3. In the Key field, enter the masking rule name. This identifies the type of sensitive data being masked.
    4. In the Masking Rule field, define the pattern used to detect sensitive data.
    5. In the Masked Value field, enter the replacement value. This value displays instead of the sensitive data.
    6. (Optional) Add additional details in the Description field.
      Add Masking Rule modal for LogFiles LogSource
    7. Select save Save. The masking rule is added.
      Note:
      • To edit an existing masking rule, select the masking rule and update the required fields.
      • To disable a sensitive data masking rule, delete the rule.
  10. Select save Save.
    The LogSource is configured and available for use.

14-day access to the full LogicMonitor platform