Forrester Total Economic Impact™ study finds Edwin AI delivered a 313% ROI for composite organization.

Read more

The LogFiles LogSource enables LM Logs to collect logs that are written to files on disk by a deployed and registered LogicMonitor OpenTelemetry (OTel) Collector. After the Collector is configured, you can create a LogFiles LogSource and apply it to the monitored resource that generates the log files.
For more information on configuring the OTel Collector, see OpenTelemetry Installation.

Note: Starting with OpenTelemetry Collector version 5.2.0.0, LogicMonitor uses improved log-to-resource mapping to associate logs with the correct cloud resource instead of defaulting them to the LM OTel Collector Name. If logs are not mapped to a resource, they are classified as “deviceless”. This occurs because either the host is not monitored in LogicMonitor or it is not a cloud based resource.

To manually associate logs with a resource, set the LM_DEVICE_ATTRIBUTES environment variable using key-value pairs (for example, LM_DEVICE_ATTRIBUTES="key1=value1".)

Requirements for Configuring Log Files LogSource

To configure Log Files LogSources, you must have the following:

Note: This LogSource requires a LogicMonitor OpenTelemetry (LM OTEL) Collector to be installed, registered in LogicMonitor, and in a Running state. Log collection does not occur unless an active LM OTEL Collector is available to collect and forward the log data.

AppliesTo in Log Files LogSource

The AppliesTo Filter (ATF) determines which log data this LogSource applies to.

For Log Files collected through OpenTelemetry, the AppliesTo expression must reference the OpenTelemetry Collector identifier. If the AppliesTo expression does not match the OpenTelemetry Collector name, the LogSource does not process incoming logs.

AppliesTo example for LogFiles LogSource

Important: Configure the AppliesTo expression to match the LM OpenTelemetry (OTel) Collector resource, not the monitored resource that generates the log files. If the AppliesTo expression matches the monitored resource instead of the LM OpenTelemetry Collector resource, the LogSource will not collect logs.

Filtering in Log Files LogSource

After the AppliesTo Filter matches, the system evaluates include and exclude filters to determine whether a log is ingested or dropped. You can toggle the AND or OR switch to control how this evaluation occurs. By default, these switches are disabled.

The following table lists the available parameters when configuring LogFiles LogSource:

AttributesComparison operatorValue example
MessageRegexMatch, RegexNotMatch.Regular expression

Log Fields in Log Files LogSource

You can configure Log Fields (tags) to send additional metadata with the logs.

The following table lists the available parameters:

MethodKey exampleValue exampleDescription
Static“Customer”“Customer_XYZ”Sends a fixed key-value pair with every log.
Dynamic(REGEX)“Host”“host=*”Extracts the value from the log message using a regular expression.
LM Property(Token)“Device”“##system.deviceId##”Uses a LogicMonitor property token to populate the value at runtime.
Dynamic Group Regexuser,host,statususer=(\w+)\s+host=([\w.]+)\s+status=(\d+)Enables extraction of multiple metadata fields from a single log line using one regular expression with multiple capture groups

Note: You can use the optional resource_type property to define the resource type applied to all ingested logs.

The resource type value that you specify is used for all logs. If the value is not specified, the system assigns a resource type based on the value of the predef.externalResourceType field in the resource settings.

Sensitive Data Masking in Log Files LogSource

Sensitive data masking enables you to mask sensitive information before logs are ingested into LM Logs. This prevents exposure of sensitive data, such as personally identifiable information (PII), while preserving log usability for analysis and alerting. LogicMonitor provides pre-configured masking rules that can be edited or deleted and you may also configure your own custom rules.
For more information, see Default Sensitive Data Masking Rules.

Configuring a Log Files LogSource

Configuring Log Files LogSource

  1. In LogicMonitor, navigate to more options MoreModules > select Add Datasource Add.
  2. In the Add window, select LogSource. The Add New LogSource window displays.
  3. Configure the Basic Information for Info section with required information.
    For more information, see Configuring Basic Information for LogSource
  4. In the Summary for Info section, do the following:
    1. Select “LM Logs: Log Files“ in the Type dropdown menu.
    2. Provide the path name for the log file in the Log File Path field.
      You can enter one of the following:
      • A single log file path.
        For example:\\share\logfile.log
      • A log file path that uses wildcards to collect multiple log files or rollover log files across directories.
        For example:C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log
        Add New Logsource Info section
  5. Assign Access Groups to enable who can view, manage, and interact with the logs.
    For more information, see Access Groups.
  6. In the AppliesTo field, enter an AppliesTo expression that matches the OpenTelemetry Collector name deployed on the resource.
    For more information, see AppliesTo Function Scripting Overview.
  7. Configure the fields in the Filters section.
    For more information for configuring Filters for LogSource, see Configuring Filters for LogSource.
  8. Configure Log Fields (tags) to send additional metadata with the logs.
    For more information for configuring Log Fields in LogSource, see Configuring Log Fields in LogSource .
  9. In the Sensitive Data Masking section, you can either select Load Default Data Masking Rules to use the predefined masking rules provided by LogicMonitor or create custom masking rules to define how sensitive data is detected and masked.
    To create a custom masking rule, do the following:
    1. Select Add icon blue Add Sensitive Data Masking Rules.
    2. Select an option from the Type dropdown menu. For example, Regex.
    3. In the Key field, enter the masking rule name. This identifies the type of sensitive data being masked.
    4. In the Masking Rule field, define the pattern used to detect sensitive data.
    5. In the Masked Value field, enter the replacement value. This value displays instead of the sensitive data.
    6. (Optional) Add additional details in the Description field.
      Add Masking Rule modal for LogFiles LogSource
    7. Select save Save. The masking rule is added.
      Note:
      • To edit an existing masking rule, select the masking rule and update the required fields.
      • To disable a sensitive data masking rule, delete the rule.
  10. Select save Save.
    The LogSource is configured and available for use.