Syslog Log Ingestion using the LM Collector

You can ingest syslog logs by modifying the LM Collector config file. This legacy method enables the following:

• Date and timestamp parsing—Interpret date and timestamps from incoming syslog messages, regardless of their format.
• Configure resource mapping—Map incoming syslog logs to specific categories based on predefined rules or configurations.
• Configure filters—Process or route logs to specific destinations based on predefined conditions before forwarding them.

Filters for Sensitive Log Information 

Filters can also be used to reduce the volume of non-essential syslog log messages that are sent to the logs ingestion API queue.

The following table lists the filters you can edit in the agent.conf file:

Filter Name	Description	Examples
severity	Define filters for syslog messages with a severity level equal to or more urgent than the specified value. Severity levels can be (in descending order of urgency): Emergency, Critical, Alert, Error, Warning, Notify, Informational, Debug.	logsource.syslog.filter.1.severity.equal=informational logsource.syslog.filter.2.severity.moreurgentthan=informational
facility	Define filters for syslog messages based on the facility value: kernel messages, user-level messages, clock daemon, and so on.	logsource.syslog.filter.11.facility.equal=kernel messages logsource.syslog.filter.12.facility.notequal=clock daemon logsource.syslog.filter.13.facility.contain=NTP logsource.syslog.filter.14.facility.notcontain=NTP logsource.syslog.filter.15.facility.exist=* logsource.syslog.filter.16.facility.notexist=* logsource.syslog.filter.17.facility.greaterthan=NTP logsource.syslog.filter.18.facility.greaterequal=NTP logsource.syslog.filter.19.facility.lessthan=NTP logsource.syslog.filter.20.facility.lessequal=NTP logsource.syslog.filter.21.facility.regexmatch=Target[\S\s]*-[Ww][Aa] logsource.syslog.filter.22.facility.regexnotmatch=Target[\S\s]*kernel
message	Define filters for syslog messages based on the contents of the message itself, using keywords or a regular expression pattern to match.	logsource.syslog.filter.3.message.equal=negotiate IPsec phase 1 logsource.syslog.filter.4.message.notequal=negotiate IPsec phase 1 logsource.syslog.filter.5.message.contain=negotiate IPsec phase 1 logsource.syslog.filter.6.message.notcontain=negotiate IPsec phase 1 logsource.syslog.filter.7.message.regexmatch=(negotiate)+\w logsource.syslog.filter.8.message.regexnotmatch=(negotiate)+\w logsource.syslog.filter.9.message.exist=* logsource.syslog.filter.10.message.notexist=*
application	Define filters for syslog messages based on the application, using keywords or a regular expression pattern to match.	logsource.syslog.filter.23.application.equal=snmpd logsource.syslog.filter.24.application.notequal=snmpd logsource.syslog.filter.25.application.contain=snmpd logsource.syslog.filter.26.application.notcontain=MySQL logsource.syslog.filter.27.application.regexmatch=Target[\S\s]*-[Ww][Aa] logsource.syslog.filter.28.application.regexnotmatch=Target[\S\s]*admin logsource.syslog.filter.29.application.exist=* logsource.syslog.filter.30.application.notexist=*

Resource Mapping for Manual Syslog Log Ingestion

Once the Collector receives syslog logs, they are mapped to existing monitored resources before they are forwarded to the logs ingestion API. If the log cannot be mapped to an LM-monitored resource, you can still view and and search the log by “_resource.id=0”.

Note: One key advantage of using LogSource for syslog log ingestion is its streamlined resource mapping, which simplifies the log classification process. For more information see Syslog LogSource Configuration.

The following table lists the properties you can edit for device mapping in the agent.conf file for syslog logs sent from the Collector:

Property	Description	Default
lmlogs.syslog.property.name	Resource properties to use for device mapping.	system.hostname
lmlogs.syslog.hostname.format	The expected format of the resource property specified in lmlogs.syslog.property.name. Possible values are IP, FQDN, HOSTNAME. If set to IP, HOSTNAME, or FQDN, the syslog message is resolved by making a DNS query.	IP
lmlogs.syslog.UseHostNameFromMessages	Property to use the hostname from messages.	false
lmlogs.syslog.UseSocketAddressIfhostNameisInvalid	If the hostname is not valid, you can use the socket address for device mapping.	true
lmlogs.logsource.syslog.unmapped.resource	If resource mapping fails, you can use a backup resource to associate logs to. One backup resource can be defined for each Collector, and all logs that fail to map to its resource for any reason, will be mapped to this backup resource. When configuring a global backup resource, you can choose any resource property such as system.displayname followed by a colon and the value of the LogicMonitor property. For example, to configure “collector_syslog” as the backup resource: lmlogs.logsource.syslog.unmapped.resource=system.displayname:collector_syslog	N/A

If the value of system.hostname for your monitored resources does not match their IPs reported in the Syslog events, then these default settings will not work and device mapping will fail. In this case, you will need to change the lmlogs.syslog.property.name to a property which matches the value of the lmlogs.syslog.hostname.format setting.

If you do not have an existing LogicMonitor property matching any of these values, you can create a PropertySource to add the attribute across your environment or set the property manually. For details, see Creating PropertySources.

Requirements for Manually Ingesting Syslog Logs

To manually configure the LM Collector to ingest syslog logs, you need the following:

• LM Collector — Syslog log ingestion requires a Collector installation on your machine.
• Port 514 (UDP) — Must be open and accessible on both the machine hosting the Collector and the firewall. This may require disabling UDP for rsyslog.

Manually Configuring the LM Collector to Ingest Syslog Logs

If you installed a Collector that has been upgraded from v35.200 or earlier and never collected logs, the agent.conf file must be edited to enable LM Logs and the Collector to ingest syslogs.

1. In LogicMonitor, navigate to Settings > Collectors.
2. Select the Collectors tab, select the Collector Group you want to configure.  
3. Select  next to the Collector and then select Collector Configuration. On the Collector Configuration page, the settings under the Agent Config tab are displayed.
4. Uncomment the following properties and set the value to true:

   # Enable LM Logs
   lmlogs.syslog.enabled=true
   
   # Enable the Collector to ingest syslogs
   eventcollector.syslog.enable=true

   Copy
   If the property does not exist, you can add it.
5. If you want to add a filter or property that does not currently exist in the agent.conf file, add parameters from the filters and resource mapping tables above. 
6. Select Save and Restart to restart the collector and apply the changes.