Rules are excellent when you know precisely what you want to match, typically based on experience. Yet rules only let you observe what you have learned to look for. This is where artificial intelligence (AI) and machine learning (ML) contribute significantly to observability—detecting errors and early warning signs that were previously unobservable. LM Envision supports metric and log anomaly detection.
This blog discusses how LM Envision Log Anomalies uncovers previously unknown anomalies. Customers can choose to be alerted about these anomalies. Regardless, the anomaly information is present for those conducting triage and troubleshooting.
Solving a Rubik’s cube with a blindfold
In monitoring and observability, “finding needles in a haystack” is often used. However, the situation is worse than that. This term misleads by suggesting IT professionals are searching for something familiar. The problem is more like finding unknown needles in a haystack or solving a Rubik’s cube with a blindfold.
What operation and engineering teams need to learn about is not yet known. These are the unknown, unknowns that, once illuminated, can become known, knowns, and subsequently identified with rules. IT teams need tools that remove the blindfold.
Removing the blindfold
Anomaly detection is a generic AI/ML term for detecting abnormal patterns. IT teams can perform pattern detections on metrics, logs, and other data sources.
One area of abnormal pattern detection that customers have been interested in is detecting previously unseen logs. These logs often precede outages and are traditionally only discovered after an outage investigation. IT professionals can detect these logs through LM Envision’s log anomaly detection before an outage occurs. Previously unseen logs can also uncover security violations, unauthorized configuration changes, and more.
Customers can choose to receive alerts for detected log anomalies. Additionally, anomalies will always be available for troubleshooting at a later time.
How log anomaly detection works
LM Envision uses Natural Language Processing (NLP) technology, to learn the structure of ingested log messages. Logs can be ingested directly by LM Envision or forwarded from other Log tools.
LM Envision tokenizes logs into parts that are common and variable. This allows similar logs to be merged into a single profile, dramatically reducing analysis compute resources and noise.
Anomalies trigger alerts only when customers create log pipelines or filters with specified severity levels Enabling log anomaly detection gives LM Envision customers control over additional alert noise.
When using LM Envision’s log visualization, the user only needs to select the “Anomaly” button to see any anomalies within the viewing period.
AI-powered anomaly detection for proactive IT management
IT teams need help searching for or finding what they do not know what to look for. LM Envision’s AI-powered layered iIntelligence utilizes anomaly detection to detect unknown unknowns. This includes uncovering previously unseen logs that may indicate a future outage or security violation.
