Datasources

Complex Datapoints

Complex datapoints are a means of post-processing the data returned by monitored systems. This are useful when the  data presented by the target device isn't in suitable in it's raw form. For example, on a storage system you typically want to track and alert on percent utilization, which can be calculated based on total capacity and used capacity.

Since the values of complex datapoints are stored in their calculated forms, they can be used in graphs, reports, dashboards and other places in your account.

Complex Datapoint Types

There are two methods for calculating complex datapoints:

Standard Complex Datapoints: use our simple expression syntax to calculate a new value based on existing Normal Datapoints. With this option, you can reference other normal datapoints as well as device properties. For example:

 100*CpuRawIdle/(CpuRawSystem+CpuRawIdle+CpuRawWait+CpuRawUser)  


Groovy Complex Datapoints:
employ a Groovy script to process the raw collected data. In a Groovy Complex Datapoint you have access to the raw data payload, but not the actual datapoints (e.g. those calculated as a "counter" or "derive"). Also, Groovy allows for calculations based on both device and instance properties. Typically one would use a Groovy complex datapoint for manipulating data in a way that cannot be achieved with a standard complex datapoint.

See our details on Groovy Complex Datapoints for more details and examples.

Standard Complex Datapoint Syntax

Using Normal Datapoints

Standard Complex Datapoints are built with arithmetic expressions and other operators to transform normal datapoints.

For example the SNMP interface MIB provides an OID that reports the inbound octets on an interface (InOctets). Operations Engineers tend to think in terms of Mbps rather than Octets, so we convert this value to InMbps as follows:

InOctets*8/1000/1000

 

Using Device Properties

One can also refer to a devices properties in a Standard Complex Datapoint expression by enclosing the property name in double hash marks.

For example, say you want to calculate throughput based on a bandwidth limit set for a particular device. You'd set a device property named "manualbandwidth", which you could then use in a datapoint calculation as follows:

InOctets*8/##MANUALBANDWIDTH##

In this use-case, care must be taken to make sure the custom property is defined on all the devices to which the datasource applies. If the property does not exist on a device it will be replaced with an empty string, rendering the above expression an error.

To prevent this issue you can create a complex datapoint that retrieves the device property, and a second complex datapoint to test for whether or not it the first has a valid value, as in the below:

 

See our documentation on Datapoint Expressions for more details on building standard complex datapoints.