Network Traffic Flow Monitoring (New UI)

Overview

Network traffic flow (NetFlow) monitoring collects IP network traffic as it enters or exits an interface. LogicMonitor Collectors receive and analyze data from resources that support common flow export protocols. LogicMonitor can report on the following statistics:

  • Top talkers
  • Top source/destination endpoints
  • Top flows
  • Top ports
  • Top applications
  • Quality of service (QoS)

The following network flow export protocols are supported:

  • NetFlow versions 5, 7, and 9
  • Flexible NetFlow 
  • IPFIX
  • sFlow versions 1, 3, and 5.
    Note: sFlow version 5 requires a LogicMonitor Collector version 29.105 or higher.
  • JFlow version 5
  • NBAR2
    Note: The ability to collect NBAR2 data is limited to LogicMonitor Enterprise accounts and requires a LogicMonitor Collector version 29.101 or higher. If you intend to collect NBAR2 data, you must set the netflow.nbar.enable property on the LogicMonitor Collector to TRUE. For more information, see the “Configuring the LogicMonitor Collector for Network Traffic Flow Monitoring” section of this article.

Best Practices

  • Ensure your Collector has the capacity to monitor network traffic flows. For more information, see Collector Capacity
  • Minimize network hops between the LogicMonitor Collector and the resource. Network flow records are sent using the UDP communications protocol. Because UDP delivery is not guaranteed, it is recommended that you ensure the LogicMonitor Collector has the fewest network hops possible to the resource in order to minimize potential flow disruption due to network congestion or complexity. 
  • Synchronize clocks between the transmitting resource and the resource that is hosting the LogicMonitor Collector. If resources are located in different time zones, it is recommended that you use UTC or standardize on a single time zone. 
  • Eliminate port conflict. The host that is collecting network traffic data must not have any other network traffic analyzer listening on the same port. This can potentially cause contention and prevent traffic data from displaying in LogicMonitor.

Configuring the LogicMonitor Collector for Network Traffic Flow Monitoring

By default, Collectors install with standard network traffic flow monitoring settings that do not require modification in most cases. However, you can override default settings to meet the unique needs of your monitoring environment.

Name Type Default Details
netflow.enable Boolean TRUE If TRUE, the network flow module is enabled on Collector.
netflow.ports Integer 2055 The UDP listening port for network flow protocol data. The UDP port on the resource that is sending the flow data must match the UDP port specified here. Multiple ports can be configured here if you need to support multiple protocols on multiple ports (for example, netflow.ports=2055,4739).
netflow.sflow.ports Integer 6343 The UDP listening port for sFlow protocol data.
netflow.datadir String netflow The path of the HSQL database.
netflow.datadir.maxSizeInMB Integer 10240 The maximum size (in megabytes) of the network flow data directory.
netflow.log.maxNumPerMinute Integer 5 The maximum log count allowed to be written during one minute of network flow monitoring.
netflow.netflow9.templateLife Integer 720 The expiration time (in hours) of NetFlow version 9 template.
netflow.topFlowSamples Integer 1000 The maximum sample number of top flows. Allowed range is from 100 to 2000.
netflow.ignoreTimestampValidate Boolean FALSE If TRUE, the Collector ignores network flow resource time information. Currently, the only known resources that necessitate overriding the default FALSE value are SonicWalls.
netflow.nbar.enable Boolean FALSE If TRUE, the Collector begins parsing the applicationID and ApplicationType. LogicMonitor Enterprise and Collector version 29.101 or higher are required.
netflow.ipv6.enabled Boolean TRUE If FALSE, the Collector will ignore flows from with IPv6 addresses
netflow.log.largeBytesOrPackets Integer 1073741824 Logs flows in Audit Logs with packets or bytes larger than the integer specified

Enabling Network Traffic Monitoring in LogicMonitor

Network traffic monitoring is enabled on a per-resource basis.

To enable network traffic monitoring, do the following:

  1. Navigate to the Resources page and locate the resource you want to enable network traffic monitoring for. 
  2. With the resource selected, click the Manage button.
  3. From the Manage page, toggle the Enable Network Flow Analysis switch.
  4. Select the Collector that will be used to receive exported network flow data. Network flow collection duties cannot be assigned to an Auto-Balanced Collector Group.
  5. Click Save.

Note: If your network flow exporter is sending data from an IP address that is not the same as the monitored IP address of the resource, customize the netflow.allowips property on the resource with the IP addresses from which network flow originates. This property accepts either a single IP or a comma-separated list as its value. For more information, see Resource and Instance Properties.

Enabling Network Traffic Flow Monitoring on a Resource

In addition to enabling network traffic monitoring in LogicMonitor, you must also enable it on your resource. Configurations vary depending on the resource, vendor, network topology, and protocol you are using. It is recommended that you review manufacturer guidelines for your specific resources.

The following resource configurations are applicable to all protocols:

  • Network flow monitoring must be enabled per interface.
  • A version number must be specified.
  • A source interface for the flow exporter must be specified.
  • The UDP port configured for the exporter must match the port specified in the Collector’s agent.conf file. For more information, see the “Configuring the LogicMonitor Collector for Network Traffic Flow Monitoring” section of this article.
  • The clock on the resources should be synchronized with the clock on the Collector host.
  • The IP address of the destination (the LogicMonitor Collector) must be specified.
  • (NetFlow version 9 only) Additional template configuration options must be set.
  • (sFlow only) Packet data must be provided in the enterprise=0 and format=1 packet configuration as described in RFC2233. In addition, sFlow uses port 6343.
  • (NBAR2) The option application-table and option application-attributes must be enabled on the exporter configuration of the resource. For more information, see Cisco’s NBAR Configuration Guide.

Sample Configurations

The following are sample NetFlow version 9 resource configurations. Because these sample configurations have the potential to become outdated as Cisco makes updates, refer to Cisco’s NetFlow Configuration and Flexible NetFlow Configuration guides to ensure up-to-date information.

Cisco IOS 3745 router – NetFlow Version 9, Main Cache Export

Configure global settings: source interface, NetFlow version, target NetFlow Collector, and UDP port.

To begin, enter the following at the command line:

Router#conf t

Then, enter the configurations for the global settings:

Router(config)#ip flow-export source FastEthernet0/0
Router(config)#ip flow-export version 9
Router(config)#ip flow-export destination 10.0.0.10 2055

Configure global template settings: refresh-rate, timeout-rate, and options.

To begin, enter the following at the command line:

Router#conf t

Then, enter the configurations for the global template settings:

Router(config)#ip flow-export template refresh-rate 15
Router(config)#ip flow-export template timeout-rate 90
Router(config)#ip flow-export template options export-stats
Router(config)#ip flow-export template options refresh-rate 25
Router(config)#ip flow-export template options timeout-rate 120

Configure the interface settings: enable route-cache flow

To begin, enter the following at the command line:

Router#conf t

Then, enter the configurations for the global template settings:

Router(config)#interface fa0/0
Router(config-if)#ip route-cache flow

Note (Palo Alto users): There is a limited ability to customize the name of Palo Alto interfaces. According to Palo Alto, the interface name cannot be edited. However, you do have the ability to append a numeric suffix to the interface name for subinterfaces, aggregate interfaces, VLAN interfaces, loopback interfaces, and tunnel interfaces.

Note (for Barracuda users): Those using Barracuda NG Firewalls exporting IPFIX/NetFlow v9 will need to consult Barracuda documentation for proper configuration. Specifically, you will need to adjust the following settings: change “Byte Order” to “LittleEndian” and change the IPFIX template for Export to “Default without Barracuda fields”.

Required and Supported Fields for NetFlow Exports

Field Type Number Description Comments
PROTOCOL 4 IP protocol type Mandatory
IPV4_SRC_ADDR 8 IPv4 source address Mandatory for IPv4 addresses (if the Collector is IPv6 enabled and flows have IPv6 addresses, IPv6 source and destination fields (IPV6_SRC_ADDR and IPV6_DST_ADDR) should alternately be used)
IPV4_DST_ADDR 12 IPv4 destination address
DIRECTION 61 Flow direction Optional (if not provided, the default value of 0 will be used, which indicates ingress)
SRC_TOS 5 Type of Service byte setting when entering incoming interface Optional
DST_TOS 55 Type of Service byte setting when exiting outgoing interface Optional
TCP_FLAGS 6 Cumulative of all the TCP flags seen for this flow Optional
LAST_SWITCHED_FT 21 System uptime at which the last packet of this flow was switched Optional (if not provided, current epoch time will be used as the default value)
FIRST_SWITCHED_FT 22 System uptime at which the first packet of this flow was switched Optional (if not provided, current epoch time minus 60 seconds will be used as the default value)

Multicast Group

IS-MULTICAST 206 The first bit of this octet is set to 1 if the Version field of the IP header has the value 4 and if the destination address field contains a reserved multicast address in the range from 224.0.0.0 to 239.255.255.255; otherwise, this bit is set to 0.

The second and third bits of this octet are reserved for future use.
Optional
REPLICATION_FACTOR 99 Multicast replication factor Optional
MUL_DST_PKTS 19 IP multicast outgoing packet counter with length N x 8 bits for packets associated with the IP Flow Optional
MUL_DST_BYTES 20 IP multicast outgoing byte counter with length N x 8 bits for bytes associated with the IP Flow Optional

Interface Group

INPUT_SNMP 10 SNMP ingress interface index At least one of these fields must be present
OUTPUT_SNMP 14 SNMP egress interface index

Bytes Group

IN_BYTES 1 Incoming counter with length N × 8 bits for number of bytes associated with an IP flow At least one of these fields must be present
OUT_BYTES 23 Outgoing counter with length N x 8 bits for the number of bytes associated with an IP flow

Source/Destination Port Groups

L4_SRC_PORT 7 TCP/UDP source port number At least one of these fields must be present
L4_DST_PORT 11 TCP/UDP destination port number

Packets Group

IN_PKTS 2 Incoming counter with length N x 8 bits for the number of packets associated with an IP flow At least one of these fields must be present
OUT_PKTS 24 Outgoing counter with length N x 8 bits for the number of packets associated with an IP flow

NBAR Group

APPLICATION DESCRIPTION 94 Description of the application At least one of these fields must be present
APPLICATION NAME 96 Application name associated with a classification
APPLICATION TAG 95 Eight bits of engine ID, followed by n bits of classification Mandatory
APPLICATION GROUP 12234/45002 Groups applications that belong to the same networking application At least one of these fields must be present
CATEGORY 12232/45000 Provides first-level categorization for each application
ENCRYPTED 290 Specifies whether the application is an encrypted networking protocol
P2P TECHNOLOGY 288 Specifies whether the application is based on peer-to-peer technology
SUB-CATEGORY 12233/45001 Provides second-level categorization for each application
TUNNEL TECHNOLOGY 289 Specifies whether the application tunnels the traffic of other protocols

IPv6 Group

IPV6_SRC_ADDR 27 IPv6 source address Mandatory for flows with IPv6 addresses
IPV6_DST_ADDR 28 IPv6 destination address
IPV6_SRC_MASK 29 Length of the IPv6 source mask in contiguous bits Optional
IPV6_DST_MASK 30 Length of the IPv6 destination mask in contiguous bits Optional
IPV6_FLOW_LABEL 31 IPv6 flow label as per RFC 2460 definition Optional

Sampling Group

FLOW_SAMPLER_ID 48 Identifier shown in “show flow-sampler” Optional
FLOW_SAMPLER_MODE 49 The type of algorithm used for sampling data: 0x02 random sampling Optional
SAMPLING_ALGORITHM 35 The type of algorithm used for sampled NetFlow: 0x01 Deterministic Sampling ,0x02 Random Sampling Optional
FLOW_SAMPLER_RANDOM_INTERVAL 50 Packet interval at which to sample. Use in connection with FLOW_SAMPLER_MODE Optional
SAMPLING_INTERVAL 34 Packet interval at which to sample Optional
SAMPLER_NAME 84 Name of the flow samp Optional

Extended Cisco ASA Device Group

NF_F_CONN_ID 148 An identifier of a unique flow for the resource Optional
NF_F_FLOW_CREATE_TIME_MSEC 152 The time that the flow was created, which is included in extended flow-teardown events in which the flow-create event was not sent earlier. The flow duration can be determined with the event time for the flow-teardown and flow-create times. Optional
NF_F_EVENT_TIME_MSEC 323 The time in which the event occurred, which comes from IPFIX. Use 324 for time in microseconds, and 325 for time in nanoseconds. Time has been counted as milliseconds since 0000 UTC January 1, 1970. Optional
NF_F_FLOW_BYTES 85 Mandatory for Cisco ASA 9.0
NF_F_FW_EVENT_90 40005 At least one of these fields must be present
NF_F_FW_EVENT_91 233 High-level event code. Values are as follows:

  • 0—Default (ignore)
  • 1—Flow created
  • 2—Flow deleted
  • 3—Flow denied
  • 4—Flow alert
  • 5—Flow update
NF_F_FWD_FLOW_DELTA_BYTES 231 The delta number of bytes from source to destination Mandatory for Cisco ASA 9.1
NF_F_REV_FLOW_DELTA_BYTES 232 The delta number of bytes from destination to source Mandatory for Cisco ASA 9.1

IPFIX/NetFlow Version 10 Group

flowStartSeconds 150 The absolute timestamp of the first packet of this flow Optional
flowEndSeconds 151 The absolute timestamp of the last packet of this flow. Optional
flowEndMilliseconds 153 The absolute timestamp of the last packet of this flow Optional
systemInitTimeMilliseconds 160 The absolute timestamp of the last re-initialization of the IPFIX device Optional

Viewing Network Traffic Flow Data

Network traffic flow data is displayed on the Traffic tab on the Resources page for an enabled resource. For more information, see Viewing, Filtering and Reporting on Network Traffic Flow Data.

In This Article