Enterprise Network Monitoring with Flow Data

OpenNMS Meridian is a powerful enterprise network monitoring solution, thanks in part to its flexibility and ability to connect to and monitor a wide range of network devices through network flow data. By providing comprehensive insights into device status, traffic patterns and performance metrics, Meridian empowers your organization to make data-driven decisions.

The platform’s ability to conduct detailed health checks on network devices and generate customized alerts ensures prompt issue resolution and minimal downtime. And its advanced analytics capabilities enable anomaly detection, safeguarding networks against security threats like DDoS attacks.

This blog digs into the concepts of network flow data and how OpenNMS Meridian ingests and integrates it to deliver a robust network monitoring solution.

OpenNMS Meridian is not just a monitoring tool, but a strategic asset for enterprise, ensuring networks are not only healthy and secure but also well-prepared for future growth and challenges.

Network flow data is a way to collect the meta information of a packet entering or leaving a network device. OpenNMS can leverage that metadata with the help of analytics to provide insight on sources, destinations, types of traffic, and the quality of service.

You can think of flow data like airport departure and arrival logs. Consider a network as an airport where data packets are like departing and arriving flights. Flows can be likened to the departure and arrival logs that track various details of each flight, such as the airline flight number, departure, city arrival, city passenger count, and flight duration.

Network engineers face many challenges. It can be difficult to diagnose performance issues, especially in real-time, as users encounter them. A network engineer must look at many aspects of the potential issue. Is this an application issue, or is it the user's device, or could it actually be something on the network?

Even when it has been determined to be an issue with the network, an admin will typically go through a list of questions before they can even begin the troubleshooting process:

• When did the issue start?
• Is the issue constant or intermittent?
• Is the device on the enterprise network or someone's home network?
• Is the issue related to all applications or just a particular one?
• Is there a particular part of the application that is slower, or are there other users experiencing similar issues with that application?

Network elements for data flows consist of exporters, devices like routers, switches, and firewalls; collectors, in our case OpenNMS Meridian; and management and analytics applications, also OpenNMS Meridian. It's useful to understand that while flows are often exported from networking equipment like routers and switches, flow data can also come directly from servers, and they can export telemetry data through the sFlow protocol.

OpenNMS is a collector for data flows. It leverages something called telemetryd or the telemetry daemon to ingest flow information from exporters. The telemetry daemon provides a framework to handle sensor data pushed to Meridian. The framework supports applications that use different protocols to transfer metrics. By default, we use a single port listener, which works for many flow protocols like jFlow, sFlow, and several versions of NetFlow, including IPFIX. With telemetryd, operators can define listeners supporting different protocols to receive the telemetry data and adapters transferring the received data into generic formats like flows or performance data.

OpenNMS stores flow records into Elasticsearch using an OpenNMS-created plugin that installs into your Elasticsearch cluster. You can set up persistence policy for Elasticsearch to only keep flow records for a specific period of time. OpenNMS enriches flow records by using information it already has about systems and its inventory. It tags data flows and groups them based on rules. OpenNMS can leverage node data and the metadata associated with nodes like categories to enrich flow records. This enrichment adds context to speed time to resolution when troubleshooting and can enhance forensic network analysis.

OpenNMS uses a classification engine that applies rules to filter and classify flows. The flow classification engine bundled with Meridian is adapted from IANA standards and includes a predefined set of rules that define more than 6,200 applications for basic communication protocols. You can classify flows by a combination of parameters, including source and destination, port source and destination address IP protocol, and exporter.

Conversations can be identified based on classified flow traffic between a set of hosts, information about quality of service provided by DSCP (Differentiated Services Code Point), and OpenNMS can enrich information about the specific host involved in a flow. Classifications help you determine how flows are associated with a particular appliance service or other component and how they affect your network.

For example, Bitcoin traffic on Port A 333 or all flows to Port AD marked as HTTP. Meridian allows for customized rules to classify flows. A rule includes a name for the classification or application and additional parameters such as source and destination ports and addresses that must match.

OpenNMS overcomes the challenge of collecting large amounts of flow data by distributing collection across Minions. Minions are a lightweight stateless service used to add capacity and reduce the load on Meridian core. By enabling horizontal scaling of collection, minions can be deployed as virtual machines, containers, hardware appliances, or physical servers.

Meridian overcomes the challenges of enriching large amounts of flow data by distributing processing workload through sentinels. Sentinels are purpose-built modules that allow Meridian to scale flow enrichment horizontally by transparently offloading work and placing enriched flow records directly into Elasticsearch. Sentinels can be installed on containers, virtual machines, or physical hardware and have requirements similar to Meridian.

OpenNMS can scale to your network size needs. Some of our customers ingest and enrich over 350,000 flows per second.

In summary, Meridian can ingest many types of flow and telemetry data. With the analytics Meridian provides for sources, destinations, types of traffic, and the quality of service, it's possible to gain holistic insight into your network. Using the powerful Meridian flow enrichment capabilities, you can quickly pinpoint the sources of networking issues and reduce time to resolution. OpenNMs Meridian can be deployed on a single host and can scale with your needs by using Minions and sentinels to enable collecting and enriching large amounts of data.

If you have a complex or demanding use case for flows, then get in touch. We're here to help.