Guest

Cisco Extensible Network Controller (XNC)

Cisco Nexus Data Broker: Scalable and Cost-Effective Solution for Network Traffic Visibility

  • Viewing Options

  • PDF (972.0 KB)
  • Feedback

What You Will Learn

This document discusses a software-defined approach to network traffic monitoring and visibility (packet broker solution) by using Cisco Nexus® Data Broker software and Cisco Nexus switches. This approach uses OpenFlow-enabled Cisco Nexus switches along with Cisco Nexus Data Broker software, which extends the use of general switching devices as fully functional network traffic-monitoring switches without the limitations observed in some conventional approaches that exist today.

Challenge

Visibility into application traffic has traditionally been important for infrastructure operations to maintain security, resolve problems, and perform resource planning. Now, however, as a result of technological advances and the ubiquity of the Internet, organizations increasingly are seeking not just visibility but real-time feedback about their business systems to more effectively engage their customers. Also, with the rapid evolution of cloud-based technologies, there is a strong need for scalable and cost-effective network traffic aggregation and monitoring solutions.

Why Matrix Switches?

To understand the need for matrix switches, consider some of the methods commonly deployed in the data center to monitor network traffic and flows.

Hubs

For a long time, hubs were used for network monitoring in the Ethernet segment. The use of hubs to monitor network data traffic presents numerous fundamental problems. The most notable problem is that hubs forward all traffic on all ports to all other ports. Hubs also increase the total size of the collision domain, preventing the network from operating efficiently. In contrast, a network switch reduces this domain by learning MAC addresses connected to each physical interface and therefore can forward traffic destined for those ports respectively. The end nodes connected to the switch do not all see all traffic on other devices, and they each receive only the frames that they are intended to receive. In general, hubs also support only half-duplex and 10- and 100-Mbps transfer speeds.

Cisco Switched Port Analyzer

The Cisco® Switched Port Analyzer (SPAN) feature was introduced on switches to address the fundamental difference between switches and hubs. When a hub receives a packet on one port, the hub sends a copy of that packet from all ports except the one on which the hub received the packet. The switch builds a Layer 2 forwarding table on the basis of the source MAC addresses of the various packets that the switch receives. After this forwarding table is built, the switch forwards traffic destined for a MAC address directly to the corresponding port. To be able to monitor traffic on the switches, the SPAN feature is needed (Figure 1).

Figure 1. Cisco SPAN

SPAN uses the following terminology:

Ingress traffic: Traffic that enters the switch

Egress traffic: Traffic that leaves the switch

Source (SPAN) port: A port that is monitored with the SPAN feature

Source (SPAN) VLAN: A VLAN whose traffic is monitored with the SPAN feature

Destination (SPAN) port: A port that monitors source ports, usually ports with a sniffer connected

SPAN, however, also has limitations. In most common hardware switching architectures that exist today, a single application-specific integrated circuit (ASIC), or multiple ASICs grouped together to build a switching fabric, forwards packets at the hardware level, and very little data-plane information is sent to the CPU. Even modular chassis today have adopted the distributed forwarding model; however, when the software needs to tell the hardware to duplicate and redirect packets as in the case of SPAN, that process consumes additional CPU cycles. As a result, the number of SPAN and port monitoring sessions that can be configured on the switch is limited. Numerous factors affect network device performance when using SPAN:

Process of sending packets to the CPU and forwarding them to their destinations

Basic monitoring of the interface

Volume of SPAN traffic (even more crucial on a 10 Gigabit Ethernet network)

Network Taps

Another method of packet monitoring uses physical hardware taps. These network taps can be extremely useful in monitoring traffic because they provide direct inline access to data flowing through the network. In many cases, it is desirable for a third party to monitor the traffic between two points in the network. If the network between points A and B consists of a physical cable, a network tap may be the best way to accomplish this monitoring. The network tap has at least three ports: anA port, aBport, and amonitorport. A tap inserted between the A and B ports passes all traffic through unimpeded, but it also copies that same data to its monitor port, enabling a third party to listen (Figure 2).

Figure 2. Network Tap

Taps have many benefits:

They can handle full-duplex data transmission.

They are nonobtrusive and not detectable by the network, with no physical or logical addressing.

Some support full inline power with the capability to build a distributed tap.

Whether you are trying to gain visibility into the server-to-server data communication at the edge or virtual edge of your network or to provide a copy of traffic to the intrusion-prevention-system (IPS) appliance at the Internet edge of your network, you can use network taps nearly anywhere in the environment. However, this deployment can add significant costs and operation complexities, introducing cabling challenges in a large-scale environment.

The Solution: Cisco Nexus Data Broker

The traditional approach to help with traffic-monitoring and troubleshooting tasks in the data center is purpose-built matrix switches that can aggregate multiple taps and SPAN and also connect to multiple systems for security, compliance and application performance monitoring. The traditional approach poses three primary challenges:

The approach is too expensive to scale the visibility to meet today’s business requirements.

The purpose-built switches are statically programmed with predetermined filtering and forwarding rules, sothey cannot act in an event-based way to provide traffic visibility in real time.

Support for interconnecting multiple switches for a scalable deployment that suits your data center architecture is limited.

Using the Cisco Nexus Data Broker, Cisco’s approach replaces the purpose-built matrix switches with one or more OpenFlow-enabled Cisco Nexus switches. The traffic is tapped into this bank of switches in the same manner as in a matrix network. However, with Cisco Nexus Data Broker, you can interconnect these Cisco Nexus switches to build a scalable tap and SPAN aggregation infrastructure. You also can combine tap and SPAN sources to bring the copy of the production traffic to this infrastructure. In addition, you can distribute the tap and SPAN sources and traffic monitoring and analysis tools across multiple Cisco Nexus switches.

In Figure 3, the Cisco Nexus switches are connected to various points in the network at which packet monitoring is advantageous. From each network element, SPAN ports or optical taps can be used to send traffic flows directly to this matrix switch. The matrix switch itself is directly connected to all the tools used to monitor the events in the network fabric. These monitoring devices include Remote Monitoring (RMON) probes, application firewalls, IDS devices, and packet sniffer tools.

Figure 3. Tap and SPAN Aggregation with Cisco Nexus Data Broker

The Cisco Nexus Data Broker application discovers the tap and SPAN aggregation network topology and maintains the state of this network. It also maintains the mapping information of the monitoring tools connected to the Cisco Nexus switch and port.

With Cisco Nexus Data Broker, the network administrator can aggregate the tap and SPAN traffic from multiple input sources, replicate, and forward the traffic to multiple monitoring tools that may be connected to different Cisco Nexus switches. Also, the network administrator can filter traffic based on Layer 1 through Layer 4 header information and forward the traffic to multiple monitoring tools.

These functions allow you to build a fully distributed traffic-monitoring network that works as one logical device, with the capability to change the filtering and forwarding rules dynamically. Additionally, because Cisco Nexus Data Broker provides a REST interface, network operators can write applications to detect and capture unique traffic according to administrative requirements or traffic-monitoring needs. This solution thus allows unique and important traffic patterns to flow directly to the analysis tools in real time. With the elasticity and ease of use of the complete monitoring solution, Cisco Nexus Data Broker can provide greater business agility through a cost-effective and scalable approach for traffic monitoring. All these features can be very attractive to organizations that have or require some type of monitoring solution today. Table 1 provides a summary of the Cisco Nexus Data Broker functions.

Table 1. Cisco Nexus Data Broker Functions Summary

Feature

Benefit

Supported topology for Cisco Monitor Manager network

Cisco Nexus Data Broker software discovers the Cisco Nexus switches and associated topology for tap and SPAN aggregation.
The software allows you to configure ports as monitoring tool ports or input tap and SPAN ports.
You can set end-device names for easy identification in the topology.

Support for QinQ to tag input source tap or SPAN port

You can configure the software to tag traffic with a VLAN for each input tap or SPAN port.
Q-in-Q in edge tap and SPAN ports allows you to uniquely identify the source of traffic and preserve production VLAN information.

Symmetric hashing or symmetric load balancing+

You can configure the hashing based on Layer 3 (IP address) or Layer 3 + Layer 4 (protocol ports) for load balancing the traffic across a port-channel link.
You can configure the software to spread the traffic across multiple tool instances to meet the high-traffic-volume scale.

Rules for matching monitored traffic

You can match traffic based on Layer 1 through Layer 4 criteria.
You can configure the software to send only the required traffic to the monitoring tools without flooding the tools with unnecessary traffic.
You can configure the action to set the VLAN ID for the matched traffic.

Replicate and forward traffic

You can configure the software to aggregate traffic from multiple input tap and SPAN ports that could be spread across multiple Cisco Nexus switches.
You can configure the software replicate and forward traffic to multiple monitoring tools that can be connected across multiple Cisco Nexus switches.
This solution is the only one that supports any:many forwarding across a topology.

Time-stamping*

You can time-stamp a packet at ingress using Precision Time Protocol (PTP; IEEE 1588), thereby providing nanosecond accuracy. You can time-stamp the packets using PTP for critical transaction monitoring and archiving data for regulatory compliance and advanced troubleshooting.

Packet truncation*

You can configure the software to truncate a packet beyond specified bytes.
The minimum is 64 bytes.
You can retain a header only for analysis and troubleshooting.
You can discard the payload for security or compliance reasons.

React to changes in the tap and SPAN aggregation network states

You can monitor and keep track of network condition changes.
You can configure the software to react to link or node failures by automatically reprogramming the flows through an alternative path.

End-to-end path visibility

For each traffic forwarding rule, the solution provides a complete end-to-end path visibility all the way from source ports to the monitoring tools, including the path through the network.

Management for multiple disjointed Cisco monitoring networks

You can manage multiple independent traffic-monitoring networks, which may be disjointed, using the same Cisco Nexus Data Broker instance. For example, if you have five data centers and you want to deploy an independent Nexus Data Broker solution for each data center, you can manage all five independent deployments using a single Cisco Nexus Data Broker instance by creating a logical partition (network slice) for each monitoring network.
In addition to the ability to manage multiple disjointed Cisco monitoring networks, with the robust role-based access control (RBAC) support, you can assign users to specific network slices so that they will be able to manage only their specific portion of the traffic-monitoring network.

Role Based Access Control (RBAC)

Application access can be integrated with corporate AAA server for both authentication and authorization
You can create port groups and associate the port groups with specific user roles
Capability to assign users to specific roles and port groups; users can manage only those ports

Cisco’s solution also provides various access mechanisms to the Cisco Nexus Data Broker functions:

Web-based GUI interface for all configuration and user management (Figure 4).

Robust REST API for programmatic access of functions and enablement of event-directed filtering and forwarding approach (Figure 5).

Figure 4. Cisco Nexus Data Broker Web GUI Access Mechanisms

Figure 5. Cisco Nexus Data Broker REST API Access Mechanisms

Cisco Nexus Data Broker Embedded Option

In order to reduce the overhead involved in the traffic-monitoring solution deployment and to keep it simple when customers want to deploy the solution using a single switch (Cisco Nexus 3000, Nexus 3100, or Nexus 3500), you can use the Cisco Nexus Data Broker Embedded option, which allows you to run the software on the Cisco Nexus 3000, Nexus 3100, or Nexus 3500 Linux container without any need for a separate virtual machine (Figure 6). In this option a special open virtual appliance (OVA) is provided; the OVA needs to be deployed on the Cisco Nexus switch and activated. All the features of the application are also available in this option except:

You can use it to manage flows only on the switch on which the Cisco Nexus Data Broker Embedded OVA is deployed.

You can deploy only one instance in each switch (no high-availability support).

Figure 6. Cisco Nexus Data Broker Embedded Deployment

Device Support Matrix for Cisco Nexus Data Broker

Table 2 lists the Cisco Nexus switches that support Cisco Nexus Data Broker.

Table 2. Cisco Nexus Data Broker Device Support Matrix

Device Model

Cisco Nexus Data Broker Embedded Support

Cisco Nexus 3000 (3048, 3064, and 3016 models)

Yes

Cisco Nexus 3100 (3132, 3172, and 3164 models)

Yes

Cisco Nexus 3500 (3548 and 3524 models)

Yes

Cisco Nexus 5500

No

Cisco Nexus 6000

No

Cisco Nexus 7000

No

Cisco Nexus 7700

No

Features of the Cisco Nexus Data Broker

Table 3 presents the main features of Cisco Nexus Data Broker.

Table 3. Cisco Nexus Data Broker Features

Cisco Nexus Data Broker Software Features

Functional Category

Feature Description

Configuration

GUI

Cisco Nexus Data Broker software provides a web-based GUI for management of all configurations and functions. The GUI provides access features, including:

Topology and device management and assignment of port type
Mapping of the ports to the end monitoring or analysis tools
Configuration of filters to match traffic according to business needs
Set up of traffic flows from network-edge ports to tool-delivery ports
Event logging and troubleshooting
Flow and port statistics details
RBAC user and role management

Northbound API

Cisco Nexus Data Broker REST-based API provides access to all functions that can be performed through the GUI.

Port-type assignment

Ports must be designated as edge tap or SPAN (input) or delivery (output) ports to be used for configuring network connections. This feature, in combination with RBAC, increases network security.

InterSwitch Links (ISLs) Discovery

ISLs are ports that interconnect two switches. These ISLs can use individual ports or port channels. These ISLs are automatically discovered by the Cisco Nexus Data Broker software and maintains the state of these ISLs.

Traffic Delivery (Basic)

One-to-one connection

This type establishes a one-to-one connection from an edge-network port to a tool-delivery port across the network with no oversubscription.

One-to-many connection

This type establishes a one-to-many connection from an edge-network port to multiple tool-delivery ports.

Many-to-one connection

This type establishes a many-to-one connection from multiple edge-network ports to a single tool-delivery port.

Combination

One-to-one, one-to-many, and many-to-one connections can be established for different flows at the same time in the same monitored network.

Port-speed adaptation

One-to-one, one-to-many, and many-to-one connections can be established between ports with different speeds. For instance, a 40-Gbps port can deliver traffic to a 10-Gbps tool port to allow use of traditional tools over high-speed production network interfaces.

Symmetric load balancing

You can configure symmetric hashing based on Layer 3 (IP address) information or Layer 3 and Layer 4 (protocol + port) information to load balance the traffic to multiple monitoring tools.

Failure resiliency

If a path fails, each flow is automatically rerouted to an alternative path by the controller. If rerouting is not possible, an event is logged.

Traffic Delivery (Advanced)

Packet filtering

Traffic forwarding is based on the full-flow specification, allowing detailed traffic filtering to limit the traffic to the delivery port to only what is strictly necessary.

VLAN tag rewrite

You can change the original VLAN tag from the edge port to the delivery port either through the filter mechanism or by tagging at the edge port.

VLAN tag insertion

You can add a VLAN tag to the original packet to be delivered to allow a tool to identify the origin of the traffic.

QinQ support

If the packet is already tagged, Cisco Nexus Data Broker software can add a second tag that allows you to preserve the original tag information and also identify the edge tap or SPAN port from which the traffic is received.

Time-stamping

You can configure the software to time-stamp packets using the Precision Time Protocol (PTP) with nanosecond accuracy for compliance, troubleshooting, or application-performance management.

Packet truncation

You can strip packet payloads for security and compliance purposes. Minimum packet length is 64 bytes, and you can specify the number of bytes to be retained. Packet payload is then truncated beyond the specified byte size and delivered to monitoring tools.

Network Design

Multilevel design

Cisco Nexus Data Broker software can support multiple Cisco Nexus switches connected in any topology. Analysis and monitoring devices can be connected anywhere in the topology. Typical tapping network architectures follow:

Two- or three-level networks (edge, distribution layer [optional], and core) in which the delivery ports are connected to the core switches
Nonblocking leaf-and-spine architectures, in which both the edge and the delivery ports are connected to the leaf switches

Loop prevention

Built-in logic prevents creation of network loops. This feature supports one-to-one, one-to-many, many-to-one, and any-to-many replication and redirection policies.

Scalability

Cisco Nexus Data Broker can support up to 100 switches and 4000 edge and delivery ports per instance.

High availability

Cisco Nexus Data Broker supports high availability through active-active clustering. Using the current release, up to five instances can be part of the same cluster.

Security and Operations

RBAC

Each port can be exclusively assigned to one or more user groups.

Logging

Cisco Nexus Data Broker provides system logs as well as user audit logs. In addition, it supports different logging levels depending on administrator needs.

Path rerouting to guarantee delivery

If traffic is critical, data loss can impair compliance. In this case, if a network link or node failure occurs, the data flow is automatically rerouted using an alternate network path to prevent data loss and to meet compliance requirements.

Cisco Nexus Data Broker System Requirements

Cisco Nexus Data Broker software

Minimum system requirements follow:

64-bit Linux operating system (Fedora, Ubuntu, or Redhat)
8 GB of RAM, 6-core CPU, and 40 GB of free space in the partition in which the controller will be installed
Java Release 1.7

For More Information

Cisco Nexus Data Broker