Chapter 1. New and Changed Information

The following table provides an overview of the significant changes to this guide up to this current release. The table does not provide an exhaustive list of all changes that are made to the guide or of the new features up to this release.

New Features and Changed Behavior

Cisco APIC Release Version

Feature

Description

5.1(3)

OpFlex support for ERSPAN with OVS

This guide became available.

5.2(1)

OpFlex support for ERSPAN with OVS on Kubernetes

Added a procedure for configuring OpFlex for ERSPAN with OVS on Kubernetes setups.

- About OpFlex support for ERSPAN with OVS

- Benefits of OpFlex support for ERSPAN with OVS

- OpFlex support for ERSPAN with OVS Limitations and Restrictions

- Prerequisites for Configuring OpFlex support for ERSPAN with OVS

- OpFlex support for ERSPAN with OVS Configuration Workflow

- Configuring the OpFlex support for ERSPAN with OVS on OpenStack setups

- Configuring OpFlex Support for ERSPAN with OVS on Kubernetes Setups

- Verify the OpFlex support for ERSPAN with OVS

About OpFlex support for ERSPAN with OVS

The OpFlex support for ERSPAN with OVS feature extends the basic port mirroring capability from Layer 2 to Layer 3 which allows the mirrored traffic to be sent through a routable IP network from specific interfaces.

Benefits of OpFlex support for ERSPAN with OVS

The OpFlex support for ERSPAN with OVS provides several benefits:

- The ability to get mirrored traffic directly from specific interfaces providing visibility of local traffic.

- Debugging network issues by tracking the control and data frames and security analysis.

OpFlex support for ERSPAN with OVS Limitations and Restrictions

Be aware of the following issues when configuring OpFlex support for ERSPAN with OVS:

- Red Hat Enterprise Linux 7 (RHEL7) is not supported.

Prerequisites for Configuring OpFlex support for ERSPAN with OVS

You must complete the following tasks before you configure OpFlex support for ERSPAN with OVS on OpenStack/Kubernetes setups:

- ERSPAN leverages support added to Open vSwitch (OVS). ERSPAN sessions are initialized on ports connected to a switch in OVS. In order to support ERSPAN, the hosts running OVS must meet the following requirements:

- Linux kernel version 4.18 or later

- OVS version 2.10 or later

Note

Red Hat Enterprise Linux 7 (RHEL7) does not meet these requirements, but Red Hat Enterprise Linux 8 (RHEL8) does.

- You must have RHEL8 OS installed.

- The host where OpenStack CLI commands are run must also have the Group Based Policy (GBP) python client library installed.

- You must have basic working knowledge of an OpenStack environment.

- (applicable only for ERSPAN with OVS on Kubernetes setups) You must have Ubuntu 20 installed.

OpFlex support for ERSPAN with OVS Configuration Workflow

This section describes a high-level overview of the tasks you perform to configure OpFlex support for ERSPAN with OVS:

Procedure

Step 1

Get the neutron port that is used to create the ERSPAN source. The MAC address of this port will be the source of the configuration.

For more information, see Configuring the OpFlex support for ERSPAN with OVS on OpenStack setups.

Step 2

Run the OpenStack CLI commands described in the next section to configure erspan source and destination sessions.

For more information, see Configuring the OpFlex support for ERSPAN with OVS on OpenStack setups.

Step 3

Verify the configuration.

For more information, see Verify the OpFlex support for ERSPAN with OVS.

Configuring the OpFlex support for ERSPAN with OVS on OpenStack setups

This section describes how to configure the OpFlex support for ERSPAN with OVS on OpenStack setups.

Procedure

Step 1

Configuring ERSPAN sessions.

This release introduces support for ERSPAN sessions on OpenStack ports. ERSPAN sessions are configured using the --apic-erspan-config option in the OpenStack python client/CLI. This option is supprted both when creating and updating a port in OpenStack.

# openstack port create --apic-erspan-config <apic_erspan_config> --network <network> <name>
# openstack port set --apic-erspan-config <apic_erspan_config> <port>

The <apic_erspan_config> field consists of the following comma-separated parameters:

- 'dest-ip': the ERSPAN destination IP address

- 'flow-id': the flow ID to use for the session (1-1023)

- 'direction': 'in', 'out', or 'both' (port-centric)

The dest-ip and flow-id fields are mandatory, while the direction is optional (default value is "both"). The OpenStack port UUID plus direction define a unique ERSPAN source, while the destination IP and flow ID define a unique ERSPAN destination. ERSPAN sessions can only be active when the port is bound to an opflex type segment. This means that hierarchical port binding (HPB) and 'vlan' type segments are not supported, nor are SVI networks. Ports must also have a vnic_type of "normal", and have a device_owner prefix of "compute:". Multiple ERSPAN sessions can be applied to a single OpenStack port, simply by adding additional --apic-erspan-config options.

Step 2

Verifying ERSPAN configuration in OpenStack.

The status of the ERSPAN session can be examined through the apic:synchronization_state property of the port. Run the following command to see the state:

# openstack port show UUID or name-of-port

The aggregate state of the ERSPAN configuration is reflected in the port's apic_synchronization_state. This field can have the following values:

- N/A: Either:

- The state has not yet been synced with ACI; or

- There either is no ERSPAN state, or the port is not bound

- build: ERSPAN state is being synchronized with ACI

- error: ERSPAN state could not be synchronized with ACI

- synced: All ERSPAN state is synchronized with ACI. ERSPAN sessions should now be active

ERSPAN traffic is sent from the local vSwitch to the host, and the host's IP stack forwards the encapsulated packet. ERSPAN sessions may experience some drop-out when live-migrating the ports with ERSPAN configuration, due to port rebinding.

Step 3

Verifying ERSPAN configuration in Open vSwitch.

You can confirm the ERSPAN sessions on host vSwitches using the "ovs-vsctl show" command:

[root@overcloud-novacompute-0 heat-admin]# ovs-vsctl show
021cf127-1978-4096-9897-a5d0e0b20b23
Manager "ptcp:6640:127.0.0.1"
is_connected: true
....
Bridge br-int
fail_mode: secure
Port "172.28.184.58-2"
Interface "172.28.184.58-2"
type: erspan
options: {erspan_dir="1", erspan_hwid="4", erspan_ver="2", key="2", remote_ip="172.28.184.58"}
Port "172.28.184.58-3"
Interface "172.28.184.58-3"

Step 4

Removing ERSPAN configuration in OpenStack.

When the port is unbound, the ERSPAN session is terminated. However, the ERSPAN configuration is still present in the port, and if the port is bound again, then the ERSPAN session will be resumed. The ERSPAN configuration state can only be removed explicitly or when the port is deleted. To remove all the ERSPAN configuration from the port, enter the following command:

# openstack port set --no-apic-erspan-config <port>

Note

There is a known issue, which prevents ERSPAN state from being cleaned up on the vSwitch. The workaround is to manually remove ERSPAN ports from the vSwitch. For example:

Example:

# sudo ovs-vsctl del-port br-int 172.28.184.58-2

Configuring OpFlex Support for ERSPAN with OVS on Kubernetes Setups

This section describes how to configure the OpFlex support for ERSPAN with OVS on Kubernetes setups.

Procedure

Step 1

Verify if the Custom Resources Definition (CRD) is available. Check for faults and violations using the following commands:

$ kubectl get crd
$ kubectl describe crd erspanpolicies.aci.erspan

Step 2

Apply the CRD yaml file with valid inputs; using the $ kubectl apply -f <yaml_file> command.

A sample CRD yaml file is shown below:

apiVersion: aci.erspan/v1alpha
kind: ErspanPolicy
metadata:
name: erspan-policy
namespace: default
spec:
selector:
labels:
app: consul
namespace: default
source:
adminState: "start"
direction: "both"
destination:
destIP: "1.1.1.1"
flowID: 1

Verify the OpFlex support for ERSPAN with OVS

This section describes how to verify the OpFlex support for ERSPAN with OVS on OpenStack setups.

Procedure

Step 1

Log in to the Cisco APIC GUI, on the menu bar, choose Fabric > Access Policies.

Step 2

You need to verify the successful creation of ERSPAN Source. In the Navigation pane, choose Policies > Troubleshooting > VSPAN > VSPAN Sessions and click on a VSPAN session.

Step 3

You need to verify the successful creation of ERSPAN Destination. In the Navigation pane, choose Policies > Troubleshooting > VSPAN > VSPAN Destination Groups and click on a VSPAN Destination Group.

Step 4

You need to verify the successful binding of ERSPAN session with VPCs. In the Navigation pane, choose Interfaces > Leaf Interfaces > Policy Groups > VPC Interface and click on the VPC interface policy group.

Step 5

Inside the opflex_agent container, enter the following command:

# gbp_inspect -prq SpanSession

Step 6

On the compute nodes, check for the mirror, enter the following command:

$ ovs-vsctl list mirror

Notices

Full Cisco Trademarks with Software License

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.

Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: https://www.cisco.com/c/en/us/about/legal/trademarks.html. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)