Configuring SPAN

This chapter describes how to configure an Ethernet switched port analyzer (SPAN) to analyze traffic between ports on Cisco NX-OS devices.

About SPAN

SPAN analyzes all traffic between source ports by directing the SPAN session traffic to a destination port with an external analyzer attached to it.

You can define the sources and destinations to monitor in a SPAN session on the local device.

SPAN Sources

The interfaces from which traffic can be monitored are called SPAN sources. Sources designate the traffic to monitor and whether to copy ingress (Rx), egress (Tx), or both directions of traffic. SPAN sources include the following:

  • Ethernet ports (but not subinterfaces)

Characteristics of Source Ports

SPAN source ports have the following characteristics:

  • A port configured as a source port cannot also be configured as a destination port.

SPAN Destinations

SPAN destinations refer to the interfaces that monitor source ports. Destination ports receive the copied traffic from SPAN sources. SPAN destinations include the following:

  • Ethernet ports in either access or trunk mode

Characteristics of Destination Ports

SPAN destination ports have the following characteristics:

  • A port configured as a destination port cannot also be configured as a source port.

  • The same destination interface cannot be used for multiple SPAN sessions. However, an interface can act as a destination for a SPAN and an ERSPAN session.

  • Destination ports do not participate in any spanning tree instance. SPAN output includes bridge protocol data unit (BPDU) Spanning Tree Protocol hello packets.

SPAN Sessions

You can create SPAN sessions to designate sources and destinations to monitor.

See the Cisco Nexus 9000 Series NX-OS Verified Scalability Guide for information on the number of supported SPAN sessions.

This figure shows a SPAN configuration. Packets on three Ethernet ports are copied to destination port Ethernet 2/5. Only traffic in the direction specified is copied.

Figure 1. SPAN Configuration

Localized SPAN Sessions

A SPAN session is localized when all of the source interfaces are on the same line card. A session destination interface can be on any line card.

SPAN Truncation

Beginning with Cisco NX-OS Release 7.0(3)I7(1), you can configure the truncation of source packets for each SPAN session based on the size of the MTU. Truncation helps to decrease SPAN bandwidth by reducing the size of monitored packets. Any SPAN packet that is larger than the configured MTU size is truncated to the given size. For example, if you configure the MTU as 300 bytes, the packets with greater than 300 bytes are truncated to 300 bytes.

SPAN truncation is disabled by default. To use truncation, you must enable it for each SPAN session.

ACL TCAM Regions

You can change the size of the ACL ternary content addressable memory (TCAM) regions in the hardware. For information on the TCAM regions used by SPAN sessions, see the "Configuring IP ACLs" chapter of the Cisco Nexus 9000 Series NX-OS Security Configuration Guide.

Prerequisites for SPAN

SPAN has the following prerequisites:

  • You must first configure the ports on each device to support the desired SPAN configuration. For more information, see the Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide.

Guidelines and Limitations for SPAN


Note

For scale information, see the release-specific Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

SPAN has the following configuration guidelines and limitations:

  • Traffic that is denied by an ACL may still reach the SPAN destination port because SPAN replication is performed on the ingress side prior to the ACL enforcement (ACL dropping traffic).

  • For SPAN session limits, see the Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

  • All SPAN replication is performed in the hardware. The supervisor CPU is not involved.

  • The Cisco Nexus 3232C and 3264Q switches do not support SPAN on CPU as destination.

  • You can configure a SPAN session on the local device only. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • Packets with FCS errors are not mirrored in a SPAN session.

  • FEX and SPAN port-channel destinations are not supported on the Cisco Nexus 9500 platform switches with an -EX or –FX type line card.

  • You can configure only one destination port in a SPAN session.

  • SPAN mirroring is not supported for PBR traffic.

  • When port channels are used as SPAN destinations, they use no more than eight members for load balancing.

  • Beginning with Cisco NX-OS Release 7.0(3)I1(1), a maximum of 48 source interfaces are supported per SPAN session (Rx and Tx, Rx, or Tx).

  • SPAN does not support destinations on Cisco Nexus 9408PC-CFP2 line card ports.

  • Configuring two SPAN or ERSPAN sessions on the same source interface with only one filter is not supported. If the same source is used in multiple SPAN or ERSPAN sessions either all the sessions must have different filters or no sessions should have filters.

  • Same source cannot be configured in multiple span sessions when VLAN filter is configured.

  • The following guidelines apply to SPAN copies of access port dot1q headers:

    • When traffic ingresses from a trunk port and egresses to an access port, an egress SPAN copy of an access port on a switch interface always has a dot1q header.

    • When traffic ingresses from an access port and egresses to a trunk port, an ingress SPAN copy of an access port on a switch interface does not have a dot1q header.

    • When traffic ingresses from an access port and egresses to an access port, an ingress/egress SPAN copy of an access port on a switch interface does not have a dot1q header.

  • You cannot configure a port as both a source and destination port.

  • Enabling UniDirectional Link Detection (UDLD) on the SPAN source and destination ports simultaneously is not supported. If UDLD frames are expected to be captured on the source port of such SPAN session, disable UDLD on the destination port of the SPAN session.

  • SPAN is not supported for management ports.

  • Statistics are not support for the filter access group.

  • SPAN is supported in Layer 3 mode; however, SPAN is not supported on Layer 3 subinterfaces or Layer 3 port-channel subinterfaces.

  • Beginning with Cisco NX-OS Release 7.0(3)I4(1), the same source can be part of multiple sessions.

  • When a SPAN session contains source ports that are monitored in the transmit or transmit and receive direction, packets that these ports receive might be replicated to the SPAN destination port even though the packets are not actually transmitted on the source ports. Some examples of this behavior on source ports are as follows:

    • Traffic that results from flooding

    • Broadcast and multicast traffic

  • SPAN sessions cannot capture packets with broadcast or multicast MAC addresses that reach the supervisor, such as ARP requests and Open Shortest Path First (OSPF) protocol hello packets, if the source of the session is the supervisor Ethernet in-band interface. To capture these packets, you must use the physical interface as the source in the SPAN sessions.

  • VLAN SPAN monitors only the traffic that enters Layer 2 ports in the VLAN.

  • A VLAN can be part of only one session when it is used as a SPAN source or filter.

  • VLANs can be SPAN sources in the ingress and egress direction on Cisco Nexus 9508 switches with N9K-X9636C-R and N9K-X9636Q-R line cards. For all other switches, VLANs are supported as SPAN sources only in the ingress direction.

  • VLAN ACL redirects to SPAN destination ports are not supported.

  • When using a VLAN ACL to filter a SPAN, only action forward is supported; action drop and action redirect are not supported.

  • The combination of VLAN source session and port source session is not supported. If the traffic stream matches the VLAN source session as well as port source session, two copies are needed at two destination ports. Due to the hardware limitation, only the VLAN source SPAN and the specific destination port receive the SPAN packets. This limitation applies only to the following Cisco devices:

    Table 1. Cisco Nexus 9000 Series Switches

    Cisco Nexus 93120TX

    Cisco Nexus 93128TX

    Cisco Nexus 9332PQ

    Cisco Nexus 9372PX

    Cisco Nexus 9372PX-E

    Cisco Nexus 9372TX

    Cisco Nexus 9396PX

    Cisco Nexus 9372TX-E

    Cisco Nexus 9396TX
    Table 2. Cisco Nexus 9000 Series Line Cards, Fabric Modules, and GEM Modules

    N9K-X9408PC-CFP2

    N9K-X9536PQ

    N9K-C9508-FM

    N9K-X9432PQ

    N9K-X9564PX

    N9K-C9504-FM

    N9K-X9464PX

    N9K-X9564TX

    N9K-C9516-FM

    N9K-X9464TX

    N9K-X9636PQ

    N9K-M4PC-CFP2

  • For VXLAN/VTEP, SPAN source or destination is supported on any port.

  • The number of SPAN sessions per line card reduces to two if the same interface is configured as a bidirectional source in more than one session. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • A single forwarding engine instance supports four SPAN sessions. For Cisco Nexus 9300 Series switches, if the first three sessions have bidirectional sources, the fourth session has hardware resources only for Rx sources. This limitation might also apply to Cisco Nexus 9500 Series switches, depending on the SPAN source's forwarding engine instance mappings. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • An access-group filter in a SPAN session must be configured as vlan-accessmap. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • Beginning with Cisco NX-OS Release 7.0(3)I7(3), NetFlow, and SPAN functionality is supported on Cisco Nexus 9336C-FX2 and Cisco Nexus 93240YC-FX2 switches.

  • Supervisor-generated stream of bytes module header (SOBMH) packets have all of the information to go out on an interface and can bypass all forwarding lookups in the hardware, including SPAN and ERSPAN. CPU-generated frames for Layer 3 interfaces and the Bridge Protocol Data Unit (BPDU) class of packets are sent using SOBMH. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards. The Cisco Nexus 9636C-R and 9636Q-R both support inband SPAN and local SPAN.

  • In Cisco NX-OS Release 7.0(3)I2(1) and earlier releases, IPv6 ACL filters for Layer 2 ports are not supported on Cisco Nexus 9000 Series switches and the Cisco Nexus 3164Q switch.

  • Cisco NX-OS does not span Link Layer Discovery Protocol (LLDP) or Link Aggregation Control Protocol (LACP) packets when the source interface is not a host interface port channel.

  • Beginning with Cisco NX-OS Release 7.0(3)I4(1), Cisco Nexus 9300, and 9500 platform switches support multiple ACL filters on the same source.

The following guidelines and limitations apply to egress (Tx) SPAN:

  • The following limitations apply to egress (Tx) SPAN and these switches:

    • Cisco Nexus 92160YC-X

    • Cisco Nexus 92304QC

    • Cisco Nexus 9272Q

    • Cisco Nexus 9236C

    • Cisco Nexus 92300YC

    ACL filtering is not supported (applies to both unicast and Broadcast, Unknown Unicast and Multicast (BUM) traffic)

    VLAN filtering is supported, but only for unicast traffic

    VLAN filtering is not supported for BUM traffic

  • SPAN copies for multicast packets are made prior to rewrite. Therefore, the TTL, VLAN ID, any remarking due to egress policy, and so on, are not captured in the SPAN copy.

  • If SPAN is mirroring the traffic which ingresses on an interface in an ASIC instance and egresses on a Layer 3 interface (SPAN Source) on a different ASIC instance, then TX mirrored packet will have a VLAN ID 4095 on Cisco Nexus 9500 platform modular switches using non-EX line cards.

  • Only Cisco Nexus 9300-EX platform switches support SPAN for multicast Tx traffic across different slices, beginning with Cisco NX-OS Release 7.0(3)I7(1). The slices must be on the same leaf spine engine (LSE).

  • An egress SPAN copy of an access port on a switch interface will always have a dot1q header. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • For Tx interface SPAN with Layer 2 switch port and port-channel sources on Cisco Nexus 9300-EX Series switches, only one copy is made per receiver unit regardless of how many Layer 2 members are receiving the stream in the same VLAN. For example, if e1/1-8 are all Tx direction SPAN sources and all are joined to the same group, the SPAN destination port sees one pre-rewrite copy of the stream, not eight copies. In addition, if for any reason one or more of those ports drops the packets on egress (for example, due to congestion), the packets may still reach the SPAN destination port. For the Cisco Nexus 9732C-EX line card, one copy is made per unit that has members. For port-channel sources, the Layer 2 member that will SPAN is the first port-channel member.

  • The flows for post-routed unknown unicast flooded packets are in the SPAN session, even if the SPAN session is configured to not monitor the ports on which this flow is forwarded. This limitation applies to Network Forwarding Engine (NFE) and NFE2-enabled EOR switches and SPAN sessions that have Tx port sources.

  • Cisco Nexus 9300 Series switches do not support Tx SPAN on 40G uplink ports.


    Note

    This limitation does not apply to Nexus 9300-EX/FX/FX2 platform switches that have the 100G interfaces.

  • Prior to Cisco NX-OS Release 7.0(3)I5(2), Tx SPAN is not supported for multicast, unknown multicast, and broadcast traffic when the SPAN source port(s) and the SPAN destination port are on different forwarding engine slices. Beginning with Cisco NX-OS Release 7.0(3)I5(2), SPAN Tx broadcast, and SPAN Tx multicast are supported for Layer 2 port and port-channel sources across slices on Cisco Nexus 9300-EX platform switches and the Cisco Nexus 9732C-EX line card but only when IGMP snooping is disabled. (Otherwise, the slice limitation still applies.) These features are not supported for Layer 3 port sources, FEX ports (with unicast or multicast traffic), and VLAN sources.

The following guidelines and limitations apply to ingress (Rx) SPAN:

  • A SPAN copy of Cisco Nexus 9300 Series switch 40G uplink interfaces will miss the dot1q information when spanned in the Rx direction.


    Note

    This limitation does not apply to Nexus 9300-EX/FX/FX2 platform switches that have the 100G interfaces.

  • VLAN sources are spanned only in the Rx direction. This limitation does not apply to the following switch platforms which support VLAN spanning in both directions:

    • Cisco Nexus 9300-EX platform switches

    • Cisco Nexus 9300-FX platform switches

    • Cisco Nexus 9504, 9508, and 9516 switches with the 97160YC-EX line card.

    • Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • If a VLAN source is configured as both directions in one session and the physical interface source is configured in two other sessions, Rx SPAN is not supported for the physical interface source session. This limitation applies to the Cisco Nexus 97160YC-EX line card.

  • For Cisco Nexus 9200 platform switches, Rx SPAN is not supported for multicast without a forwarding interface on the same slice as the SPAN destination port.

  • Session filtering functionality (VLAN or ACL filters) is supported only for Rx sources. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

The following guidelines and limitations apply to FEX ports:

  • The FEX NIF interfaces or port-channels cannot be used as a SPAN source or SPAN destination. If the FEX NIF interfaces or port-channels are specified as a SPAN source or SPAN destination, the software displays an unsupported error.

  • Cisco Nexus 9300 and 9500 platform switches (excluding Cisco Nexus 9300-EX platform switches) support FEX ports as SPAN sources in the ingress direction for all traffic and in the egress direction only for known Layer 2 unicast traffic flows through the switch and FEX. Routed traffic might not be seen on FEX HIF egress SPAN.

  • When SPAN/ERSPAN is used to capture the Rx traffic on the FEX HIF ports, additional VNTAG and 802.1q tags are present in the captured traffic.

  • In Cisco NX-OS Release 7.0(3)I7(2) and earlier, SPAN traffic does not appear on the SPAN destination interface if the SPAN source is a FEX host interface that allows a VLAN extended over VXLAN. This limitation is resolved in Cisco NX-OS Release 7.0(3)I7(3) and later releases.

  • VLAN and ACL filters are not supported for FEX ports.

  • If the sources used in bidirectional SPAN sessions are from the same FEX, the hardware resources are limited to two SPAN sessions.

The following guidelines and limitations apply to Cisco Nexus 9200 and 9300-EX Series switches:

  • Cisco Nexus 9300-EX platform switches support FEX ports as SPAN sources only in the ingress direction.

  • On Cisco Nexus 9300-EX/FX/FX2 platform switches, and Cisco Nexus 9500 platform switches with EX/FX modules, SPAN and sFlow cannot both be enabled simultaneously. If one is active, the other cannot be enabled. However, on the Cisco Nexus 9300-EX/FX/FX2 and the Cisco Nexus 9500 platform switches with EX modules, both Netflow and SPAN can both be enabled simultaneously, providing a viable alternative to using Sflow and SPAN.

  • UDF-based SPAN is supported beginning with Cisco NX-OS Release 7.0(3)I4(1).

  • Tx SPAN for multicast, unknown multicast, and broadcast traffic are not supported on the Cisco Nexus 9200 platform switches.

  • When multiple egress ports on the same slice are congested by egressing SPAN traffic, those egress ports will not get the line rate.

  • Using the ACL filter to span subinterface traffic on the parent interface is not supported.

  • The CPU SPAN source can be added only for the Rx direction (SPAN packets coming from the CPU).

  • Multiple ACL filters are not supported on the same source.

  • SPAN packets to the CPU are rate limited and are dropped in the inband path. You can change the rate limit using the hardware rate-limiter span command. You can analyze SPAN copies on the supervisor using the ethanalyzer local interface inband mirror detail command.

The following guidelines and limitations apply to SPAN truncation:

  • Truncation is supported only for Cisco Nexus 9300-EX and 9300-FX platform switches, beginning with Cisco NX-OS Release 7.0(3)I7(1).

  • Truncation is supported only for local and SPAN source sessions. It is not supported for SPAN destination sessions.

  • Configuring MTU on a SPAN session truncates all of the packets egressing on the SPAN destination (for that session) to the MTU value specified.

  • The cyclic redundancy check (CRC) is recalculated for the truncated packet.

  • The bytes specified are retained starting from the header of the packets. The rest are truncated if the packet is longer than the MTU.

SPAN Limitations for the Cisco Nexus 3000 Platform Switches

The following guidelines and limitations apply only the Nexus 3000 Series switches running Cisco Nexus 9000 code:

  • The Cisco Nexus 3232C and 3264Q switches do not support SPAN on CPU as destination.

SPAN Limitations for the Cisco Nexus 9200 Platform Switches (excluding 9232E-B1)


Note

For scale information, see the release-specific Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

The following guidelines and limitations apply only the Cisco Nexus 9200 platform switches:

  • For Cisco Nexus 9200 platform switches, Rx SPAN is not supported for multicast without a forwarding interface on the same slice as the SPAN destination port.

  • Tx SPAN for multicast, unknown multicast, and broadcast traffic are not supported on the Cisco Nexus 9200 platform switches.

  • Tx SPAN of CPU-generated packets is not supported on Cisco Nexus 9200 platform switches.

  • UDF-based SPAN is supported on the Cisco Nexus 9200 platform switches.

  • The Cisco Nexus 9200 platform switches do not support Multiple ACL filters on the same source.

  • VLAN Tx SPAN is supported on the Cisco Nexus 9200 platform switches.

  • When multiple egress ports on the same slice are congested by egressing SPAN traffic, those egress ports will not get the line rate on the Cisco Nexus 9200 platform switches.

  • Using the ACL filter to span subinterface traffic on the parent interface is not supported on the Cisco Nexus 9200 platform switches.

  • On the Cisco Nexus 9200 platform switches, the CPU SPAN source can be added only for the Rx direction (SPAN packets coming from the CPU).

  • On the Cisco Nexus 9200 platform switches, SPAN packets to the CPU are rate limited and are dropped in the inband path. You can change the rate limit using the hardware rate-limiter span command. You can analyze SPAN copies on the supervisor using the ethanalyzer local interface inband mirror detail command.

SPAN Limitations for the Cisco Nexus 9300 Platform Switches


Note

For scale information, see the release-specific Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

The following guidelines and limitations apply only the Cisco Nexus 9300 platform switches:

  • The following filtering limitations apply to egress (Tx) SPAN on all Cisco Nexus 9300-EX/FX/FX2 platform switches:

    • ACL filtering is not supported (applies to both unicast and Broadcast, Unknown Unicast and Multicast (BUM) traffic)

    • VLAN filtering is supported, but only for unicast traffic

    • VLAN filtering is not supported for BUM traffic

  • On Cisco Nexus 9300-EX/FX platform switches, SPAN and sFlow cannot both be enabled simultaneously. If one is active, the other cannot be enabled. However, on Cisco Nexus 9300-EX/FX/FX2 platform switches, both NetFlow and SPAN can be enabled simultaneously, providing a viable alternative to using sFlow and SPAN.


    Note

    Cisco Nexus 9300-FX2 switches support sFlow and SPAN co-existence.

  • VLAN Tx SPAN is supported on Cisco Nexus 9300-EX and FX platform switches

  • Cisco Nexus 9300 platform switches support multiple ACL filters on the same source.

  • A single forwarding engine instance supports four SPAN sessions. For Cisco Nexus 9300 platform switches, if the first three sessions have bidirectional sources, the fourth session has hardware resources only for Rx sources.

  • Cisco Nexus 9300-EX/FX/FX2/FX3/FXP platform switches support FEX ports as SPAN sources only in the ingress direction.

  • Cisco Nexus 9300 platform switches (excluding Cisco Nexus 9300-EX/FX/FX2/FX3/FXP switches) support FEX ports as SPAN sources in the ingress direction for all traffic and in the egress direction only for known Layer 2 unicast traffic flows through the switch and FEX. Routed traffic might not be seen on FEX HIF egress SPAN.

  • Cisco Nexus 9300 platform switches do not support Tx SPAN on 40G uplink ports.


    Note

    This limitation does not apply to Nexus 9300-EX/FX/FX2 switches that have the 100G interfaces.

  • Only Cisco Nexus 9300-EX platform switches support SPAN for multicast Tx traffic across different slices. The slices must be on the same leaf spine engine (LSE).

  • For Tx interface SPAN with Layer 2 switch port and port-channel sources on Cisco Nexus 9300-EX/FX/FX2 platform switches, only one copy is made per receiver unit regardless of how many Layer 2 members are receiving the stream in the same VLAN. For example, if e1/1-8 are all Tx direction SPAN sources and all are joined to the same group, the SPAN destination port sees one pre-rewrite copy of the stream, not eight copies. In addition, if for any reason one or more of those ports drops the packets on egress (for example, due to congestion), the packets may still reach the SPAN destination port. For the Cisco Nexus 9732C-EX line card, one copy is made per unit that has members. For port-channel sources, the Layer 2 member that will SPAN is the first port-channel member.

  • SPAN Tx broadcast and SPAN Tx multicast are supported for Layer 2 port and port-channel sources across slices on Cisco Nexus 9300-EX/FX/FX2 platform switches, and the Cisco Nexus 9732C-EX line card, but only when IGMP snooping is disabled. (Otherwise, the slice limitation still applies.) These features are not supported for Layer 3 port sources, FEX ports (with unicast or multicast traffic), and VLAN sources.

  • For SPAN Tx multicast for Layer 2, SPAN copies are created independent of multicast replication. Due to this, multicast and SPAN packet have different values for VLAN tag, which is the ingress interface VLAN ID.

  • A SPAN copy of Cisco Nexus 9300 platform switch 40G uplink interfaces will miss the dot1q information when spanned in the Rx direction.


    Note

    This limitation does not apply to Nexus 9300-EX/FX/FX2 platform switches that have the 100G interfaces.

  • UDF-based SPAN is supported on the Cisco Nexus 9300-EX/FX/FX2 platform switches.

  • UDF-SPAN acl-filtering only supports source interface rx. This limitation applies to the following switches:

    • Cisco Nexus 9332PQ

    • Cisco Nexus 9372PX

    • Cisco Nexus 9372PX-E

    • Cisco Nexus 9372TX

    • Cisco Nexus 9372TX-E

    • Cisco Nexus 93120TX

  • The Cisco Nexus 9300-EX/FX/FX2 platform switches do not support Multiple ACL filters on the same source.

  • When multiple egress ports on the same slice are congested by egressing SPAN traffic, those egress ports will not get the line rate on the Cisco Nexus 9300-EX/FX/FX2 platform switches.

  • Using the ACL filter to span subinterface traffic on the parent interface is not supported on the Cisco Nexus 9300-EX/FX/FX2 platform switches.

  • On the Cisco Nexus 9300-EX/FX/FX2 platform switches, the CPU SPAN source can be added only for the Rx direction (SPAN packets coming from the CPU).

  • On the Cisco Nexus 9300-EX/FX/FX2 platform switches, SPAN packets to the CPU are rate limited and are dropped in the inband path. You can change the rate limit using the hardware rate-limiter span command. You can analyze SPAN copies on the supervisor using the ethanalyzer local interface inband mirror detail command.

SPAN Limitations for the Cisco Nexus 9500 Platform Switches


Note

For scale information, see the release-specific Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

The following guidelines and limitations apply only the Cisco Nexus 9500 platform switches:

  • The following filtering limitations apply to egress (Tx) SPAN on 9500 platform switches with EX or FX line cards:

    • ACL filtering is not supported (applies to both unicast and Broadcast, Unknown Unicast and Multicast (BUM) traffic)

    • VLAN filtering is supported, but only for unicast traffic

    • VLAN filtering is not supported for BUM traffic

  • FEX and SPAN port-channel destinations are not supported on the Cisco Nexus 9500 platform switches with EX or FX line cards.

  • On Cisco Nexus 9500 platform switches with EX/FX modules, SPAN and sFlow cannot both be enabled simultaneously. If one is active, the other cannot be enabled. However, on the Cisco Nexus 9500 platform switches with EX or FX line cards, NetFlow and SPAN can both be enabled simultaneously, providing a viable alternative to using sFlow and SPAN.

  • Cisco Nexus 9500 platform switches support VLAN Tx SPAN with the following line cards:

    • Cisco Nexus 97160YC-EX

    • Cisco Nexus 9732C-EX

    • Cisco Nexus 9732C-FX

    • Cisco Nexus 9736C-EX

    • Cisco Nexus 9736C-FX

    • Cisco Nexus 9736Q-FX

    • Cisco Nexus 9788TC-FX

  • Cisco Nexus 9500 platform switches support multiple ACL filters on the same source.

  • Tx SPAN of CPU-generated packets is not supported on Cisco Nexus 9500 platform switches with EX-based line cards.

  • TCAM carving is not required for SPAN/ERSPAN on the following line cards:

    • Cisco Nexus 9636C-R

    • Cisco Nexus 9636Q-R

    • Cisco Nexus 9636C-RX

    • Cisco Nexus 96136YC-R


    Note

    All other switches supporting SPAN/ERSPAN must use TCAM carving.

  • On the Cisco Nexus 9500 platform switches, depending on the SPAN source's forwarding engine instance mappings, a single forwarding engine instance may support four SPAN sessions. This guideline does not apply for Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • Multiple ACL filters are not supported on the same source.

  • Cisco Nexus 9500 platform switches support FEX ports as SPAN sources in the ingress direction for all traffic and in the egress direction only for known Layer 2 unicast traffic flows through the switch and FEX. Routed traffic might not be seen on FEX HIF egress SPAN.

  • SPAN does not support destinations on Cisco Nexus 9408PC-CFP2 line card ports.

  • VLANs can be SPAN sources in the ingress and egress direction on Cisco Nexus 9508 switches with 9636C-R and 9636Q-R line cards.

  • UDF-SPAN acl-filtering only supports source interface rx. This limitation applies to the following line cards:

    • Cisco Nexus 9564PX

    • Cisco Nexus 9464TX2

    • Cisco Nexus 9464TX

    • Cisco Nexus 9464TX2

    • Cisco Nexus 9564TX

    • Cisco Nexus 9464PX

    • Cisco Nexus 9536PQ

    • Cisco Nexus 9636PQ

    • Cisco Nexus 9432PQ

Default Settings for SPAN

The following table lists the default settings for SPAN parameters.

Parameters Default
SPAN sessions Created in the shut state

Configuring SPAN


Note
Cisco NX-OS commands for this feature may differ from those in Cisco IOS.

Configuring a SPAN Session

You can configure a SPAN session on the local device only. By default, SPAN sessions are created in the shut state.


Note
For bidirectional traditional sessions, you can configure the sessions without specifying the direction of the traffic.

Before you begin

You must configure the destination ports in access or trunk mode. For more information, see the Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide.

Procedure

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

interface ethernet slot/port

Example:

switch(config)# interface ethernet 2/5
switch(config-if)#

Enters interface configuration mode on the selected slot and port.

Step 3

switchport

Example:

switch(config-if)# switchport

Configures switchport parameters for the selected slot and port or range of ports.

Step 4

switchport monitor

Example:

switch(config-if)# switchport monitor

Configures the switchport interface as a SPAN destination.

Step 5

(Optional) Repeat Steps 2 through 4 to configure monitoring on additional SPAN destinations.

(Optional)

—

Step 6

no monitor session session-number

Example:

switch(config)# no monitor session 3

Clears the configuration of the specified SPAN session. The new session configuration is added to the existing session configuration.

Step 7

monitor session session-number [shut]

Example:

Example:

switch(config)# monitor session 3 shut
switch(config-monitor)#

Enters the monitor configuration mode. The new session configuration is added to the existing session configuration. By default, the session is created in the shut state, and the session is a local SPAN session. The optional keyword shut specifies a shut state for the selected session.

Step 8

description description

Example:

switch(config-monitor)# description my_span_session_3

Configures a description for the session. By default, no description is defined. The description can be up to 32 alphanumeric characters.

Step 9

source {interface type [rx | tx | both] | [rx]}

Example:

switch(config-monitor)# source interface ethernet 2/1-3, ethernet 3/1 rx

Example:

switch(config-monitor)# source interface port-channel 2

You can configure one or more sources, as either a series of comma-separated entries or a range of numbers.

You can specify the traffic direction to copy as ingress (rx), egress (tx), or both.

For a unidirectional session, the direction of the source must match the direction specified in the session.

Step 10

(Optional) filter access-group acl-filter

Example:

switch(config-monitor)# filter access-group ACL1
(Optional)

Associates an ACL with the SPAN session.

Step 11

destination interface type slot/port

Example:

switch(config-monitor)# destination interface ethernet 2/5

Example:

switch(config-monitor)# destination interface sup-eth 0

Configures a destination for copied source packets.

Note

 
The SPAN destination port must be either an access port or a trunk port.

Note

 
You must enable monitor mode on the destination port.

You can configure the CPU as the SPAN destination for Cisco Nexus 9200 Series switches, beginning with Cisco NX-OS Release 7.0(3)I4(1), and Cisco Nexus 9300-EX Series switches, beginning with Cisco NX-OS Release 7.0(3)I4(2). To do so, enter sup-eth 0 for the interface type.

Step 12

no shut

Example:

switch(config-monitor)# no shut

Enables the SPAN session. By default, the session is created in the shut state.

Step 13

(Optional) show monitor session {all | session-number | range session-range} [brief]

Example:

switch(config-monitor)# show monitor session 3
(Optional)

Displays the SPAN configuration.

Step 14

(Optional) copy running-config startup-config

Example:

switch(config)# copy running-config startup-config
(Optional)

Copies the running configuration to the startup configuration.

Configuring UDF-Based SPAN

You can configure the device to match on user-defined fields (UDFs) of the outer or inner packet fields (header or payload) and to send the matching packets to the SPAN destination. Doing so can help you to analyze and isolate packet drops in the network.

Before you begin

Make sure that the appropriate TCAM region (SPAN) has been configured using the hardware access-list tcam region command to provide enough free space to enable UDF-based SPAN. For information, see the "Configuring ACL TCAM Region Sizes" section in the Cisco Nexus 9000 Series NX-OS Security Configuration Guide.

Procedure

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

udf udf-name offset-base offset length

Example:

switch(config)# udf udf-x packet-start 12 1
switch(config)# udf udf-y header outer l3 20 2

Defines the UDF as follows:

  • udf-name—Specifies the name of the UDF. You can enter up to 16 alphanumeric characters for the name.

  • offset-base—Specifies the UDF offset base as follows, where header is the packet header to consider for the offset: packet-start | header {outer | inner {l3 | l4}} .

  • offset—Specifies the number of bytes offset from the offset base. To match the first byte from the offset base (Layer 3/Layer 4 header), configure the offset as 0.

  • length—Specifies the number of bytes from the offset. Only 1 or 2 bytes are supported. To match additional bytes, you must define multiple UDFs.

You can define multiple UDFs, but Cisco recommends defining only required UDFs.

Step 3

hardware access-list tcam region span qualify udf udf-names

Example:

switch(config)# hardware access-list tcam region span qualify udf udf-x udf-y

Attaches the UDFs to one of the following TCAM regions:

  • SPAN —Applies to Layer 2 & Layer 3 ports.

You can attach up to 2 UDFs to a TCAM region.

Note

 

Make sure enough free space is available; otherwise, this command will be rejected. If necessary, you can reduce the TCAM space from unused regions and then re-enter this command. For more information, see the "Configuring ACL TCAM Region Sizes" section in the Cisco Nexus 9000 Series NX-OS Security Configuration Guide.

Note

 

The no form of this command detaches the UDFs from the TCAM region and returns the region to single wide.

Step 4

copy running-config startup-config

Example:

switch(config)# copy running-config startup-config

Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

Step 5

reload

Example:

switch(config)# reload

Reloads the device.

Note

 

Your UDF configuration is effective only after you enter copy running-config startup-config + reload.

Step 6

ip access-list span-acl

Example:

switch(config)# ip access-list span-acl-udf-only
switch(config-acl)#

Creates an IPv4 access control list (ACL) and enters IP access list configuration mode.

Step 7

Enter one of the following commands:

  • permit udf udf-name value mask
  • permit ip source destination udf udf-name value mask

Example:

switch(config-acl)# permit udf udf-x 0x40 0xF0 udf-y 0x1001 0xF00F

Example:

switch(config-acl)# permit ip 10.0.0./24 any udf udf-x 0x02 0x0F udf-y 0x1001 0xF00F

Configures the ACL to match only on UDFs (example 1) or to match on UDFs along with the current access control entries (ACEs) for the outer packet fields (example 2).

A single ACL can have ACEs with and without UDFs together. Each ACE can have different UDF fields to match, or all ACEs can match for the same list of UDFs.

Step 8

(Optional) copy running-config startup-config

Example:

switch(config)# copy running-config startup-config
 (Optional)

Copies the running configuration to the startup configuration.

Configuring SPAN Truncation

You can configure truncation for local and SPAN source sessions only.

Procedure

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

source interface type slot/port [rx | tx | both]

Example:

switch(config-monitor)# source interface ethernet 1/5 both

Configures the source interface.

Step 3

mtu size

Example:

switch(config-monitor)# mtu 512

Configures the MTU size for truncation. Any SPAN packet that is larger than the configured MTU size is truncated to the configured size. The MTU size range is 320 to 1518 bytes for Cisco Nexus 9300-EX platform switches and 64 to 1518 bytes for Cisco Nexus 9300-FX platform switches.

Step 4

destination interface type slot/port

Example:

switch(config-monitor)# destination interface Ethernet 1/39

Configures the Ethernet SPAN destination port.

Step 5

no shut

Example:

switch(config-monitor)# no shut

Enables the SPAN session. By default, the session is created in the shut state.

Step 6

(Optional) show monitor session session

Example:

switch(config-monitor)# show monitor session 5
(Optional)

Displays the SPAN configuration.

Step 7

copy running-config startup-config

Example:

switch(config-monitor)# copy running-config startup-config

Copies the running configuration to the startup configuration.

Configuring SPAN for Multicast Tx Traffic Across Different LSE Slices

Beginning with Cisco NX-OS Release 7.0(3)I7(1), you can configure SPAN for multicast Tx traffic across different leaf spine engine (LSE) slices on Cisco Nexus 9300-EX platform switches.

Procedure

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

[no] hardware multicast global-tx-span

Example:

switch(config)# hardware multicast global-tx-span

Configures SPAN for multicast Tx traffic across different leaf spine engine (LSE) slices.

Step 3

copy running-config startup-config

Example:

switch(config)# copy running-config startup-config

Copies the running configuration to the startup configuration.

Step 4

reload

Example:

switch(config)# reload

Reloads the device.

Shutting Down or Resuming a SPAN Session

You can shut down SPAN sessions to discontinue the copying of packets from sources to destinations. You can shut down one session in order to free hardware resources to enable another session. By default, SPAN sessions are created in the shut state.

You can resume (enable) SPAN sessions to resume the copying of packets from sources to destinations. In order to enable a SPAN session that is already enabled but operationally down, you must first shut it down and then enable it.

You can configure the shut and enabled SPAN session states with either a global or monitor configuration mode command.

Procedure

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

[no] monitor session {session-range | all} shut

Example:

switch(config)# monitor session 3 shut

Shuts down the specified SPAN sessions. By default, sessions are created in the shut state.

The no form of the command resumes (enables) the specified SPAN sessions. By default, sessions are created in the shut state.

Note

 
If a monitor session is enabled but its operational status is down, to enable the session, you must first specify the monitor session shut command followed by the no monitor session shut command.

Step 3

monitor session session-number

Example:

switch(config)# monitor session 3
switch(config-monitor)#

Enters the monitor configuration mode. The new session configuration is added to the existing session configuration.

Step 4

[no] shut

Example:

switch(config-monitor)# shut

Shuts down the SPAN session. By default, the session is created in the shut state.

The no form of the command enables the SPAN session. By default, the session is created in the shut state.

Step 5

(Optional) show monitor

Example:

switch(config-monitor)# show monitor
(Optional)

Displays the status of SPAN sessions.

Step 6

(Optional) copy running-config startup-config

Example:

switch(config)# copy running-config startup-config
(Optional)

Copies the running configuration to the startup configuration.

Verifying the SPAN Configuration

To display the SPAN configuration, perform one of the following tasks:

Command Purpose
show monitor session {all | session-number | range session-range} [brief] Displays the SPAN session configuration.

Configuration Examples for SPAN

Configuration Example for a SPAN Session

To configure a SPAN session, follow these steps:

Procedure

Step 1

Configure destination ports in access mode and enable SPAN monitoring.

Example:

switch# configure terminal
switch(config)# interface ethernet 2/5
switch(config-if)# switchport
switch(config-if)# switchport monitor
switch(config-if)# no shut
switch(config-if)# exit
switch(config)#

Step 2

Configure a SPAN session.

Example:

switch(config)# no monitor session 3
switch(config)# monitor session 3
switch(config-monitor)# source interface ethernet 2/1-3, ethernet 3/1 rx
switch(config-monitor)# source interface port-channel 2
switch(config-monitor)# destination interface ethernet 2/5
switch(config-monitor)# no shut
switch(config-monitor)# exit
switch(config)# show monitor session 3
switch(config)# copy running-config startup-config

Configuration Example for a Unidirectional SPAN Session

To configure a unidirectional SPAN session, follow these steps:

Procedure

Step 1

Configure destination ports in access mode and enable SPAN monitoring.

Example:

switch# configure terminal
switch(config)# interface ethernet 2/5
switch(config-if)# switchport
switch(config-if)# switchport monitor
switch(config-if)# no shut
switch(config-if)# exit
switch(config)#

Step 2

Configure a SPAN session.

Example:

switch(config)# no monitor session 3
switch(config)# monitor session 3 rx
switch(config-monitor)# source interface ethernet 2/1-3, ethernet 3/1
switch(config-monitor)# destination interface ethernet 2/5
switch(config-monitor)# no shut
switch(config-monitor)# exit
switch(config)# show monitor session 3
switch(config)# copy running-config startup-config

Configuration Example for a SPAN ACL

This example shows how to configure a SPAN ACL: 

switch# configure terminal
switch(config)# ip access-list match_11_pkts
switch(config-acl)# permit ip 11.0.0.0 0.255.255.255 any
switch(config-acl)# exit
switch(config)# ip access-list match_12_pkts
switch(config-acl)# permit ip 12.0.0.0 0.255.255.255 any
switch(config-acl)# exit
switch(config-access-map)# match ip address match_11_pkts
switch(config-access-map)# action forward
switch(config-access-map)# exit
switch(config-access-map)# match ip address match_12_pkts
switch(config-access-map)# action forward
switch(config-access-map)# exit
switch(config)# monitor session 1
switch(config-erspan-src)# filter access_group span_filter

Configuration Examples for UDF-Based SPAN

This example shows how to configure UDF-based SPAN to match on the inner TCP flags of an encapsulated IP-in-IP packet using the following match criteria:

  • Outer source IP address: 10.0.0.2

  • Inner TCP flags: Urgent TCP flag is set

  • Bytes: Eth Hdr (14) + Outer IP (20) + Inner IP (20) + Inner TCP (20, but TCP flags at 13th byte)

  • Offset from packet-start: 14 + 20 + 20 + 13 = 67

  • UDF match value: 0x20

  • UDF mask: 0xFF 

udf udf_tcpflags packet-start 67 1
hardware access-list tcam region span qualify udf udf_tcpflags
copy running-config startup-config
reload
ip access-list acl-udf
	 permit ip 10.0.0.2/32 any udf udf_tcpflags 0x20 0xff
monitor session 1
  source interface Ethernet 1/1
  filter access-group acl-udf

This example shows how to configure UDF-based SPAN to match regular IP packets with a packet signature (DEADBEEF) at 6 bytes after a Layer 4 header start using the following match criteria:

  • Outer source IP address: 10.0.0.2

  • Inner TCP flags: Urgent TCP flag is set

  • Bytes: Eth Hdr (14) + IP (20) + TCP (20) + Payload: 112233445566DEADBEEF7788

  • Offset from Layer 4 header start: 20 + 6 = 26

  • UDF match value: 0xDEADBEEF (split into two-byte chunks and two UDFs)

  • UDF mask: 0xFFFFFFFF 

udf udf_pktsig_msb header outer l4 26 2
udf udf_pktsig_lsb header outer l4 28 2
hardware access-list tcam region span qualify udf udf_pktsig_msb udf_pktsig_lsb
copy running-config startup-config
reload
ip access-list acl-udf-pktsig
		permit udf udf_pktsig_msb 0xDEAD 0xFFFF udf udf_pktsig_lsb 0xBEEF 0xFFFF 
monitor session 1
		source interface Ethernet 1/1
		filter access-group acl-udf-pktsig

Configuration Example for SPAN Truncation

This example shows how to configure SPAN truncation for use with MPLS stripping: 

mpls strip
ip access-list mpls
  statistics per-entry
  20 permit ip any any redirect Ethernet1/5

monitor session 1
  source interface Ethernet1/5 tx
  mtu 64
  destination interface Ethernet1/6
  no shut

Configuration Examples for Multicast Tx SPAN Across LSE Slices

This example shows how to configure multicast Tx SPAN across LSE slices for Cisco Nexus 9300-EX platform switches. It also shows sample output before and after multicast Tx SPAN is configured.

Before Multicast Tx SPAN Is Configured

switch# show interface eth1/15-16, ethernet 1/27 counters

-----------------------------------------
Port           InOctets    InUcastPkts
-----------------------------------------
Eth1/15          580928              0
Eth1/16             239              0
Eth1/27               0              0

-----------------------------------------
Port        InMcastPkts    InBcastPkts
-----------------------------------------
Eth1/15            9077              0
Eth1/16               1              0
Eth1/27               0              0

-----------------------------------------
Port          OutOctets   OutUcastPkts
-----------------------------------------
Eth1/15             453              0
Eth1/16          581317              0
Eth1/27               0              0

-----------------------------------------
Port       OutMcastPkts   OutBcastPkts
-----------------------------------------
Eth1/15               4              0
Eth1/16            9080              0
Eth1/27               0              0

Configuring Multicast Tx SPAN

switch(config)# hardware multicast global-tx-span
Warning: Global Tx SPAN setting changed, please save config and reload
switch(config)# copy running-config start-up config
[########################################] 100%
Copy complete.
switch(config)# reload
This command will reboot the system. (y/n)?  [n] y

After Multicast Tx SPAN Is Configured

switch# show interface eth1/15-16, eth1/27 counters

-----------------------------------------
Port           InOctets    InUcastPkts
-----------------------------------------
Eth1/15          392576              0
Eth1/16               0              0
Eth1/27               0              0
-----------------------------------------
Port        InMcastPkts    InBcastPkts
-----------------------------------------
Eth1/15            6134              0
Eth1/16               0              0
Eth1/27               0              0

-----------------------------------------
Port          OutOctets   OutUcastPkts
-----------------------------------------
Eth1/15               0              0
Eth1/16          392644              0
Eth1/27          417112              0

-----------------------------------------
Port       OutMcastPkts   OutBcastPkts
-----------------------------------------
Eth1/15               0              0
Eth1/16            6135              0
Eth1/27            6134              0

Additional References

Related Documents

Related Topic Document Title

ACL TCAM regions

Cisco Nexus 9000 Series NX-OS Security Configuration Guide

FEX Cisco Nexus 2000 Series NX-OS Fabric Extender Software Configuration Guide for Cisco Nexus 9000 Series Switches