Overview

Supported Platforms

Starting with Cisco NX-OS release 7.0(3)I7(1), use the Nexus Switch Platform Support Matrix to know from which Cisco NX-OS releases various Cisco Nexus 9000 and 3000 switches support a selected feature.

About Interfaces

Cisco NX-OS supports multiple configuration parameters for each of the interface types supported. Most of these parameters are covered in this guide but some are described in other documents.

The following table shows where to get further information on the parameters you can configure for an interface.

Table 1. Interface Parameters

Feature

Parameters

Further Information

Basic parameters

Description, duplex, error disable, flow control, MTU, beacon

“Configuring Basic Interface Parameters”

Layer 3

Medium, IPv4 and IPv6 addresses

“Configuring Layer 3 Interfaces”

Layer 3

Bandwidth, delay, IP routing, VRFs

Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide

Cisco Nexus 9000 Series NX-OS Multicast Routing Configuration Guide

Port Channels

Channel group, LACP

“Configuring Port Channels”

Security

EOU

Cisco Nexus 9000 Series NX-OS Security Configuration Guide

Ethernet Interfaces

  • Ethernet interfaces include routed ports.

Cisco Nexus N9K-C9364C-GX and N9K-C93600CD-GX Guidelines and Limitations

Cisco Nexus N9K-X9400-16W Guidelines and Limitations

Beginning with Cisco Nexus NX-OS Release 10.5(1), the following guidelines and limitations are applicable:

Access Ports

An access port carries traffic for one VLAN. This type of port is a Layer 2 interface only.

For more information on access ports, see the “Information About Access and Trunk Interfaces” section.

Routed Ports

A routed port is a physical port that can route IP traffic to another device. A routed port is a Layer 3 interface only.

For more information on routed ports, see the “Routed Interfaces” section.

Management Interface

You can use the management Ethernet interface to connect the device to a network for remote management using a Telnet client, the Simple Network Management Protocol (SNMP), or other management agents. The management port (mgmt0) is autosensing and operates in full-duplex mode at a speed of 10/100/1000 Mb/s.

For more information on the management interface, see the Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide.

Port-Channel Interfaces

A port channel is a logical interface that is an aggregation of multiple physical interfaces. You can bundle up to 32 individual links (physical ports) into a port channel to improve bandwidth and redundancy. For more information about port-channel interfaces, see the “Configuring Port Channels” section.

Subinterfaces

You can create virtual subinterfaces using a parent interface configured as a Layer 3 interface. A parent interface can be either a physical port or a port-channel. A parent interface can be a physical port. Subinterfaces divide the parent interface into two or more virtual interfaces on which you can assign unique Layer 3 parameters such as IP addresses and dynamic routing protocols.

Loopback Interfaces

A loopback interface is a virtual interface with a single endpoint that is always up. Any packet that is transmitted over a virtual loopback interface is immediately received by that interface. Loopback interfaces emulate a physical interface. For more information about subinterfaces, see the “Loopback Interfaces” section.

Breakout Interfaces

Cisco NX-OS supports the breakout of a high bandwidth interface into one or more low bandwidth interfaces at the module level or at the per-port level.

Module Level Breakout

For module level breakout, the interface breakout command splits the high bandwidth 40G interface of a module into four 10G interfaces. The module is reloaded and the configuration for the interface is removed when the command is executed.

The following is an example of the command: 

switch# configure terminal
switch(config)# interface breakout module 1
Module will be reloaded. Are you sure you want to continue(yes/no)? yes

The no interface breakout module module_number command undoes the breakout configuration. It puts all interfaces of the module in 40G mode and deletes the configuration for the previous 10G interfaces.

Dynamic Breakout (Per-port Level Breakout)

For dynamic breakout (per-port level breakout), the interface breakout command splits high bandwidth 40G ports into four 10G breakout ports and 100G ports into four 25G breakout ports. The breakout ports are identified as Ethernet <slot>/<front-panel-port>/<breakout-port> . For example, per-port level breakout ports can be identified as Ethernet 1/2/1, Ethernet 1/2/2, Ethernet 1/2/3, and Ethernet 1/2/4.

When one or more 40G interfaces of the module are broken-out at the per-port level, the configuration for the interfaces are removed when the command is executed.


Note

A per-port level breakout does not require the module to be reloaded.

The following is an example of configuring a breakout port: 


switch(config)# interface breakout module 1 port 1 map 10g-4x
switch(config)#

The following is an example of configuring a multiple breakout ports: 


switch(config)# interface breakout module 1 port 1-4 map 10g-4x
switch(config)#

The following is an example of configuring 40G and 10G interfaces mixed together.


switch(config-if)# show int eth1/49 transceiver 
Ethernet1/49
transceiver is present
type is QSFP-40G-SR-BD
name is CISCO-AVAGO 
part number is AFBR-79EBPZ-CS2 
revision is 01

switch(config-if)# show int eth1/52 transceiver
Ethernet1/52
transceiver is present
type is QSFP-Cazadero
name is CISCO-DNI 
part number is CAZADERO-R 
revision is 03
nominal bitrate is 10000 MBit/sec per channel

switch(config-if)# show int eth1/53 transceiver 
Ethernet1/53
transceiver is present
type is QSFP-Cazadero
name is CISCO-DNI 
part number is CAZADERO-R 
revision is 03
nominal bitrate is 10000 MBit/sec per channel

switch(config)# interface breakout module 1 port 52-53 map 10g-4x 

switch(config-if)# show int br | i up
mgmt0 -- up 10.122.160.192 100 1500
Eth1/49 -- eth routed up none 40G(D) — << Running 40G
Eth1/50 -- eth routed up none 40G(D) --
Eth1/52/1 -- eth routed up none 10G(D) — << Broken out to 10G 
Eth1/53/1 -- eth routed up none 10G(D) -- << Broken out to 10G

You can undo the breakout ports with the no interface breakout command.

The following is an example to undo the breakout ports: 


switch(config)# no interface breakout module 1 port 1 map 10g-4x
switch(config)#

About the Lane Selector

The lane selector is a push button switch and 4 LEDs located on the Cisco Nexus switch (left side of front panel, labeled "LS"). The push button switch and LEDs are used to indicate the status of the ports. The lane selector is supported on Cisco Nexus Series 9000 series switches and the Cisco Nexus 3164 and 3232 switches.

By default, the LEDs indicate the link/activity status of a 1 x 40G configuration. When the ports are configured as 4 x 10G, you can access the link status of each individual 10G port with the lane selector.

By pressing the lane selector push button, the port LED shows the selected lane’s link/activity status. The 1st time the push button is pressed, the first LED displays the status of the first port. Pressing the push button a 2nd time displays the status of the second port, and so on. You can display the status of each of the four ports by pressing the push button in this manner.

For example, if port 60 is configured as 4 x 10G, pressing the lane selector push button once displays the link status of 60/1/1. Pressing the push button a second time displays the link status of 60/1/2.

When you press the push button after displaying the status of the last port, all four of the LEDs should extinguish to indicate that the lane selector has returned to display the status for the default 1 x 40G configuration.


Note

A 10G breakout port's LED blinks when the beacon feature has been configured for it.


Note

When a port is configured to be in 10G breakout mode and no lane is selected, the 40G port's LED illuminates as green even though only one of the 10G breakout ports is up.

Support for Breakout Interfaces

The table provides detailed information of the supported or not supported breakout modes. For more information, see Cisco Nexus Data Sheets.

Table 2. Breakout Modes Support Matrix

Switches

4x10G

4x25G

2x50G

2x100G

2x200G

4x50G

4x100G

N9K-X9636C-RX

Yes

Yes

Yes

No

No

No

No

N9K-X9636C-R

Yes

Yes

Yes

No

No

No

No

N9K-X9636Q-R

Yes

No

No

No

No

No

No

N9K-X96136YC-R

No

No

No

No

No

No

No

N3K-C3636C-R

Yes

Yes

Yes

No

No

No

No

N3K-C36180YC-R

Yes

Yes

Yes

No

No

No

No

N9K-93108TC-EX

Yes

Yes

Yes

No

No

No

No

N9K-93180YC-EX

Yes

Yes

Yes

No

No

No

No

N9K-93108TC-FX

Yes

Yes

Yes

No

No

No

No

N9K-93180YC-FX

Yes

Yes

Yes

No

No

No

No

N9K-9348GC-FXP

Yes

Yes

Yes

No

No

No

No

N9K-X9732C-EX

Yes

Yes

Yes

No

No

No

No

N9K-X9736C-EX

Yes

Yes

Yes

No

No

No

No

N9K-X9732C-EXM

Yes

Yes

Yes

No

No

No

No

N9K-X9736C-FX

Yes

Yes

Yes

No

No

No

No

N9K-X9736Q-FX

Yes

No

No

No

No

No

No

N9K-X9788TC-FX

Yes

Yes

Yes

No

No

No

No

N9K-X9732C-FX

Yes

Yes

Yes

No

No

No

No

N9K-C9348GC-FXP

Yes

Yes

Yes

No

No

No

No

N9K-C9336C-FX2

Yes

Yes

Yes

No

No

No

No

N9K-C93216TC-FX2

Yes

Yes

Yes

No

No

No

No

N9K-C93360YC-FX2

Yes

Yes

Yes

No

No

No

No

N9K-C9364C-GX

Yes

Yes

Yes

No

No

No

No

N9K-C9316D-GX

Yes

Yes

Yes

Yes

Yes

Yes

Yes

N9K-C93600CD-GX

Yes

Yes

Yes

Yes

Yes

Yes

Yes

N9K-X9716D-GX

Yes

Yes

Yes

Yes

Yes

Yes

Yes

N9K-C9332D-GX2B

Yes

Yes

Yes

Yes

Yes

Yes

Yes

N9K-C9348D-GX2A

Yes

Yes

Yes

Yes

Yes

Yes

Yes

N9K-X9400-16W

Yes

Yes

Yes

Yes

No

Yes

No

N9K-X9400-8D

Yes

Yes

Yes

Yes

Yes

Yes

Yes

N9K-X98900CD-A

Yes

Yes

No

Yes

Yes

Yes

Yes
Guidelines and Limitations for Breakout

  • Cisco Nexus 9516 switch does not support breakout on Modules 8 to 16.

  • Beginning with Cisco NX-OS Release 7.0(3)F2(1), the 36-port 100-Gigabit Ethernet QSFP28 line cards (N9K-X9636C-R) and 36-port 40-Gigabit Ethernet QSFP+ line cards (N9K-X9636Q-R) support breakout of 4x10-Gigabit.

Caveats

  • As of Cisco NX-OS Release 7.0(3)I7(2), manual breakout of QSA ports is not supported.

    Manual breakout is supported on the following platforms because auto-breakout does not happen successfully on them—N9K-C93128TX, N9K-9332, N9K-C9396PX, N9K-C9396TX, N9K-C9372PX, N9K-C9372TX, N9K-C9332PQ, N9K-C93120TX, N9K-9432PQ, N9K-9536PQ, N9K-9636PQ, N9K-X9632PC-QSFP100, N9K-X9432C-S, N3K-C3132Q-V, N3K-C3164Q, N3K-C3132C, N3K-C3232C, N3K-C3264Q, N3K-C3264C, N3K-3064Q, N3K-3016, N3K-3172.

    You need to perform manual breakout using "interface breakout module <module number> port <port range> map <breakout mapping>" command.

  • When a break-out port is configured as a part of a port-channel, you need to apply the configuration twice (after write-erase/reload), to ensure the effectiveness of the port-channel.

  • When you upgrade a Cisco Nexus 9000 device to Cisco NX-OS release 7.0(3)I7(2) or later, if a QSFP port is configured with the manual breakout command and is using a QSA, the configuration of the interface is no longer supported and will need to be removed. To restore the configuration, you must manually configure the breakout config on the device.

    This behavior is not applicable to the following platforms—N9K-C93128TX, N9K-9332, N9K-C9396PX, N9K-C9396TX, N9K-C9372PX, N9K-C9372TX, N9K-C9332PQ, N9K-9432PQ, N9K-9536PQ, N9K-9636PQ, N9K-X9632PC-QSFP100, N9K-X9432C-S, N3K-C3132Q-V, N3K-C3164Q, N3K-C3132C, N3K-C3232C, N3K-C3264Q, N3K-C3264C, N3K-3064Q, N3K-3016, N3K-3172—because manual breakout is supported on these platforms.

  • Cisco Nexus 9000 Series switches have 40G ports. When you breakout one of the 40G ports into 4x10G ports with a QSFP breakout cable, not all of the subinterfaces can be added to a port channel. The following error message is displayed: 

    
switch# channel-group 99 mode active
command failed: port not compatible [Buffer boost]

    Note

    As a workaround, configure no buffer-boost on all of the subinterfaces. This enables the channel-group configuration to go through.

    Using the force keyword on the port-channel does not enable all the interfaces to be added to the port-channel even though the error message suggests using the keyword.

  • Beginning with Cisco NX-OS Release 7.0(3)I7(3) you see two additional options to configure FEC such as rs-cons16 and rs-ieee as per IEEE standards.

  • Beginning with Cisco NX-OS Release 7.0(3)I7(7) you can display the admin and oper status of FEC interface information with the show interface fec command.

    Example: 
    
switch# show interface fec
---------------------------------------------------------------------------------------------------
Name		Ifindex	Admin-fec 	Oper-fec    Status 	Speed 	Type
---------------------------------------------------------------------------------------------------
Eth1/1  	0x1a000000	auto		auto	connected     10G	 SFP-H10GB-AOC2M
Eth1/2  	0x1a000200   			Rs-fec    notconneced    auto	QSFP-100G-AOC3M
Eth1/3/1	0x38014000	auto		auto	disabled	auto	QSFP-H40G-AOC3M
Eth1/3/2	0x38015000	auto		auto	disabled	auto	QSFP-H40G-AOC3M
Eth1/3/3	0x38016000	auto		auto	disabled	auto	QSFP-H40G-AOC3M
Eth1/3/4	0x38017000	auto		auto	disabled	auto	QSFP-H40G-AOC3M

Note

Auto-FEC is not supported in Cisco NX-OS Release 7.0(3)I7(x)
High Bandwidth Interfaces

The breakout of high bandwidth interfaces (module level or per-port level) are supported only on:

  • The X9636PQ, X9432PQ, X9536PQ, and X9732C-EX line cards on a Cisco Nexus 9500 Series switch.

  • The Cisco Nexus 9332PQ switch.

  • The Cisco Nexus 3164Q switch.

Cisco Nexus C92160YC Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C92160YC switch provides two different modes of operation:

  • Mode 1: 48 * 10G/25G + 4 * 40G + 2 * 100G (Default configuration)

    • Hardware profile portmode 48x25G + 2x100G + 4x40G

    • Breakout is supported in 2 * 100G ports

  • Mode 2: 48 * 10G/25G + 4 * 100G

    • Hardware profile portmode 48x25G + 4x100G

    • Breakout is supported on 3 * 100G ports (Ports 50,51 & 52).

Use the show running-config | grep portmode command to display the current operation mode.

Example: 


switch(config-if-range)# show running-config | grep portmode

hardware profile portmode 48x25G+2x100G+4x40G

See the installation guide for the Cisco Nexus C92160YC switches for more information. ( Install and Upgrade Guides for Cisco Nexus 9000 Series Switches )

With the Cisco Nexus C92160YC switch , there are two breakout modes:

  • 40G to 4x10G breakout ports

    • Enables the breakout of 40G ports into 4 X 10G ports.

    • Use the interface breakout module 1 port x map 10g-4x command.

  • 100G to 4x25G breakout ports

    • Enables the breakout of 100G ports into 4 X 25G ports.

    • Use the interface breakout module 1 port x map 25g-4x command.

Cisco Nexus C9272Q Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C9272Q Switch provides 72 40G ports. Ports 37 - 71 support breakout interfaces.

To configure a breakout interface, use the interface breakout module 1 port x map 10g-4x command.

Example: 


switch(config)# interface breakout module 1 port 38 map 10g-4x
switch(config)# show interface ethernet 1/38 capabilities | grep -i break

      Breakout capable:      yes
Cisco Nexus C9332PQ Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C9332PQ Switch provides 24 40G ports that support breakout mode and can be connected to four 10G NIF ports on a FEX. Ports 1 - 12 and 15 - 26 ports are supported. (Ports 13 and 14 ports are reserved and cannot be used for breakout mode.)


Note

All FEXs are supported.


Note

Only the Cisco Nexus 9332PQ switch provides interface breakout support for FEX fabric interfaces. (7.0(3)I3(1) and later)

Cisco Nexus 9000 C93180LC-EX Switch

For 7.0(3)I7(1) and later, Cisco Nexus 9000 C93180LC-EX switch provides three different modes of operation:

  • Mode 1: 28 x 40G + 4 x 40G/100G (Default configuration)

    • Hardware profile portmode 4x100g + 28x40g.

    • 10x4 breakout is supported on the top ports from 1 to 27 (ports 1,3,5, 7...27). If any of the top port is broken out, the corresponding bottom port becomes non-operational. For example, if port 1 is broken out port 2 becomes non-operational.

    • 1 Gigabit and 10 Gigabit QSA is supported on ports 29, 30, 31, and 32. However, QSAs on the top and bottom front panel ports must be of same speed.

    • Ports 29, 30, 31, and 32 support 10x4, 25x4, and 50x2 breakout.

  • Mode 2: 24 x 40G + 6 x 40G/100G

    • Hardware profile portmode 6x100g + 24x40g.

    • 10x4 breakout is supported on the top ports from 1 to 23 (ports 1,3,5, 7...23). If any of the top port is broken out the corresponding bottom port becomes non-operational.

    • Ports 25, 27, 29, 30, 31, and 32 support 10x4, 25x4, and 50x2 breakout.

    • 1 Gigabit and 10 Gigabit QSA is supported on ports 29, 30, 31, and 32. However, QSAs on the top and bottom front panel ports must be of same speed.

  • Mode 3: 18 x 40G/100G

    • Hardware profile portmode 18x100g.

    • 10x4, 25x4, and 50x2 breakout is supported on top ports from 1 to 27 (ports 1,3,5, 7...27) and on ports 29,30,31,32.

    • 1 Gigabit and 10 Gigabit QSA is supported on all the 18 ports.

Changing Mode 3 to any other mode or vice versa requires copy running-config startup-config command followed by reload command to take effect. However, moving between Modes 1 and 2 is dynamic and requires only copy running-config startup-config command.

Use the show running-config | grep portmode command to display the current operation mode.

Example: 


switch(config-if-range)# show running-config | grep portmode

hardware profile portmode 4x100G+28x40G

With the Cisco Nexus C93180LC-EX switch, there are three breakout modes:

  • 40G to 4x10G breakout ports

    • Enables the breakout of 40G ports into 4 X 10G ports.

    • Use the interface breakout module 1 port x map 10g-4x command.

  • 100G to 4x25G breakout ports

    • Enables the breakout of 100G ports into 4 X 25G ports.

    • Use the interface breakout module 1 port x map 25g-4x command.

  • 100G to 2x50G breakout ports

    • Enables the breakout of 100G ports into 2 X 50G ports.

    • Use the interface breakout module 1 port x map 50g-2x command.

Virtual Device Contexts

Cisco NX-OS can segment operating system and hardware resources into virtual device contexts (VDCs) that emulate virtual devices. The Cisco Nexus 9000 Series switch does not support multiple VDCs. All switch resources are managed in the default VDC.

High Availability for Interfaces

Interfaces support stateful and stateless restarts. A stateful restart occurs on a supervisor switchover. After the switchover, Cisco NX-OS applies the runtime configuration.