Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Port Group Restrictions

Port Group Speed Selection

Dual-Rate Transceivers

Troubleshoot

Introduction

This document describes restrictions for ports on C9600-LC-40YL4CD and C9600-LC-48YL line cards when used with the C9600X-SUP-2 and C9500X-60L4D.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these software and hardware versions:

- C9606R chassis

- C9600X-SUP-2

- C9600-LC-40YL4CD

- C9600-LC-48YL

- C9500X-60L4D

- Cisco IOS® XE 17.7.1 and later

Note: The port group restrictions discussed in this article are not applicable to other non small form-factor pluggable (SFP) line cards, such as the C9600-LC-24C and C9600-LC-48TX, which is supported with C9600X-SUP-2 supervisor on C9606R chassis.  These restrictions are also not applicable for other speeds like 400G/200G/100G/40G transceivers on the line cards with C9600X-SUP-2 supervisor.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

The port group restrictions discussed in this article are applicable only for C9600-LC-40YL4CD and C9600-LC-48YL line cards with C9600X-SUP-2 supervisor on C9606R chassis, and the C9500X-60L4D.

Port Group Restrictions

On the C9500X-60L4D switch, and on the C9600-LC-48YL and C9600-LC-40YL4CD linecards, when used with the C9600X-SUP-2; there is a port group hardware restriction to be aware of. The front-panel ports are mapped together to form a port group; for both ports in the port group to link-up and operate seamlessly, both ports must be the same speed. If the ports in the port group are of different speeds, at least one port remain(s) in an err-disabled state due to sfp-config-mismatch.

Note: Front-panel SFP ports that are mapped to the same port group MUST have the SAME speed transceivers inserted, for them both to be recognized, link up and operate.

As shown in the images, on the C9600-LC-48YL and C9600-LC-40YL4CD linecards, front-panel ports 1 & 2 are grouped together to form port group 1. Similarly, ports 3 & 4 are grouped together to form port group 2; and so on.

On the C9500X-60L4D, the front-panel ports 1 & 2 are grouped together to form port group 1. Similarly, ports 4 & 5 are grouped together to form port group 2; ports 7 & 8 are grouped together to form port group 3; and so on.

Note: It is important to note with the C9500X-60L4D, the port group restriction does not apply to the 400G and 50G ports on the bottom row (ports 3, 6, 9, 12, and so on).

Port Group Speed Selection

If different speed transceivers are inserted on the front-panel ports which are mapped to the same port group, then the port on which the transceiver was inserted second, shall be placed into error-disabled state. Use the command show interfaces status err-disabled to verify the reason for error disable to be "sfp-config-mismatch".

Since the behavior is based on which transceiver is inserted first, there can be unexpected err-disable events during reloads, supervisor failovers, and line card Online Insertion and Removal (OIR). In order to make this behavior deterministic, a new port group command line interface (CLI) command has been introduced starting in 17.7.1 Cisco IOS release.

Switch(config)# hw-module slot <line card slot> port-group ?

<1-24>  port-group to apply feature

range   Range of port-group

Switch(config)# hw-module slot <line card slot> port-group <port-group number> select ?

10G  Speed of 10Gbps

25G  Speed of 25Gbps

50G  Speed of 50Gbps

When operating in mixed speed (10G & 25G) transceivers, configure this port group CLI for a given port group to always linkup at a specific speed.

Example (in line card slot 1):

- Front-panel ports 1 & 2, where port 1 has 10G transceiver inserted and port 2 has 25G transceiver inserted.

- If the intent is to make 10G always preferred, configure the port group config CLI as seen in the example:

hw-module slot 1 port-group 1 select 10G

In this example, the port having 25G transceivers are kept in err-disabled state due to "sfp-config-mismatch", across the events like transceivers OIR, reload, supervisor failover, line card OIR.

Example (in line card slot 2):

- Front-panel ports 1 & 2, where port 1 has 10G transceiver inserted and port 2 has 25G transceiver inserted.

- If the intent is to make 25G always preferred, configure the port group config CLI as seen in the example:

hw-module slot 2 port-group 1 select 25G

In this example, the port with a 10G transceiver is kept in err-disabled state due to "sfp-config-mismatch" across the events like transceiver OIR, reload, supervisor failover, line card OIR.

Note: Use the command "show hw-module slot port-group speed" to verify operating speed of a given port group on a specific line card slot.

In general, it is recommended to use the same speed transceivers in both ports within a port group, such as ports 1 & 2 and 3 & 4, and so on, for these ports on the line cards to operate seamlessly.

Dual-Rate Transceivers

When dual-rate transceivers, such as 10/25G, are inserted in ports on these line cards, these dual-rate transceivers operate and link up at higher speed, such as 25G. When you want to configure these transceivers to operate at lower speed of 10G, first issue the "shutdown" command to administratively shutdown the port, then specifically configure "speed 10G" on the port, and then issue "no shutdown".  If the port goes into error-disabled state, perform "shutdown" and "no shutdown" again on the same port, for the port to link up.

For other dual-rate transceivers, such as 25/50G, please see the Cisco Optics-to-Device Compatibility Matrix for compatibility.

Note: When using dual-rate transceivers, the port group speed selection CLI (hw-module slot port-group select ) is not supported. If dual-rate transceivers are inserted on the front-panel ports which are mapped to the same port group, and there is mismatch in speeds; it is recommended to shutdown the port which must not be brought up.

Example:

- Front-panel port 1 has 10/25G dual-rate transceiver and is configured for 10G; front-panel port 2 has 25G transceiver inserted.

- If the intent is to make the 10G in port 1 always preferred, shutdown port 2.

Example:

- Front-panel port 1 has 10/25G dual-rate transceiver and is configured for 10G; front-panel port 2 has 10/25G transceiver and is configured for 25G.

- If the intent is to make the 25G in port 2 always preferred, shutdown port 1.

- Ideally in this example, port 1 would be configured to also be 25G, so both ports can linkup and operate.

Troubleshoot

Use these show commands to troubleshoot issues related to front-panel ports getting err-disabled due to sfp-config-mismatch

- show interface status err-disabled (watch out for interface err-disabled due to sfp-config-mismatch)

- show hw-module slot <line card slot> port-group speed (verify operating speed of a given port group on a specific line card slot. "NONE" is the default)

Example: C9600-LC-40YL4CD in slot 5

Switch# show hw-module slot 5 port-group speed

--------------------------------------------------

Port-groupPorts                         Speed

--------------------------------------------------

1          Fif5/0/1, Fif5/0/2           NONE

2          Fif5/0/3, Fif5/0/4           NONE

3          Fif5/0/5, Fif5/0/6           NONE

4          Fif5/0/7, Fif5/0/8           NONE

5          Fif5/0/9, Fif5/0/10          NONE

6          Fif5/0/11, Fif5/0/12         NONE

7          Fif5/0/13, Fif5/0/14         NONE

8          Fif5/0/15, Fif5/0/16         NONE

9          Fif5/0/17, Fif5/0/18         NONE

10         Fif5/0/19, Fif5/0/20         NONE

11         Fif5/0/21, Fif5/0/22         NONE

12         Fif5/0/23, Fif5/0/24         NONE

13         Fif5/0/25, Fif5/0/26         NONE

14         Fif5/0/27, Fif5/0/28         NONE

15         Fif5/0/29, Fif5/0/30         NONE

16         Fif5/0/31, Fif5/0/32         NONE

17         Fif5/0/33, Fif5/0/34         NONE

18         Fif5/0/35, Fif5/0/36         NONE

19         Fif5/0/37, Fif5/0/38         NONE

20         Fif5/0/39, Fif5/0/40         NONE

Example: C9600-LC-48YL in slot 6

Switch# show hw-module slot 6 port-group speed

--------------------------------------------------

Port-groupPorts                         Speed

--------------------------------------------------

1          Fif6/0/1, Fif6/0/2           NONE

2          Fif6/0/3, Fif6/0/4           NONE

3          Fif6/0/5, Fif6/0/6           NONE

4          Fif6/0/7, Fif6/0/8           NONE

5          Fif6/0/9, Fif6/0/10          NONE

6          Fif6/0/11, Fif6/0/12         NONE

7          Fif6/0/13, Fif6/0/14         NONE

8          Fif6/0/15, Fif6/0/16         NONE

9          Fif6/0/17, Fif6/0/18         NONE

10         Fif6/0/19, Fif6/0/20         NONE

11         Fif6/0/21, Fif6/0/22         NONE

12         Fif6/0/23, Fif6/0/24         NONE

13         Fif6/0/25, Fif6/0/26         NONE

14         Fif6/0/27, Fif6/0/28         NONE

15         Fif6/0/29, Fif6/0/30         NONE

16         Fif6/0/31, Fif6/0/32         NONE

17         Fif6/0/33, Fif6/0/34         NONE

18         Fif6/0/35, Fif6/0/36         NONE

19         Fif6/0/37, Fif6/0/38         NONE

20         Fif6/0/39, Fif6/0/40         NONE

21         Fif6/0/41, Fif6/0/42         NONE

22         Fif6/0/43, Fif6/0/44         NONE

23         Fif6/0/45, Fif6/0/46         NONE

24         Fif6/0/47, Fif6/0/48         NONE

Example: C9500X-60L4D

Switch# show hw-module slot 1 port-group speed

--------------------------------------------------

Port-groupPorts                         Speed

--------------------------------------------------

1          Fif1/0/1, Fif1/0/2           NONE

2          Fif1/0/4, Fif1/0/5           NONE

3          Fif1/0/7, Fif1/0/8           NONE

4          Fif1/0/10, Fif1/0/11         NONE

5          Fif1/0/13, Fif1/0/14         NONE

6          Fif1/0/16, Fif1/0/17         NONE

7          Fif1/0/19, Fif1/0/20         NONE

8          Fif1/0/22, Fif1/0/23         NONE

9          Fif1/0/25, Fif1/0/26         NONE

10         Fif1/0/28, Fif1/0/29         NONE

11         Fif1/0/35, Fif1/0/36         NONE

12         Fif1/0/38, Fif1/0/39         NONE

13         Fif1/0/41, Fif1/0/42         NONE

14         Fif1/0/44, Fif1/0/45         NONE

15         Fif1/0/47, Fif1/0/48         NONE

16         Fif1/0/50, Fif1/0/51         NONE

17         Fif1/0/53, Fif1/0/54         NONE

18         Fif1/0/56, Fif1/0/57         NONE

19         Fif1/0/59, Fif1/0/60         NONE

20         Fif1/0/62, Fif1/0/63         NONE

To configure a specific port group to operate at specific speed on a given line card slot, use hw-module slot <line card slot> port-group <port-group range> select <speed>

Example 1: Configure & verify port group 1 & 3 to operate at 10G in slot 5

Switch(config#) hw-module slot 5 port-group 1 select 10G

Switch(config#) hw-module slot 5 port-group 3 select 10G

Switch# show running-config | include port-group

hw-module slot 5 port-group 1 select 10G    <-- Configuration in running config to set these ports to 10G

hw-module slot 5 port-group 3 select 10G    <-- Configuration in running config to set these ports to 10G

Switch# show hw-module slot 5 port-group speed

--------------------------------------------------

Port-groupPorts                         Speed

--------------------------------------------------

1          Fif5/0/1, Fif5/0/2           10G      <-- In 10G mode

2          Fif5/0/3, Fif5/0/4           NONE

3          Fif5/0/5, Fif5/0/6           10G      <-- In 10G mode

4          Fif5/0/7, Fif5/0/8           NONE

5          Fif5/0/9, Fif5/0/10          NONE

6          Fif5/0/11, Fif5/0/12         NONE

7          Fif5/0/13, Fif5/0/14         NONE

8          Fif5/0/15, Fif5/0/16         NONE

9          Fif5/0/17, Fif5/0/18         NONE

10         Fif5/0/19, Fif5/0/20         NONE

11         Fif5/0/21, Fif5/0/22         NONE

12         Fif5/0/23, Fif5/0/24         NONE

13         Fif5/0/25, Fif5/0/26         NONE

14         Fif5/0/27, Fif5/0/28         NONE

15         Fif5/0/29, Fif5/0/30         NONE

16         Fif5/0/31, Fif5/0/32         NONE

17         Fif5/0/33, Fif5/0/34         NONE

18         Fif5/0/35, Fif5/0/36         NONE

19         Fif5/0/37, Fif5/0/38         NONE

20         Fif5/0/39, Fif5/0/40         NONE

Example 2: Configure & verify port group 6 & 8 to operate at 25G in slot 5

Switch(config#) hw-module slot 5 port-group 6 select 25G

Switch(config#) hw-module slot 5 port-group 8 select 25G

Switch# show running-config | include port-group

hw-module slot 5 port-group 6 select 25G <-- Configuration in running config to set these ports to 25G

hw-module slot 5 port-group 8 select 25G <-- Configuration in running config to set these ports to 25G

Switch# show hw-module slot 5 port-group speed

--------------------------------------------------

Port-groupPorts                         Speed

--------------------------------------------------

1          Fif5/0/1, Fif5/0/2           NONE

2          Fif5/0/3, Fif5/0/4           NONE

3          Fif5/0/5, Fif5/0/6           NONE

4          Fif5/0/7, Fif5/0/8           NONE

5          Fif5/0/9, Fif5/0/10          NONE

6          Fif5/0/11, Fif5/0/12         25G <-- In 25G mode

7          Fif5/0/13, Fif5/0/14         NONE

8          Fif5/0/15, Fif5/0/16         25G <-- In 25G mode

9          Fif5/0/17, Fif5/0/18         NONE

10         Fif5/0/19, Fif5/0/20         NONE

11         Fif5/0/21, Fif5/0/22         NONE

12         Fif5/0/23, Fif5/0/24         NONE

13         Fif5/0/25, Fif5/0/26         NONE

14         Fif5/0/27, Fif5/0/28         NONE

15         Fif5/0/29, Fif5/0/30         NONE

16         Fif5/0/31, Fif5/0/32         NONE

17         Fif5/0/33, Fif5/0/34         NONE

18         Fif5/0/35, Fif5/0/36         NONE

19         Fif5/0/37, Fif5/0/38         NONE

20         Fif5/0/39, Fif5/0/40         NONE

Note: If ports are getting err-disabled due to "sfp-config-mismatch" even when same speed transceivers are inserted on ports which are mapped to same port group, collect Input/Output Manager Daemon (IOMD) & Forwarding Engine Driver (FED) tracelogs to troubleshoot the issue further.

Use these commands to further troubleshoot err-disabled ports due to "sfp-config-mismatch":

show logging process iomd to-file bootflash:

show logging process fed to-file bootflash: