New and Changed Information

The following table provides an overview of the significant changes up to the current release. The table does not provide an exhaustive list of all changes or of the new features up to this release.

Table 1. New Features and Changed Behavior

Cisco APIC Release

Feature

Description

5.1(3)

Cisco ACI dual-role switches

This feature is introduced.

About Cisco ACI Dual-Role Switches

Some Cisco Nexus 9000 switches can have dual roles, meaning you can discover the switch as either a leaf switch or spine switch when you run them in the Cisco Application Centric Infrastructure (ACI) mode. Such a switch enables you to the change the switch's role depending on changes in your fabric requirements.

You can change the role of the following switches:

Switch

 Default Role

First Supported Release

N9K-C9364D-GX2A

Spine

15.2(3)

N9K-C9332D-GX2B

Leaf

15.2(3)

N9K-C93600CD-GX

Leaf

15.2(1)

N9K-C9364C-GX

Leaf

15.1(3)

Note

The Cisco Application Policy Infrastructure Controller (APIC) determines a switch's role by how you connected the switch to your fabric. You do not modify a policy to change the role.

Guidelines and Limitations for Cisco ACI Dual-Role Switches

The following guidelines and limitations apply to dual-role Cisco Application Centric Infrastructure (ACI) switches:

  • Switch-to-switch connectivity, such as spine switch-to-leaf switch and leaf switch-to-sub-leaf switch must use fabric links.

  • In the Cisco Application Policy Infrastructure Controller (APIC) 5.1(1) release and later, Cisco APIC-to-switch connectivity must use downlinks.

  • A leaf switch has default downlinks and fabric links that you can change using a port profile (fabric link/downlink conversion) after the switch is discovered by the Cisco APIC. The last two default fabric links cannot be converted to downlinks.

Converting a Dual-Role Switch Using the GUI

The following procedure converts a dual-role switch to another role using the GUI.


Note

You cannot change the role of switches that are directly connected to the Cisco Application Policy Infrastructure Controller (APIC). In such a case, a role change will result in a wiring issue and thereby render the entire fabric unreachable from the Cisco APIC.

Procedure

Step 1

On the menu bar, choose Fabric > Inventory.

Step 2

In the Navigation pane, choose Fabric Membership.

Step 3

In the Work pane, choose Registered Nodes.

Step 4

Right-click the node (switch) that you want to convert and choose Remove From Controller, then click OK.

This action decomissions the switch, which can take up to 10 minutes to complete.
Step 5

For the next steps, see the section that corresponds to your use case:

Cisco APIC-connected Dual-Role Spine Switch Discovery

In this use case, you want to discover a dual-role spine switch as a leaf switch. Perform the following actions:

  1. Connect the dual-role spine switch directly to the Cisco Application Policy Infrastructure Controller (APIC), which causes the switch to be automatically converted to a leaf switch by the policy element (PE) based on its LLDP adjacency to the Cisco APIC. After the conversion, the switch shows up in as an unregistered node in the Cisco APIC

  2. Register the switch. For more information about registering a switch, see the "Fabric Initialization and Switch Discovery" chapter of the Cisco APIC Getting Started Guide.

Dual-Role Leaf Switch Discovery as a Spine Switch

In this use case, you want to convert a dual-role leaf switch to a spine switch. Perform the following actions:

  1. Connect the dual-role leaf switch to another leaf switch, which causes the Cisco Application Policy Infrastructure Controller (APIC) to raise a fault because of an LLDP wiring issue.

  2. Register the dual-role spine switch as a spine switch. For more information about registering a switch, see the "Fabric Initialization and Switch Discovery" chapter of the Cisco APIC Getting Started Guide.

The switch will be clean rebooted automatically after being processed and the Cisco APIC will discover the switch as a spine.

Dual-Role Spine Switch Discovery as a Leaf Switch

In this use case, you want to convert a dual-role spine switch to a leaf switch. Perform the following actions:

  1. Connect the dual-role spine switch to another spine switch, which causes the Cisco Application Policy Infrastructure Controller (APIC) to raise a fault because of an LLDP wiring issue.

  2. Register the dual-role spine switch as a leaf switch. For more information about registering a switch, see the "Fabric Initialization and Switch Discovery" chapter of the Cisco APIC Getting Started Guide.

The switch will be clean rebooted automatically after being processed and the Cisco APIC will discover the switch as a leaf.

Dual-Role Spine Switch Discovery as Tier-2 Leaf Switch

In this use case, you want to convert a dual-role spine switch to a tier-2 leaf switch. To convert the dual-role spine switch, register the switch as a tier-2 leaf switch. For more information about registering a switch, see the "Fabric Initialization and Switch Discovery" chapter of the Cisco APIC Getting Started Guide.

The switch will be clean rebooted automatically after being processed and the Cisco APIC will discover the switch as a tier-2 leaf.

Other Use Cases That Do Not Change the Discovery Workflow

The following list contains use cases that do not require a change in the discovery workflow:

  • A dual-role spine switch is connected to a leaf switch that is connected to the Cisco Application Policy Infrastructure Controller (APIC).

  • A dual-role leaf switch is connected to the Cisco APIC and to a spine switch.

  • A dual-role leaf switch is connected to a spine switch without a connection to the Cisco APIC.

  • A dual-role spine or leaf switch is connected as a Cisco ACI Multi-Pod spine switch or as a remote leaf switch. These switches are completely driven by a node registration policy over DHCP. LLDP does not exist in this configuration.

For these use cases, you must register the dual-role switch as a regular switch if you want to register a dual-role spine as a spine or a dual-role leaf switch as a leaf. For more information about registering a switch, see the "Fabric Initialization and Switch Discovery" chapter of the Cisco APIC Getting Started Guide.

Troubleshooting Dual-Role Switch Support

This section provides troubleshooting suggestions if you encounter any issues after converting a switch to a different role.

Viewing a Switch's Current Configured Role

View the /bootflash/crdcfg_card_mode.cfg configuration file to verify the switch's configured role: 

leaftospine# cat /bootflash/crdcfg_card_mode.cfg
0

The "0" indicates that the switch is configured as a spine switch.

spinetoleaf# cat /bootflash/crdcfg_card_mode.cfg
1

The "1" indicates that the switch is configured as a leaf switch.

You can also view the /mnt/pss/pfm.log log file to verify the switch's configured role: 

leaftospine# grep -B 4 -A 4 "set spine role" /mnt/pss/pfm.log
2020 Nov 25 01:57:36.463454: pfm_dual_role_hdlr@12459: Got card-index 21165
2020 Nov 25 01:57:36.463509: pfm_dual_role_hdlr@12474: chunkId received 2130 eqpch
chunkid 2129 eqpch pe chunkid 2130
2020 Nov 25 01:57:36.463538: pfm_dual_role_hdlr@12484:
MTS_OPC_MODIFY_ImEqptChPolicyelem(316741) received
2020 Nov 25 01:57:36.463585: pfm_dual_role_hdlr@12508: config_role 3 (SPINE)
2020 Nov 25 01:57:36.463601: pfm_dual_role_hdlr@12518: set spine role
2020 Nov 25 01:57:36.473081: pfm_dual_role_hdlr@12521: removing port profile if it
presents
2020 Nov 25 01:57:36.494195: pfm_get_decommission_admin_state@7138: adminst 1 mode 0
...

The highlighted parts specify the switch's role (a spine switch in the example).

Viewing a Switch's Dual-Role Properties

Inspect the eqptCh managed object for the switch's current dual-role properties and configuration:

leaftospine# moquery -c eqptCh
Total Objects shown: 1

# eqpt.Ch
dn           : sys/ch
hybridMode   : yes
id           : 1
model        : N9K-C9332D-GX2B
operSt       : online
operStQual   : Chassis came online
rev          : 1
rn           : ch
role         : spine

If the hybridMode property has a value of "yes," then the switch is capable of having dual roles. The role property specifies the switch's current role.

Use the Fabric Ports of Dual-Role Switches

The fabric ports of dual-role switches must be connected to other nodes. The l1Phys managed object's portT property must be set to fab, which specifies that the port type is fabric. 

rh-leaf# cat /mit/sys/phys-\[eth1--49\]/summary | grep portT
portT                          : fab

rh-leaftospine# show lldp nei
Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
Device ID            Local Intf      Hold-time  Capability  Port ID
vaci-spine3           Eth1/3          120        BR          Eth1/5
apic1                 Eth1/10         120                    eth2-2
vaci-spine3           Eth1/49         120        BR          Eth1/24
Total entries displayed: 3

Miscellaneous Troubleshooting Information

  • Make sure there is no direct link to the Cisco Application Policy Infrastructure Controller (APIC) when a dual-role switch must be configured as a spine switch.

  • If you configure a dual-role switch with a different role than desired by mistake, you must decommission the switch by using Remove From Controller, then register the switch with the correct role. There will be a node-mismatch fault raised initially on the switch until the switch finishes being converted to the correct role.

  • If you connect a dual-role switch using a fabric link to the Cisco APIC, the Cisco APIC raises fault F0106.