Guest

Cisco Nexus 5000 Series Switches

Migration from Virtual PortChannel to Cisco FabricPath

  • Viewing Options

  • PDF (2.2 MB)
  • Feedback

The specifications and information regarding the products in this manual are subject to change without notice. All statements, information, and recommendations in this manual are believed to be accurate but are presented without warranty of any kind, express or implied. Users must take full responsibility for their application of any products.

The software license and limited warranty for the accompanying product are set forth in the information packet that shipped with the product and are incorporated herein by this reference. If you are unable to locate the software license or limited warranty, contact your Cisco representative for a copy.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

Notwithstanding any other warranty herein, all document files and software of these suppliers are provided “as is” with all faults. Cisco and the above-named suppliers disclaim all warranties, expressed or implied, including, without limitation, those of merchantability, fitness for a particular purpose and non-infringement or arising from a course of dealing, usage, or trade practice.

In no event shall Cisco or its suppliers be liable for any indirect, special, consequential, or incidental damages, including, without limitation, lost profits or loss or damage to data arising out of the use or inability to use this manual, even if Cisco or its suppliers have been advised of the possibility of such damages.

Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco's trademarks can be found at http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

©1992-2012 Cisco Systems, Inc. All rights reserved.


List of Figures and Tables. 4

Document Information. 5

About the Authors and Contributors. 5

Introduction. 6

Document Scope. 6

Business Requirements. 6

Current Layer 2 Domain Designs. 6

Layer 2 Designs with Cisco FabricPath. 7

Cisco FabricPath Design Considerations. 7

Cisco FabricPath Terminology. 7

Port Selection and Connectivity Design Considerations. 8

Cisco FabricPath Domain Expansion. 9

Cisco Nexus 7000 F2-Series Line-Card Design Considerations. 10

vPC+ Design Considerations. 10

Software Requirements as Design Considerations. 10

Design Considerations Summary. 11

Cisco FabricPath Migration Use Cases. 11

Migration Scenario 1: Cisco Nexus 7000 M1-Series and F1-Series Chassis Migration. 12

Migration Scenario 2: Cisco Nexus 7000 M1-Series, F1-Series, and F2-Series Chassis Migration. 13

Cisco FabricPath Migration Overview.. 14

Scenario 1: Cisco Nexus 7000 M1-Series and F1-Series Chassis Migration. 14

Scenario 1: Configuration. 15

Scenario 1: Enable Cisco FabricPath on the Switch. 15

Scenario 1: Cisco FabricPath Basic Configuration. 15

Scenario 1: Configure Cisco FabricPath Spine. 15

Scenario 1: Configure Cisco FabricPath Leaf 16

Scenario 1: Migration Validation. 17

Scenario 1: Verify vPC and vPC+ Status After Configuring Cisco FabricPath Spine on Peer Link. 17

Scenario 1: Verify Cisco FabricPath Configuration on the Leaf 19

Scenario 1: Verify Cisco FabricPath Configuration. 20

Scenario 2: Cisco Nexus 7000 M1-Series, F1-Series, and F2 Chassis Migration. 22

Scenario 2: Configuration. 22

Scenario 2: Enable Cisco FabricPath on the Switch. 22

Scenario 2: Cisco FabricPath Basic Configuration. 23

Scenario 2: Configure Cisco FabricPath Spine. 23

Scenario 2: Configure Cisco FabricPath Leaf 24

Scenario 2: Migration Validation. 25

Scenario 2: Verify vPC and vPC+ Status After Configuring Cisco FabricPath Spine on Peer Link. 26

Scenario 2: Verify Cisco FabricPath Configuration on the Leaf 27

Scenario 2: Verify Cisco FabricPath Configuration. 28

Troubleshooting. 31

References. 33

Appendix A: Configuration Before Migration. 33

Appendix B: Configuration after Migration for Case 1. 40

Appendix C: Configuration after Migration for Scenario 2. 47


Document Information

About the Authors and Contributors

Lev Terebizh, CCIE® 6712.
Lev is a network consulting engineer with Cisco on the Data Center and Virtualization (DC&V) Network Architecture team. He has held a wide variety of network design, consulting, and solution validation positions. His technical expertise includes more than 20 years of extensive experience in designing and supporting complex networks, with a special focus on the data center.

Lev has an MS degree in telecommunications. He has held a CCIE certification for more than 10 years.

Azeem Suleman, Double CCIE 23427.
Azeem is a solutions architect with Cisco on the DC&V Network Architecture team. He has written several documents, white papers, and technical tips related to the data center and security. He has been a representative to the Architecture Board and has been designing large-scale enterprise networks for more than a decade.

Azeem has BS degree in computer engineering from SSUET, Pakistan, and an MS degree from the University of Texas at Austin. He has published in the IEEE Conference publication and has been with Cisco® Advanced Services since 2009.

Contributors

Balaji Senthamil Selvan: Consulting engineer with Cisco on the DC&V Network Architecture team
Faraz Siddiqui: Network consulting engineer with Cisco on the DC&V Network Architecture team
Vinay Suvarna: Consulting engineer with Cisco on the DC&V Network Architecture team

Technical Reviews Conducted by Industry Experts

Talha Hashmi: Manager with Cisco on the DC&V Network Architecture team
Francois Tallet: Product manager with Cisco in ECBU
Maurizio Portolani: Senior technical marketing engineer with Cisco in SAVBU
Vimala Veerappan: Technical marketing engineer with Cisco in SAVBU
Yasin Sahebdin: Network consulting engineer with Cisco Advanced Services


Introduction

The demand for high efficiency in today’s data centers is stimulating the growth of new technologies and consumption models. As IT infrastructures are consolidated, technologies such as server and network virtualization provide efficiency while introducing new scalability challenges. The large consolidated and virtualized infrastructures need to be flexible and scalable to meet dynamic demands.

In the past, the size and flexibility of Layer 2 domains in the data center and the separation of the Layer 2 domains were dictated by the Spanning Tree Protocol, but this approach does not meet today’s requirements. New technologies, such as the virtual PortChannel (vPC), help overcome some of the limitations of spanning tree, but they do not address the stability and scalability challenges. Cisco® FabricPath helps enable the building of scalable Layer 2 domains, without the limitations of spanning tree or restrictions related to the vPC solution.

The information in this document assumes that readers have a basic understanding of Cisco FabricPath, spanning tree, and Layer 2 networking technologies and are interested in the process of migrating from vPC to Cisco FabricPath.

Document Scope

This document provides a brief overview of the Cisco FabricPath technology and walks the reader through the steps for migrating to Cisco FabricPath from vPC. The document also demonstrates how to configure a Cisco FabricPath deployment and how to scale a network enabled for Cisco FabricPath by adding new devices in the future by presenting selected use cases.

This document highlights the following:

Configuration simplicity is built into Cisco FabricPath.

A Cisco FabricPath domain does not run the Spanning Tree Protocol, although spanning tree still exists at the edge of the Cisco FabricPath domain but in considerably smaller domain sizes.

Adding new devices to the Cisco FabricPath domain is easy and does not affect the domain in the same way that it would a spanning tree domain.

Business Requirements

Although technologies such as vPC complement Spanning Tree Protocol, they do not provide independence from the inherent challenges of Spanning Tree Protocol, including looping. Cisco FabricPath technology allows easy reconfiguration of a network with little disruption and is safe for use to extend the network within and across data centers without the risk of looping.

The sections that follow in this document discuss the Layer 2 design needs and challenges involving both the Spanning Tree Protocol and Cisco FabricPath in the data center environment.

Current Layer 2 Domain Designs

Most existing Layer 2 domains use the traditional Spanning Tree Protocol. However, spanning tree is inefficient in its use of the bandwidth of redundant links. Spanning Tree Protocol deployments usually have at least half the links blocked and do not participate in traffic forwarding. Another disadvantage is convergence. Each time a network changes, the spanning tree has to be recalculated. This disadvantage especially applies to Layer 2 domains.

Layer 2 domains designed with vPC technology make better use of the redundant links but still use Spanning Tree Protocol as a backup mechanism. As a result, a user must still follow spanning tree best practices.

Layer 2 Designs with Cisco FabricPath

By introducing a new control protocol (based on the Intermediate System to Intermediate System [IS-IS] Protocol) and a new data plane, Cisco FabricPath can work around most of the limitations that affect a traditional Ethernet network. In fact, Cisco FabricPath offers benefits of both Layer 2 and Layer 3 technologies. Equal Cost Multipathing (ECMP) allows Cisco FabricPath to use the total bandwidth of multiple parallel links.

Cisco FabricPath frames, including time-to-live (TTL) and reverse-path forwarding check (RPFC) frames, are applied to multidestination traffic. Also, unlike vPC, Cisco FabricPath can handle an arbitrary network topology.

All the elements borrowed from Layer 3 technologies make Cisco FabricPath safe for extension to an entire data center without the risk of looping and allow easy reconfiguration of a network with little disruption.

Cisco FabricPath Design Considerations

A typical Cisco FabricPath network topology is the Clos fabric, shown in Figure 1. A Clos fabric consists of two kinds of node: leaf switches and spine switches. A particular leaf switch is connected to all the spine switches, and a particular spine switch is connected to all the leaf switches.

The goal for the network is to provide optimal connectivity between the leaf switches, with the hosts attached using Classical Ethernet (edge) ports.

Cisco FabricPath Terminology

Table 1 defines important Cisco FabricPath terms. These terms are used throughout this document.

Table 1. Cisco FabricPath Terminology

Cisco FabricPath Terminology

Term

Definition

Cisco FabricPath domain

Layer 2 domain formed by interconnected Cisco FabricPath core interfaces and carrying Cisco FabricPath VLAN traffic: All traffic in the Cisco FabricPath domain is Cisco FabricPath encapsulated.

Cisco FabricPath core port

Interface connected to the Cisco FabricPath domain: The Cisco FabricPath core interface carries traffic encapsulated in Cisco FabricPath frames and can also be referred to as a FabricPath (FP) port. A Cisco FabricPath core port must be connected to another Cisco FabricPath core port. The Cisco FabricPath core port carries all Cisco FabricPath VLANs and, therefore, can be conceptually considered as a trunk port.

Cisco FabricPath edge port

Interface at the edge of the Cisco FabricPath domain: Cisco FabricPath edge interfaces carry traffic encapsulated in regular Ethernet frames and also can be referred as Classical Ethernet (CE) ports. Cisco FabricPath edge ports can be connected to any standard Ethernet port. Cisco FabricPath edge ports are used to attach any regular Ethernet device to the Cisco FabricPath domain.

Cisco FabricPath VLAN

VLAN allowed to cross a Cisco FabricPath domain.

Classical Ethernet (CE) VLAN

VLAN not allowed to cross a Cisco FabricPath domain, but allowed to exist on the edge interfaces of the Cisco FabricPath attached device.

Port Selection and Connectivity Design Considerations

To make the right design decisions about physical connectivity, you need to know about the interface capabilities for Cisco FabricPath. Any Cisco Nexus® 5500 platform switch interface can be configured as a core port (FabricPath mode) or edge port (CE mode). For the Cisco Nexus 7000 Series Switches, configuration of an interface as a core port (FabricPath mode) or an edge port (CE mode) depends on the line-card model. Table 2 lists the available interface mode capabilities for various line cards.

Table 2. Interface Mode Capabilities

Line card model

Interface mode

Classical Ethernet

FabricPath

N7K-M108XP

YES

NO

N7K-M132XP

YES

NO

N7K-M148XP

YES

NO

N7K-F132XP-15

YES

YES

N7K-F248XP-25

YES

YES

N5K-C5548P-FA

YES

YES

N5K-C5548UP-FA

YES

YES

N5K-C5596UP-FA

YES

YES

Basically, in a Cisco FabricPath network all ports (that is, core or edge ports) must be capable of supporting Cisco FabricPath. To enforce this restriction, a new Cisco FabricPath VLAN type has been created. Cisco FabricPath VLANs do not become active on ports that do not support Cisco FabricPath. Figure 2 shows that only the Cisco Nexus 7000 F-Series I/O modules support Cisco FabricPath on the Cisco Nexus 7000 Series Switches. Note that although no Cisco FabricPath VLAN can be configured on Cisco Nexus 7000 M-Series I/O module ports, those modules can still be used to create switched virtual interfaces (SVIs) for routing traffic between Cisco FabricPath VLANs handled by Cisco Nexus 7000 F1-Series I/O modules. The ports on Cisco Nexus 5500 platform switches also are all capable of supporting Cisco FabricPath.

Cisco FabricPath Domain Expansion

Figure 3 illustrates the available connectivity options for the Cisco FabricPath domain.

The first two options are examples of a case in which you cannot extend the Cisco FabricPath domain because the Cisco Nexus 7000 M1-Series interface cannot be configured in FabricPath mode.

The third option prohibits the Cisco FabricPath domain extension because the Cisco Nexus 7000 M1-Series interface cannot carry the Cisco FabricPath VLAN.

The connectivity options shown in Error! Reference source not found. represent Cisco FabricPath extendable options. You do not have any interface limitations for expansion of Cisco FabricPath with the presented selection of the interfaces. The first case shows the option for the Cisco Nexus 7000 Series, and the second example presents the same option for the Cisco Nexus 5500 platform.

Figure 4. Extending Cisco FabricPath Domain

Cisco Nexus 7000 F2-Series Line-Card Design Considerations

Ports from the Cisco Nexus 7000 F2-Series line card must be placed in separate virtual device contexts (VDCs). Cisco Nexus 7000 F2-Series line cards support Layer 3, so you can now build routing on Cisco Nexus 7000 M1-Series or F2-Series line cards.

vPC+ Design Considerations

Cisco FabricPath supports ECMP, which eliminates the need in vPC for the Cisco FabricPath domain. Outside the Cisco FabricPath domain, you still have to use vPC to provide active-active connectivity. In the context of Cisco FabricPath, the vPC feature has been renamed vPC+. The only significant differences between vPC and vPC+ are that the latter requires the configuration of a virtual switch ID and the peer link now consists of Cisco FabricPath core ports.

Software Requirements as Design Considerations

Be sure to keep in mind the minimum software requirements for each piece of hardware used in your Cisco FabricPath design (Table 3).

Table 3. Cisco FabricPath Minimum Software Requirements

Cisco FabricPath Minimum Software Requirements

Hardware Model

Minimum Cisco NX-OS Software Release

Recommended Cisco NX-OS Release

N7K-F132XP-15

Release 5.1

Release 5.2(4)*

N7K-F248XP-25

Release 6.0

Release 6.0(3)*

N5K-C5548P-FA

Release 5.1.3

Release 5.1.3N2(1)*

N5K-C5548UP-FA

Release 5.1.3

Release 5.1.3N2(1)*

N5K-C5596UP-FA

Release 5.1.3

Release 5.1.3N2(1)*

* These requirements may change, so please consult your Cisco representative.

Design Considerations Summary

The following is a summary of the specific design details:

Cisco FabricPath traffic is forwarded only on Cisco FabricPath VLANs.

Only ports on Cisco Nexus 7000 F1-Series and F2-Series I/O modules and on Cisco Nexus 5500 platform switches can be used as Cisco FabricPath core and edge ports. In particular, Cisco FabricPath VLANs will not be activated on Cisco Nexus 7000 M-Series I/O modules on Cisco Nexus 7000 Series Switches.

vPC is called vPC+ in the context of Cisco FabricPath.

vPC+ is basically equivalent to vPC but uses Cisco FabricPath on the peer link.

Cisco Nexus 7000 F2-Series ports must be placed in separate VDCs.

Cisco Nexus 7000 F2-Series line cards are Layer 3 capable, so you can now build routing on Cisco Nexus 7000 M1-Series or F2-Series line cards.

Cisco FabricPath Migration Use Cases

The migration to Cisco FabricPath can begin from different starting points and have few different steps to complete the process. This section presents a review of selected cases and details the migration steps.

The starting point for the most common migration case may be the one depicted in Figure 5.

To initiate the migration, two scenarios can be considered.

Scenario 1 represents a migration to Cisco FabricPath on the Cisco Nexus 7000 M1-Series or F1-Series chassis. This scenario also covers the case in which a Cisco Nexus 7000 F1-Series line card is added to the Cisco Nexus 7000 M1-Series chassis with only one difference requiring changes in cabling.

Scenario 2 represents a migration to Cisco FabricPath with the addition of a Cisco Nexus 7000 F2-Series line card to a Cisco Nexus 7000 M1-Series or F1-Series chassis. In this scenario, you have two spine layers. One layer is built from Cisco Nexus 7000 F1-Series ports, and the second layer is built from Cisco Nexus 7000 F2-Series ports.

In both cases, you will have aggregation layer switches built from a Cisco Nexus 7000 M1-Series or F1-Series chassis and connected to the access layer switches built with Cisco Nexus 5500 platform switches. Aggregation-to-access layer connection will use double-sided vPC.

For migration planning in both scenarios, the way in which the Cisco Nexus 2000 Series Fabric Extender is connected to the parent switch. A single-connected fabric extender will not be affected by the Cisco FabricPath migration. However, a dual-connected fabric extender will be affected by the migration, because the migration from vPC to vPC+ requires a vPC flap, causing the fabric extender to be unregistered and registered again. For the testing, dual-connected fabric extenders were used because these would provide the most valuable information for migration planning.

Both scenarios require the building of a spine or core and a leaf or edge. A Cisco FabricPath domain spine must be built first, and then that is expanded to the leaf. In the test scenarios, the spine is built from Cisco Nexus 7000 Series Switches, and the leaf is built on the Cisco Nexus 5500 platform.

Migration Scenario 1: Cisco Nexus 7000 M1-Series and F1-Series Chassis Migration

In scenario 1 (Figure 6), the Cisco FabricPath spine is built on Cisco Nexus 7000 Series Switches with Cisco Nexus 7000 F1-Series line cards, and the leaf is built from the Cisco Nexus 5500 platform. Cisco Nexus 2000 Series Fabric Extenders will be dual-connected to the Cisco Nexus 5500 platform switches. Before migration starts, the Cisco Nexus 7000 Series Switches will have a vPC peer configured on Cisco Nexus 7000 M1-Series ports.

To simplify migration testing, Cisco Nexus 5500 is already connected to the Cisco Nexus 7000 F1-Series ports of the Cisco Nexus 7000 Series Switch. If a Cisco Nexus 5500 platform switch is connected to the Cisco Nexus 7000 M1-Series ports of the Cisco Nexus 7000 Series Switch, migration requires the additional step of moving connections from the Cisco Nexus 5500 platform switch from the Cisco Nexus 7000 M1-Series to the Cisco Nexus 7000 F1-Series ports on the Cisco Nexus 7000 Series Switch.

By the end of the migration, all peer links and links between the spine and leaf switches will be working in FabricPath mode. Green connection lines in Figure 7 show links configured in FabricPath mode.

Migration Scenario 2: Cisco Nexus 7000 M1-Series, F1-Series, and F2-Series Chassis Migration

In scenario 2, the Cisco FabricPath spine is built on the Cisco Nexus 7000 Series with the Cisco Nexus 7000 F2-Series, and the leaf is built from the Cisco Nexus 7000 F1-Series and Cisco Nexus 5500 Series platform. This scenario has two leafs.

The Cisco Nexus 2000 Series Fabric Extenders will be dual-connected to the Cisco Nexus 5500 platform switches. Before migration starts, the Cisco Nexus 7000 Series Switches will have a vPC peer configured on the Cisco Nexus 7000 M1-Series ports. To simplify migration steps, the Cisco Nexus 5500 platform switch is already connected to the Cisco Nexus 7000 F1-Series ports of the Cisco Nexus 7000 Series Switch. If the Cisco Nexus 5500 platform switch were connected to the Cisco Nexus 7000 M1-Series ports of the Cisco Nexus 7000 Series Switch, migration would require the additional step of moving connections from the Cisco Nexus 5500 platform from the Cisco Nexus 7000 M1-Series to the Cisco Nexus 7000 F1-Series ports on the Cisco Nexus 7000 Series.

The network diagram for this scenario before migration is exactly the same as the one for scenario 1. By the end of the migration, all peer links and links between the spine and leaf switches will be working in FabricPath mode. The green connection lines in Figure 8 show links configured in FabricPath mode.

Cisco FabricPath Migration Overview

Taking into Consideration available hardware and existing deployments, these are the two main scenarios for migration to Cisco FabricPath:

The first test case scenario is a design in which Cisco FabricPath is added to the network with few changes in the cabling and without any additional hardware, and with a Layer 2 domain buildout of the Cisco Nexus 7000 Series Switches with Cisco Nexus 7000 F1-Series line cards and Cisco Nexus 5500 platform switches with Cisco Nexus 2232 fabric extenders.

The second test case scenario is a design in which Cisco FabricPath is added to the network using additional Cisco Nexus 7000 F2-Series line cards forming two Cisco FabricPath spines out of Cisco Nexus 7000 F1-Series and F2-Series ports. This case’s starting point is exactly the same as in the first case, but it requires additional cabling.

Scenario 1: Cisco Nexus 7000 M1-Series and F1-Series Chassis Migration

Migration starts with movement of the peer link from the Cisco Nexus 7000 M1-Series port to the Cisco Nexus 7000 F1-Series port. A peer link cannot be built on Cisco Nexus 7000 M1-Series and F1-Series ports at the same time, so you need to remove the Cisco Nexus 7000 M1-Series ports first and then add the Cisco Nexus 7000 F1-Series ports to the peer link. This configuration will cause a peer-link failure. The primary vPC peer will stay operational when the peer link is down. The secondary vPC peer will shut down all vPC ports in the event of a peer link failure to avoid an active-active situation. It takes less than a minute for a peer link to come up after this step of migration.

Scenario 1: Configuration

Scenario 1: Enable Cisco FabricPath on the Switch

The next step is to implement a basic Cisco FabricPath configuration on all devices participating in the migration. Configuration starts with the activation of Cisco FabricPath (an enhanced Layer 2 license is required). The following commands provide an example of Cisco FabricPath activation.

conf t
install feature-set fabricpath
feature-set fabricpath

Scenario 1: Cisco FabricPath Basic Configuration

Each of the devices in the Cisco FabricPath domain must have a configured Cisco FabricPath switch ID. This step is optional, but you should configure a switch ID rather than allow a default configuration that may be hard to comprehend and troubleshoot at a later time. For the switch ID configuration, the recommended approach is to use single-digit numbers for the spine switches, double-digit numbers for the leaf switches, and four-digit numbers for the virtual switch ID on vPC+.

fabricpath switch-id 1

An additional item to configure is the Cisco FabricPath priority to assign the root for the multidestination tree. The recommended approach is to configure two spine switches as the primary and secondary roots for multidestination trees.

fabricpath domain default
root-priority 255

Note: This entire basic configuration will not cause any downtime on the network.

Scenario 1: Configure Cisco FabricPath Spine

The next step in the migration involves building a Cisco FabricPath core on the peer link of the spine. That core will be the first small piece of the Cisco FabricPath domain you are building. You will need to change the peer-link interfaces to FabricPath mode.

After that, you need to change all VLANs that must be able to cross this Cisco FabricPath link to FabricPath mode. This mode will allow traffic for configured VLANs to cross the Cisco FabricPath domain. The last part of this step changes vPC to vPC+ by adding a Cisco FabricPath switch ID to the vPC domain configuration. That step creates a virtual entity for the vPC domain to be represented in the Cisco FabricPath domain.

/* Turn peer-link into fabricpath
interface port-channel 1
switchport mode fabricpath
/* Change vPC to vPC+
vpc domain 11
fabric switch-id 1011
/* Configure fabricpath VLANs
vlan 1,10-19,100,200,1010-1599
mode fabricpath

At the same time, you need to configure spanning tree so that spine switches will be the spanning tree root for all connected Classical Ethernet segments. If you skip this step, all ports will be up, but traffic will be blocked by spanning tree.

/* Configure fabricpath edge as spanning tree root
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096

At this point, the Cisco FabricPath domain is limited to the peer link on the spine switches. The Cisco Nexus 5500 platform switch would not see any difference. It will be a vPC connection to the Cisco Nexus 7000 Series Switch. But on the Cisco Nexus 7000 Series Switch, it will be vPC+, so you will have a vPC domain connected to the vPC+ domain.

Note: Changing vPC to vPC+ requires flapping all vPCs. This process is disruptive.

Scenario 1: Configure Cisco FabricPath Leaf

The next step needs to be repeated as many times as you have access-layer switch pairs (if you are dealing with access-layer vPC domains). In other words, you will start expanding the Cisco FabricPath domain from the spine to the leaf by adding one leaf at a time.

In this step, disruption or downtime will affect only pairs of access switches and all fabric extenders connected to the pairs. The migration itself will look like migration on the spine. You need to change the peer link to FabricPath mode, change VLAN mode to FabricPath mode, and change vPC to vPC+. On both a spine and leaf, you have to remove ports from vPC and configure ports in FabricPath mode. vPCs have to be deleted, but that can be done after migration is complete as part of configuration cleanup.

/* On the spine: change vPC connection to fabricpath
interface Ethernet 3/1-2
no channel-group 112 mode active
switchport mode fabricpath
no interface port-channel 112
/* On the leaf: Turn peer-link into fabricpath
interface port-channel 1
switchport mode fabricpath
/* On the leaf: Change vPC to vPC+
vpc domain 112
fabric switch-id 1112
/* On the leaf: Configure fabricpath VLANs
vlan 1,10-19,100,200,1010-1599
mode fabricpath
/* Configure fabricpath edge as spanning tree root
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096
mode fabricpath
/* On the leaf: change vPC connection to fabricpath
interface Ethernet 1/1-2
no channel-group 112 mode active
switchport mode fabricpath
no interface port-channel 112

Scenario 1: Migration Validation

The following commands are used to validate a successful completion of the migration.

show vpc brief - brief display of vPC status
show interface brief - verify interface status
show fabricpath switch-id - show fabricpath switch-id
show fabricpath isis adjacency - display IS-IS adjacency information
show fabricpath route - show FabricPath route information
show spanning-tree - verify spanning tree protocol status
show spanning-tree vlan 100 - verify spanning tree status for VLAN

You need to verify the vPC status at a few different points of the migration process. First, you should verify that the spine has vPC+ and that the leaf has vPC configured properly.

When you have the interfaces configured in FabricPath mode, you can validate the Cisco FabricPath configuration.

When the spine exists only on the peer link, you can validate the vPC-to-vPC+ configuration only. When you have expanded the Cisco FabricPath domain to the links between switches, you can start validating the Cisco FabricPath domain itself.

Scenario 1: Verify vPC and vPC+ Status After Configuring Cisco FabricPath Spine on Peer Link

When vPC is converted to vPC+ in the Cisco FabricPath domain, you need to verify that vPC+ is in the up state.

N7K1-1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 11
vPC+ switch id : 1011
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
vPC fabricpath status : peer is reachable through fabricpath
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 2
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Fabricpath load balancing : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1,10-19,100,200,1010-1599
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans vPC+ Attribute
-- ---- ------ ----------- ------ ------------ --------------
112 Po112 up success success 1,10-19,100,2 DF: Yes, FP
00,1010-1599 MAC:
1011.11.4513
134 Po134 up success success 1,10-19,100,2 DF: Yes, FP
00,1010-1599 MAC:
1011.12.4513

When the vPC is connected to the vPC+ on the leaf (FEX connections), we need to verify that vPC are in and up state.

N5548UP-2(config)# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 112
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary, operational primary
Number of vPCs configured : 67
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1,10-19,100,200,1010-1599
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
101 Po101 up success success -
102 Po102 up success success -
112 Po112 up success success 1,10-19,100
,200,1010-1
599
102400 Eth101/1/1 up success success 1,10-19,100
,200,1010-1
599

Scenario 1: Verify Cisco FabricPath Configuration on the Leaf

When all vPC links between the spine and leaf are converted to FabricPath mode, the only vPC+ you will see in the up state will be a peer link. You can verify the status of the link between switches using the command shown here.

N5548UP-2(config-if)# show interface brief
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
Eth1/1 1 eth f-path up none 10G(D) 112
Eth1/2 1 eth f-path up none 10G(D) 112
Eth1/3 1 eth f-path up none 10G(D) 1
Eth1/4 1 eth f-path up none 10G(D) 1
Eth1/5 1 eth access down SFP not inserted 10G(D) --
Eth1/6 1 eth access down SFP not inserted 10G(D) --
Eth1/7 1 eth trunk down SFP not inserted 10G(D) --
Eth1/8 1 eth trunk down SFP not inserted 10G(D) --
Eth1/9 1 eth f-path up Administratively down 10G(D) --
Eth1/10 1 eth f-path up Administratively down 10G(D) --
Eth1/11 1 eth trunk down SFP not inserted 10G(D) --
Eth1/12 1 eth trunk down SFP not inserted 10G(D) --
Eth1/13 1 eth trunk down SFP not inserted 10G(D) --
Eth1/14 1 eth trunk down SFP not inserted 10G(D) --
Eth1/15 1 eth trunk down SFP not inserted 10G(D) --
Eth1/16 1 eth trunk down SFP not inserted 10G(D) --
Eth1/17 1 eth fabric up none 10G(D) 101
Eth1/18 1 eth fabric up none 10G(D) 102
Eth1/19 1 eth trunk down SFP not inserted 10G(D) --

Scenario 1: Verify Cisco FabricPath Configuration

When you have the Cisco FabricPath configuration expanded to the leaf switches, you can see a Cisco FabricPath switch ID forming the Cisco FabricPath domain. The show fabricpath switch-id command displays a list of the switches included in the Cisco FabricPath domain.

N7K1-1# show fabricpath switch-id
FABRICPATH SWITCH-ID TABLE
Legend: '*' - this system
=========================================================================
SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED
----------+----------------+------------+-----------+--------------------
*1 001b.54c2.1cc1 Primary Confirmed Yes No
2 001b.54c2.1e41 Primary Confirmed Yes No
11 547f.ee52.87bc Primary Confirmed Yes No
12 547f.ee29.4ec1 Primary Confirmed Yes No
13 547f.ee24.4381 Primary Confirmed Yes No
14 547f.ee04.023c Primary Confirmed Yes No
1011 001b.54c2.1cc1 Primary Confirmed No Yes
1011 001b.54c2.1e41 Primary Confirmed No Yes
1112 547f.ee29.4ec1 Primary Confirmed No Yes
1112 547f.ee52.87bc Primary Confirmed No Yes
1134 547f.ee04.023c Primary Confirmed No Yes
1134 547f.ee24.4381 Primary Confirmed No Yes
Total Switch-ids: 12

The show fabricpath isis adjacency command displays a list of the directly connected switches in the Cisco FabricPath domain.

N7K1-1# show fabricpath isis adjacency
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
N7K2-1 N/A 1 UP 00:00:27 port-channel1
N5548UP-1 N/A 1 UP 00:00:32 Ethernet3/1
N5548UP-2 N/A 1 UP 00:00:25 Ethernet3/2
N5548UP-3 N/A 1 UP 00:00:23 Ethernet3/3
N5548UP-4 N/A 1 UP 00:00:31 Ethernet3/4

The Cisco FabricPath routing table shows Layer 2 routes to reach the Cisco FabricPath switch identified by the switch ID.

N7K2-1(config)# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/2/0, number of next-hops: 0
via ---- , [60/0], 8 day/s 03:21:19, local
0/1011/11, number of next-hops: 1
via Po112, [80/0], 1 day/s 02:53:02, vpcm
0/1011/12, number of next-hops: 1
via Po134, [80/0], 1 day/s 02:53:02, vpcm
1/1/0, number of next-hops: 1
via Po1, [115/20], 1 day/s 02:53:01, isis_fabricpath-default
1/1011/0, number of next-hops: 0
via ---- , [60/0], 1 day/s 02:53:02, local
2/1011/0, number of next-hops: 0
via ---- , [60/0], 1 day/s 02:53:02, local

Verify the Spanning Tree Protocol status on the switch that does not have any spanning tree domains connected to the Cisco FabricPath domain.

N7K1-1# show spanning-tree
No spanning tree instance exists.

Verify the Spanning Tree Protocol status for a particular VLAN on the edge of the Cisco FabricPath domain. This VLAN must not be in the blocking (BLK) state.

N7K1-1# show spanning-tree vlan 100
VLAN0100
Spanning tree enabled protocol rstp
Root ID Priority 4196
Address c84c.75fa.6000
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 4196 (priority 4096 sys-id-ext 100)
Address c84c.75fa.6000
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po112 Desg FWD 1 128.4207 (vPC) P2p

The following listing shows the output when Cisco FabricPath is not the root and traffic is blocked for this VLAN.

N7K2-1(config)# show spanning-tree vlan 100
VLAN0100
Spanning tree enabled protocol rstp
Root ID Priority 32868
Address c84c.75fa.6000
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32868 (priority 32768 sys-id-ext 100)
Address c84c.75fa.6000
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po112 Desg BKN*1 128.4207 (vPC) P2p *L2GW_Inc

Scenario 2: Cisco Nexus 7000 M1-Series, F1-Series, and F2 Chassis Migration

Migration starts with the movement of peer links from Cisco Nexus 7000 M1-Series ports to Cisco Nexus 7000 F1-Series ports. It is not possible to build a peer link on Cisco Nexus 7000 M1-Series and F1-Series ports at the same time, so you need to remove Cisco Nexus 7000 M1-Series ports first and then add Cisco Nexus 7000 F1-Series ports to the peer link. This configuration will cause a peer-link failure.

The primary vPC peer will stay operational when a peer link is down. The secondary vPC peer will shut down all vPC ports in the event of a peer-link failure to avoid an active-active situation. It takes less than a minute for a peer link to come up after this step of migration.

Note: Network diagrams prior to migration in this scenario are exactly the same as those for scenario 1.

Scenario 2: Configuration

Scenario 2: Enable Cisco FabricPath on the Switch

The next step is to implement a basic Cisco FabricPath configuration on all devices participating in the migration. The configuration starts with activation of Cisco FabricPath (an Enhanced Layer 2 license is required). The following commands provide an example of Cisco FabricPath activation.

conf t
install feature-set fabricpath
feature-set fabricpath

Scenario 2: Cisco FabricPath Basic Configuration

Each device in the Cisco FabricPath domain must have a configured Cisco FabricPath switch ID. This step is optional, but you should configure a switch ID rather than allow a default configuration, which can be hard to comprehend and troubleshoot later. The recommended approach for switch ID configuration is to use single-digit numbers for the spine switches, double-digit numbers for the leaf switches, and four-digit numbers for the virtual switch IDs on vPC+.

fabricpath switch-id 1

You also need to configure is the Cisco FabricPath priority for assigning the root for the multidestination tree. The recommended approach is to configure two spine switches as the primary and secondary roots for multidestination trees.

fabricpath domain default
root-priority 255

Note: This entire basic configuration will not cause any downtime on the network.

Scenario 2: Configure Cisco FabricPath Spine

Cisco Nexus 7000 F2-Series line card ports must be placed in separate VDCs and configured in FabricPath mode. You do not need to use vPC+ on these switches because all connections will be configured for Cisco FabricPath.

The spine configuration for scenario 2 starts exactly the same way as it started in scenario 1. It will differ when you start expanding the Cisco FabricPath domain. The spine will be expanded with the Cisco Nexus 7000 F2-Series VDC used for spine switches.

As in scenario 1, you will be building a Cisco FabricPath core on the peer link of the spine. That core will be the first small part of the Cisco FabricPath domain you are building. You need to change the peer-link interfaces to FabricPath mode. Then you need to change all VLANs that must be able to cross this Cisco FabricPath link to FabricPath mode. This mode will allow traffic for configured VLANs to cross the Cisco FabricPath domain.

The last part of this step will change vPC to vPC+ by adding a Cisco FabricPath switch ID to the vPC domain configuration. That change will create a virtual entity for the vPC domain to be represented in the Cisco FabricPath domain.

/* Turn peer-link into fabricpath
interface port-channel 1
switchport mode fabricpath
/* Change vPC to vPC+
vpc domain 11
fabric switch-id 1011
/* Configure fabricpath VLANs
vlan 1,10-19,100,200,1010-1599
mode fabricpath

At the same time, you need to configure spanning tree so that spine switches are at the spanning tree root. If you skip this step, all ports will come up, but traffic will be blocked by spanning tree.

/* Configure fabricpath edge as spanning tree root
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096

At this point, the Cisco FabricPath domain is limited to the peer link on the spine switches. The Cisco Nexus 5500 platform switch will not see any difference. It will be a vPC connection to the Cisco Nexus 7000 Series Switch. On the Cisco Nexus 7000 Series Switch, it will be vPC+, so you have the vPC domain connected to the vPC+ domain.

Note: Changing vPC to vPC+ requires flapping all vPCs. This process is disruptive.

The next step is to expand the Cisco FabricPath domain and add spine switches built with Cisco Nexus 7000 F2-Series ports. This step does not require downtime because no production devices are connected to the new spine switches. In addition, new Cisco FabricPath devices in the Cisco FabricPath domain do not affect other devices in any way.

/* On M1/F1 VDC
/* Expand spine to F2 interfaces
interface Ethernet3/7-8
switchport mode fabricpath
/* On the F2 VDC
/* Expand spine to F2 interfaces
interface Ethernet 8/1
switchport mode fabricpath

Scenario 2: Configure Cisco FabricPath Leaf

The next steps will need to be repeated as many times as you have access layer switch pairs (if you are dealing with access-layer vPC domains). In other words, you will start expanding the Cisco FabricPath domain from the spine to the leaf by moving one leaf at a time from the Cisco Nexus 7000 M1-Series or F1-Series port connection to the Cisco Nexus 7000 F2-Series port connection.

In this step, disruption or downtime will affect only a pair of access switches and all fabric extenders connected to the pair. This migration will be the same as the migration on the Cisco Nexus 7000 F2-Series port spine with only one difference: that of changing physical connections from Cisco Nexus 7000 M1-Series or F1-Series to Cisco Nexus 7000 F2-Series ports.

Cisco Nexus 5548UP Switches need to connect to the spine: that is, to Cisco Nexus 7000 F2-Series ports. After that occurs, you need to change the peer link to FabricPath mode, change VLAN mode to FabricPath mode, and change vPC to vPC+, and the connection to Cisco Nexus 7000 F2-Series ports will be brought up.

On both the spine and leaf, you need to remove ports from the vPC and configure ports in FabricPath mode. The vPC must be deleted, but that can be done after migration is complete as part of configuration cleanup.

For scenario 2, you also need to move uplinks from Cisco Nexus 7000 F1-Series ports to Cisco Nexus 7000 F2-Series ports to make sure that you have enough links from the spine to the leaf.

/* On the F2 spine: configure ports in fabricpath mode
interface Ethernet 8/1-2
switchport mode fabricpath
/* On the leaf: Turn peer-link into fabricpath
interface port-channel 1
switchport mode fabricpath
/* On the leaf: Change vPC to vPC+
vpc domain 112
fabric switch-id 1112
/* On the leaf: Configure fabricpath VLANs
vlan 1,10-19,100,200,1010-1599
mode fabricpath
/* Configure fabricpath edge as spanning tree root
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096
mode fabricpath
/* On the leaf: change vPC connection to fabricpath
interface Ethernet 1/9-10
switchport mode fabricpath
/* Cleanup configuration after successful validation
/* On the F1 leaf: remove vPC configuration to access pair
interface Ethernet 3/1-2
no channel-group 112 mode active
no interface port-channel 112
/* On the leaf: remove vPC configuration for uplinks to F1 ports
interface Ethernet 1/1-2
no channel-group 112 mode active
no interface port-channel 112

Scenario 2: Migration Validation

The commands shown here are used to validate successful completion of migration.

show vpc brief - brief display of vPC status
show interface brief - verify interface status
show fabricpath switch-id - show fabricpath switch-id
show fabricpath isis adjacency - display IS-IS adjacency information
show fabricpath route - show FabricPath route information
show spanning-tree - verify spanning tree protocol status
show spanning-tree vlan 100 - verify spanning tree status for VLAN

Scenario 2: Verify vPC and vPC+ Status After Configuring Cisco FabricPath Spine on Peer Link

You need to verify the vPC status at a few different points of the migration process. First, you need to verify that the spine has vPC+ and that the leaf has vPC configured properly.

When vPC is converted to vPC+ in the Cisco FabricPath domain, you need to verify that vPC+ is in the up state.

N7K1-1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 11
vPC+ switch id : 1011
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
vPC fabricpath status : peer is reachable through fabricpath
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 2
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Fabricpath load balancing : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1,10-19,100,200,1010-1599
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans vPC+ Attribute
-- ---- ------ ----------- ------ ------------ --------------
112 Po112 up success success 1,10-19,100,2 DF: Yes, FP
00,1010-1599 MAC:
1011.11.4513
134 Po134 up success success 1,10-19,100,2 DF: Yes, FP
00,1010-1599 MAC:
1011.12.4513

When vPC is connected to vPC+ on the leaf (fabric extender connections), you need to verify that vPC is in the up state.

N5548UP-2(config)# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 112
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary, operational primary
Number of vPCs configured : 67
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1,10-19,100,200,1010-1599
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
101 Po101 up success success -
102 Po102 up success success -
112 Po112 up success success 1,10-19,100
,200,1010-1
599
102400 Eth101/1/1 up success success 1,10-19,100
,200,1010-1
599

Scenario 2: Verify Cisco FabricPath Configuration on the Leaf

When all vPC links between the spine and the leaf are converted to FabricPath mode, the only vPC+ you will see in the up state will be the peer link. You can verify the status of the link between switches using the command shown here.

N5548UP-2(config-if)# show interface brief
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
Eth1/1 1 eth trunk down Administratively down 10G(D) 112
Eth1/2 1 eth trunk down Administratively down 10G(D) 112
Eth1/3 1 eth f-path up none 10G(D) 1
Eth1/4 1 eth f-path up none 10G(D) 1
Eth1/5 1 eth access down SFP not inserted 10G(D) --
Eth1/6 1 eth access down SFP not inserted 10G(D) --
Eth1/7 1 eth trunk down SFP not inserted 10G(D) --
Eth1/8 1 eth trunk down SFP not inserted 10G(D) --
Eth1/9 1 eth f-path up none 10G(D) --
Eth1/10 1 eth f-path up none 10G(D) --
Eth1/11 1 eth trunk down SFP not inserted 10G(D) --
Eth1/12 1 eth trunk down SFP not inserted 10G(D) --
Eth1/13 1 eth trunk down SFP not inserted 10G(D) --
Eth1/14 1 eth trunk down SFP not inserted 10G(D) --
Eth1/15 1 eth trunk down SFP not inserted 10G(D) --
Eth1/16 1 eth trunk down SFP not inserted 10G(D) --
Eth1/17 1 eth fabric up none 10G(D) 101
Eth1/18 1 eth fabric up none 10G(D) 102
Eth1/19 1 eth trunk down SFP not inserted 10G(D) --
Eth1/20 1 eth trunk down SFP not inserted 10G(D) --

Scenario 2: Verify Cisco FabricPath Configuration

The show fabricpath switch-id command displays a list of the switches included in the Cisco FabricPath domain.

N7K1-1# show fabricpath switch-id
FABRICPATH SWITCH-ID TABLE
Legend: '*' - this system
=========================================================================
SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED
----------+----------------+------------+-----------+--------------------
*1 001b.54c2.1cc1 Primary Confirmed Yes No
2 001b.54c2.1e41 Primary Confirmed Yes No
11 547f.ee52.87bc Primary Confirmed Yes No
12 547f.ee29.4ec1 Primary Confirmed Yes No
13 547f.ee24.4381 Primary Confirmed Yes No
14 547f.ee04.023c Primary Confirmed Yes No
21 001b.54c2.1cc3 Primary Confirmed Yes No
22 001b.54c2.1e43 Primary Confirmed Yes No
1011 001b.54c2.1cc1 Primary Confirmed No Yes
1011 001b.54c2.1e41 Primary Confirmed No Yes
1112 547f.ee29.4ec1 Primary Confirmed No Yes
1112 547f.ee52.87bc Primary Confirmed No Yes
1134 547f.ee04.023c Primary Confirmed No Yes
1134 547f.ee24.4381 Primary Confirmed No Yes
Total Switch-ids: 14

The show fabricpath isis adjacency command displays a list of the directly connected switches in the Cisco FabricPath domain.

N7K1-1# show fabricpath isis adjacency
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
N7K2-1 N/A 1 UP 00:00:27 port-channel1
N7K1-3 N/A 1 UP 00:00:28 Ethernet3/7
N7K2-3 N/A 1 UP 00:00:28 Ethernet3/8

The Cisco FabricPath routing table shows Layer 2 routes to reach Cisco FabricPath switch identified by the switchID.

N7K1-1# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default
0/1/0, number of next-hops: 0
via ---- , [60/0], 21 day/s 13:13:41, local
0/1011/11, number of next-hops: 1
via Po112, [80/0], 12 day/s 15:47:30, vpcm
1/2/0, number of next-hops: 1
via Po1, [115/20], 13 day/s 06:07:14, isis_fabricpath-default
1/11/0, number of next-hops: 2
via Eth3/7, [115/80], 2 day/s 14:53:05, isis_fabricpath-default
via Eth3/8, [115/80], 2 day/s 14:53:05, isis_fabricpath-default
1/12/0, number of next-hops: 2
via Eth3/7, [115/80], 2 day/s 14:53:05, isis_fabricpath-default
via Eth3/8, [115/80], 2 day/s 14:52:51, isis_fabricpath-default
1/13/0, number of next-hops: 2
via Eth3/7, [115/80], 2 day/s 14:54:08, isis_fabricpath-default
via Eth3/8, [115/80], 2 day/s 14:54:08, isis_fabricpath-default
1/14/0, number of next-hops: 2
via Eth3/7, [115/80], 2 day/s 14:54:08, isis_fabricpath-default
via Eth3/8, [115/80], 2 day/s 14:53:51, isis_fabricpath-default
1/21/0, number of next-hops: 1
via Eth3/7, [115/40], 2 day/s 15:39:00, isis_fabricpath-default
1/22/0, number of next-hops: 1
via Eth3/8, [115/40], 2 day/s 15:38:36, isis_fabricpath-default
1/1011/0, number of next-hops: 0
via ---- , [60/0], 13 day/s 06:57:42, local
1/1112/0, number of next-hops: 2
via Eth3/7, [115/80], 2 day/s 14:53:05, isis_fabricpath-default
via Eth3/8, [115/80], 2 day/s 14:53:05, isis_fabricpath-default
1/1134/0, number of next-hops: 2
via Eth3/7, [115/80], 2 day/s 14:54:08, isis_fabricpath-default
via Eth3/8, [115/80], 2 day/s 14:54:08, isis_fabricpath-default
2/1011/0, number of next-hops: 0
via ---- , [60/0], 13 day/s 06:57:42, local

Verify the Spanning Tree Protocol status on the switch that does not have any spanning tree domains connected to the Cisco FabricPath domain.

N7K1-1# show spanning-tree
No spanning tree instance exists.

Verify the Spanning Tree Protocol status for the particular VLAN on the edge of the Cisco FabricPath domain. This VLAN must not be in the blocking (BLK) state.

N7K1-1# show spanning-tree vlan 100
VLAN0100
Spanning tree enabled protocol rstp
Root ID Priority 4196
Address c84c.75fa.6000
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 4196 (priority 4096 sys-id-ext 100)
Address c84c.75fa.6000
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po112 Desg FWD 1 128.4207 (vPC) P2p

The following listing shows output when Cisco FabricPath is not the root and traffic is blocked for this VLAN.

N7K2-1(config)# show spanning-tree vlan 100
VLAN0100
Spanning tree enabled protocol rstp
Root ID Priority 32868
Address c84c.75fa.6000
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32868 (priority 32768 sys-id-ext 100)
Address c84c.75fa.6000
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po112 Desg BKN*1 128.4207 (vPC) P2p *L2GW_Inc

Troubleshooting

Troubleshooting Cisco FabricPath is a straightforward process. You need to verify that each switch can see other switches as adjacent switches, and that each switch is unique in the Cisco FabricPath domain. You also need to verify that Cisco FabricPath routes are in place. You may need to verify the topology for the Cisco FabricPath domain.

Table 4 describes the main troubleshooting commands.

Table 4. Troubleshooting Commands and Descriptions

Troubleshooting Commands and Descriptions

Troubleshooting Command

Description

show license usage

Verify that the correct license is installed

show feature-set

Verify that Cisco FabricPath is enabled

show vlan

Verify that FabricPath mode is enabled

show run fabricpath

Check the Cisco FabricPath configuration

Show vpc

Verify the peer-link mode and vPC status

show fabricpath switch-id

Display a list of switches in the Cisco FabricPath domain

show fabricpath isis interface [brief]

Verify that interfaces are up and forwarding

show fabricpath isis adjacency

Display a list of adjacent Cisco FabricPath devices

show fabricpath route

Display a list of Layer 2 Cisco FabricPath routes

show tech fabricpath

Get information for opening a case with the Cisco Technical Assistance Center (TAC)

show interface ethernet x/y capabilities

Check port information

pong dest <sw#> dest <mac-address> vlan <vlan> count <#> … [detail]

Get hop-by-hop latency measurements

Sample troubleshooting command output from actual completed testing is shown here.

N7K1-1# show feature-set
Feature Set Name ID State
-------------------- -------- --------
fcoe 1 uninstalled
fabricpath 2 enabled
fex 3 uninstalled
mpls 4 uninstalled

N7K1-1# show fabricpath isis adjacency
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID SNPA Level State Hold Time Interface
N7K2-1 N/A 1 UP 00:00:27 port-channel1
N7K1-3 N/A 1 UP 00:00:28 Ethernet3/7
N7K2-3 N/A 1 UP 00:00:28 Ethernet3/8

N7K1-1# show fabricpath switch-id
FABRICPATH SWITCH-ID TABLE
Legend: '*' - this system
=========================================================================
SWITCH-ID SYSTEM-ID FLAGS STATE STATIC EMULATED
----------+----------------+------------+-----------+--------------------
*1 001b.54c2.1cc1 Primary Confirmed Yes No
2 001b.54c2.1e41 Primary Confirmed Yes No
11 547f.ee52.87bc Primary Confirmed Yes No
12 547f.ee29.4ec1 Primary Confirmed Yes No
13 547f.ee24.4381 Primary Confirmed Yes No
14 547f.ee04.023c Primary Confirmed Yes No
21 001b.54c2.1cc3 Primary Confirmed Yes No
22 001b.54c2.1e43 Primary Confirmed Yes No
1011 001b.54c2.1cc1 Primary Confirmed No Yes
1011 001b.54c2.1e41 Primary Confirmed No Yes
1112 547f.ee29.4ec1 Primary Confirmed No Yes
1112 547f.ee52.87bc Primary Confirmed No Yes
1134 547f.ee04.023c Primary Confirmed No Yes
1134 547f.ee24.4381 Primary Confirmed No Yes
Total Switch-ids: 14

References

Cisco FabricPath for Cisco Nexus 7000 Series Switches
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-687554.html

Scale Data Centers with Cisco FabricPath
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-605488.html

Cisco FabricPath Design Guide: Using Cisco FabricPath with an Aggregation and Access Topology
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/guide_c07-690079.html

Appendix A: Configuration Before Migration

To keep the configurations shown here to manageable sizes, the listings do not include the full SVI configuration. There were a total of 600 VLANs, and 20 were created for testing. The configuration shown here demonstrates the configuration for two SVIs only.

N7K1-1#
feature lacp
feature vpc
feature interface-vlan
hostname N7K1-1
interface mgmt0
ip address 172.21.55.179/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
vpc domain 11
peer-switch
role priority 1024
system-priority 1024
peer-keepalive destination 172.21.55.183 source 172.21.55.179
peer-gateway
auto-recovery
ip arp synchronize
interface Vlan1
ip address 10.1.1.2/24
hsrp version 2
hsrp 1
preempt
priority 150
ip 10.1.1.1
no shutdown
interface Vlan11
ip address 10.1.11.2/24
hsrp version 2
hsrp 11
preempt
priority 150
ip 10.1.11.1
no shutdown
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode trunk
vpc peer-link
spanning-tree port type network
interface Ethernet1/1
description : M1 port - peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface Ethernet1/2
description : M1 port - peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface port-channel 112
description : link to N5548UP-1 and N5548UP-2
switchport
switchport mode trunk
vpc 112
interface Ethernet3/1
description : link to N5548UP-1
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
interface Ethernet3/2
description : link to N5548UP-2
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
interface port-channel 134
description : link to N5548UP-3 and N5548UP-4
switchport
switchport mode trunk
vpc 134
interface Ethernet3/3
description : link to N5548UP-3
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
interface Ethernet3/4
description : link to N5548UP-4
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
N5548UP-1#
feature vpc
feature lacp
feature fex
interface mgmt0
ip address 172.21.55.171/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
vpc domain 112
peer-keepalive destination 172.21.55.172 source 172.21.55.171 vrf management
reload restore
role priority 1024
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode trunk
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface port-channel 112
description : uplink to N7K1-1 and N7K2-1
switchport
switchport mode trunk
vpc 112
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
fex 101
pinning max-links 1
description "FEX-101"
interface port-channel101
description FEX-101
vpc 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/17
description FEX-101
switchport mode fex-fabric
fex associate 101
channel-group 101
fex 102
pinning max-links 1
description "FEX-102"
interface port-channel102
description FEX-102
vpc 102
switchport mode fex-fabric
fex associate 102
interface Ethernet1/18
description FEX-102
switchport mode fex-fabric
fex associate 102
channel-group 102
interface Ethernet101/1/1
description : Test traffic generator
switchport mode trunk
N5548UP-3#
feature vpc
feature lacp
feature fex
interface mgmt0
ip address 172.21.55.173/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
vpc domain 134
peer-keepalive destination 172.21.55.174 source 172.21.55.173 vrf management
reload restore
role priority 1024
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode trunk
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface port-channel 134
description : uplink to N7K1-1 and N7K2-1
switchport
switchport mode trunk
vpc 134
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
fex 103
pinning max-links 1
description "FEX-103"
interface port-channel103
description FEX-103
vpc 103
switchport mode fex-fabric
fex associate 103
interface Ethernet 1/17
description FEX-103
fex associate 103
switchport mode fex-fabric
channel-group 103
fex 104
pinning max-links 1
description "FEX-104"
interface port-channel104
description FEX-104
vpc 104
switchport mode fex-fabric
fex associate 104
interface Ethernet 1/18
description FEX-104
fex associate 104
switchport mode fex-fabric
channel-group 104
interface Ethernet104/1/1
description : Test traffic generator
switchport mode trunk
N7K2-1#
feature lacp
feature vpc
feature interface-vlan
hostname N7K1-1
interface mgmt0
ip address 172.21.55.183/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
vpc domain 11
peer-switch
role priority 2048
system-priority 1024
peer-keepalive destination 172.21.55.179 source 172.21.55.183
peer-gateway
auto-recovery
ip arp synchronize
interface Vlan1
ip address 10.1.1.3/24
hsrp version 2
hsrp 1
preempt
ip 10.1.1.1
no shutdown
interface Vlan11
ip address 10.1.11.3/24
hsrp version 2
hsrp 11
preempt
ip 10.1.11.1
no shutdown
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode trunk
vpc peer-link
spanning-tree port type network
interface Ethernet1/1
description : M1 port - peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface Ethernet1/2
description : M1 port - peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface port-channel 112
description : link to N5548UP-1 and N5548UP-2
switchport
switchport mode trunk
vpc 112
interface Ethernet3/1
description : link to N5548UP-1
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
interface Ethernet3/2
description : link to N5548UP-2
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
interface port-channel 134
description : link to N5548UP-3 and N5548UP-4
switchport
switchport mode trunk
vpc 134
interface Ethernet3/3
description : link to N5548UP-3
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
interface Ethernet3/4
description : link to N5548UP-4
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
N5548UP-2#
feature vpc
feature lacp
feature fex
interface mgmt0
ip address 172.21.55.172/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
vpc domain 112
peer-keepalive destination 172.21.55.171 source 172.21.55.172 vrf management
reload restore
role priority 2048
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode trunk
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface port-channel 112
description : uplink to N7K1-1 and N7K2-1
switchport
switchport mode trunk
vpc 112
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode trunk
channel-group 112 mode active
no shutdown
fex 101
pinning max-links 1
description "FEX-101"
interface port-channel101
description FEX-101
vpc 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/17
description FEX-101
switchport mode fex-fabric
fex associate 101
channel-group 101
fex 102
pinning max-links 1
description "FEX-102"
interface port-channel102
description FEX-102
vpc 102
switchport mode fex-fabric
fex associate 102
interface Ethernet1/18
description FEX-102
switchport mode fex-fabric
fex associate 102
channel-group 102
interface Ethernet101/1/1
description : Test traffic generator
switchport mode trunk
N5548UP-4#
feature vpc
feature lacp
feature fex
interface mgmt0
ip address 172.21.55.174/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
vpc domain 134
peer-keepalive destination 172.21.55.173 source 172.21.55.174 vrf management
reload restore
role priority 2048
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode trunk
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode trunk
channel-group 1 mode active
no shutdown
interface port-channel 134
description : uplink to N7K1-1 and N7K2-1
switchport
switchport mode trunk
vpc 134
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode trunk
channel-group 134 mode active
no shutdown
fex 103
pinning max-links 1
description "FEX-103"
interface port-channel103
description FEX-103
vpc 103
switchport mode fex-fabric
fex associate 103
interface Ethernet 1/17
description FEX-103
fex associate 103
switchport mode fex-fabric
channel-group 103
fex 104
pinning max-links 1
description "FEX-104"
interface port-channel104
description FEX-104
vpc 104
switchport mode fex-fabric
fex associate 104
interface Ethernet 1/18
description FEX-104
fex associate 104
switchport mode fex-fabric
channel-group 104
interface Ethernet104/1/1
description : Test traffic generator
switchport mode trunk

Appendix B: Configuration after Migration for Case 1.

N7K1-1#
feature lacp
feature vpc
feature interface-vlan
install feature-set fabricpath
feature-set fabricpath
hostname N7K1-1
interface mgmt0
ip address 172.21.55.179/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096
vpc domain 11
peer-switch
role priority 1024
system-priority 1024
peer-keepalive destination 172.21.55.183 source 172.21.55.179
peer-gateway
auto-recovery
fabric switch-id 1011
ip arp synchronize
interface Vlan1
ip address 10.1.1.2/24
hsrp version 2
hsrp 1
preempt
priority 150
ip 10.1.1.1
no shutdown
interface Vlan11
ip address 10.1.11.2/24
ip proxy-arp
hsrp version 2
hsrp 11
preempt
priority 150
ip 10.1.11.1
no shutdown
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet3/5
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet3/6
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet3/1
description : link to N5548UP-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/2
description : link to N5548UP-2
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/3
description : link to N5548UP-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/4
description : link to N5548UP-4
switchport
switchport mode fabricpath
no shutdown
fabricpath switch-id 1
fabricpath domain default
root-priority 255
N5548UP-1#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.171/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 112
peer-keepalive destination 172.21.55.172 source 172.21.55.171 vrf management
reload restore
role priority 1024
fabric switch-id 1112
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode fabricpath
no shutdown
fex 101
pinning max-links 1
description "FEX-101"
interface port-channel101
description FEX-101
vpc 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/17
description FEX-101
switchport mode fex-fabric
fex associate 101
channel-group 101
fex 102
pinning max-links 1
description "FEX-102"
interface port-channel102
description FEX-102
vpc 102
switchport mode fex-fabric
fex associate 102
interface Ethernet1/18
description FEX-102
switchport mode fex-fabric
fex associate 102
channel-group 102
interface Ethernet101/1/1
switchport mode trunk
fabricpath switch-id 11
N5548UP-3#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.173/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 134
peer-keepalive destination 172.21.55.174 source 172.21.55.173 vrf management
reload restore
role priority 1024
fabric switch-id 1134
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode fabricpath
no shutdown
fex 103
pinning max-links 1
description "FEX-103"
interface port-channel103
description FEX-103
vpc 103
switchport mode fex-fabric
fex associate 103
interface Ethernet 1/17
description FEX-103
fex associate 103
switchport mode fex-fabric
channel-group 103
fex 104
pinning max-links 1
description "FEX-104"
interface port-channel104
description FEX-104
vpc 104
switchport mode fex-fabric
fex associate 104
interface Ethernet 1/18
description FEX-104
fex associate 104
switchport mode fex-fabric
channel-group 104
interface Ethernet104/1/1
switchport mode trunk
fabricpath switch-id 13
N7K2-1#
feature lacp
feature vpc
feature interface-vlan
install feature-set fabricpath
feature-set fabricpath
hostname N7K1-1
interface mgmt0
ip address 172.21.55.183/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096
vpc domain 11
peer-switch
role priority 2048
system-priority 1024
peer-keepalive destination 172.21.55.179 source 172.21.55.183
peer-gateway
auto-recovery
fabric switch-id 1011
ip arp synchronize
interface Vlan1
ip address 10.1.1.3/24
hsrp version 2
hsrp 1
preempt
ip 10.1.1.1
no shutdown
interface Vlan11
ip address 10.1.11.3/24
ip proxy-arp
hsrp version 2
hsrp 11
preempt
ip 10.1.11.1
no shutdown
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet3/5
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
shutdown
interface Ethernet3/6
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
shutdown
interface Ethernet3/1
description : link to N5548UP-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/2
description : link to N5548UP-2
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/3
description : link to N5548UP-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/4
description : link to N5548UP-4
switchport
switchport mode fabricpath
no shutdown
fabricpath switch-id 2
fabricpath domain default
root-priority 254
N5548UP-2#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.172/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 112
peer-keepalive destination 172.21.55.171 source 172.21.55.172 vrf management
reload restore
role priority 2048
fabric switch-id 1112
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode fabricpath
no shutdown
fex 101
pinning max-links 1
description "FEX-101"
interface port-channel101
description FEX-101
vpc 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/17
description FEX-101
switchport mode fex-fabric
fex associate 101
channel-group 101
fex 102
pinning max-links 1
description "FEX-102"
interface port-channel102
description FEX-102
vpc 102
switchport mode fex-fabric
fex associate 102
interface Ethernet1/18
description FEX-102
switchport mode fex-fabric
fex associate 102
channel-group 102
interface Ethernet101/1/1
switchport mode trunk
fabricpath switch-id 12
N5548UP-4#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.174/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 134
peer-keepalive destination 172.21.55.173 source 172.21.55.174 vrf management
reload restore
role priority 2048
fabric switch-id 1134
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/1
description : uplink to N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/2
description : uplink to N7K2-1
switchport
switchport mode fabricpath
no shutdown
fex 103
pinning max-links 1
description "FEX-103"
interface port-channel103
description FEX-103
vpc 103
switchport mode fex-fabric
fex associate 103
interface Ethernet 1/17
description FEX-103
fex associate 103
switchport mode fex-fabric
channel-group 103
fex 104
pinning max-links 1
description "FEX-104"
interface port-channel104
description FEX-104
vpc 104
switchport mode fex-fabric
fex associate 104
interface Ethernet 1/18
description FEX-104
fex associate 104
switchport mode fex-fabric
channel-group 104
interface Ethernet104/1/1
switchport mode trunk
fabricpath switch-id 14

Appendix C: Configuration after Migration for Scenario 2

N7K1-1#
feature lacp
feature vpc
feature interface-vlan
install feature-set fabricpath
feature-set fabricpath
hostname N7K1-1
interface mgmt0
ip address 172.21.55.179/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096
vpc domain 11
peer-switch
role priority 1024
system-priority 1024
peer-keepalive destination 172.21.55.183 source 172.21.55.179
peer-gateway
auto-recovery
fabric switch-id 1011
ip arp synchronize
interface Vlan1
ip address 10.1.1.2/24
hsrp version 2
hsrp 1
preempt
priority 150
ip 10.1.1.1
no shutdown
interface Vlan11
ip address 10.1.11.2/24
ip proxy-arp
hsrp version 2
hsrp 11
preempt
priority 150
ip 10.1.11.1
no shutdown
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet3/5
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet3/6
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet3/1
description : link to N5548UP-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/2
description : link to N5548UP-2
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/3
description : link to N5548UP-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/4
description : link to N5548UP-4
switchport
switchport mode fabricpath
no shutdown
fabricpath switch-id 1
fabricpath domain default
root-priority 255
N7K1-3#
feature lacp
feature vpc
install feature-set fabricpath
feature-set fabricpath
hostname N7K1-3
interface mgmt0
ip address 172.21.55.181/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
interface Ethernet8/1
description To 5548-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/2
description To 5548-2
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/3
description To 5548-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/4
description To 5548-4
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/5
description To N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/6
description To N7K2-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/7
description To N7K2-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/8
description To N7K2-3
switchport
switchport mode fabricpath
no shutdown
fabricpath domain default
fabricpath switch-id 21
N5548UP-1#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.171/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 112
peer-keepalive destination 172.21.55.172 source 172.21.55.171 vrf management
reload restore
role priority 1024
fabric switch-id 1112
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/9
description : uplink to N7K1-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/10
description : uplink to N7K2-3
switchport
switchport mode fabricpath
no shutdown
fex 101
pinning max-links 1
description "FEX-101"
interface port-channel101
description FEX-101
vpc 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/17
description FEX-101
switchport mode fex-fabric
fex associate 101
channel-group 101
fex 102
pinning max-links 1
description "FEX-102"
interface port-channel102
description FEX-102
vpc 102
switchport mode fex-fabric
fex associate 102
interface Ethernet1/18
description FEX-102
switchport mode fex-fabric
fex associate 102
channel-group 102
interface Ethernet101/1/1
switchport mode trunk
fabricpath switch-id 11
N5548UP-3#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.173/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 134
peer-keepalive destination 172.21.55.174 source 172.21.55.173 vrf management
reload restore
role priority 1024
fabric switch-id 1134
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/9
description : uplink to N7K1-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/10
description : uplink to N7K2-3
switchport
switchport mode fabricpath
no shutdown
fex 103
pinning max-links 1
description "FEX-103"
interface port-channel103
description FEX-103
vpc 103
switchport mode fex-fabric
fex associate 103
interface Ethernet 1/17
description FEX-103
fex associate 103
switchport mode fex-fabric
channel-group 103
fex 104
pinning max-links 1
description "FEX-104"
interface port-channel104
description FEX-104
vpc 104
switchport mode fex-fabric
fex associate 104
interface Ethernet 1/18
description FEX-104
fex associate 104
switchport mode fex-fabric
channel-group 104
interface Ethernet104/1/1
switchport mode trunk
fabricpath switch-id 13
N7K2-1#
feature lacp
feature vpc
feature interface-vlan
install feature-set fabricpath
feature-set fabricpath
hostname N7K1-1
interface mgmt0
ip address 172.21.55.183/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 4096
vpc domain 11
peer-switch
role priority 2048
system-priority 1024
peer-keepalive destination 172.21.55.179 source 172.21.55.183
peer-gateway
auto-recovery
fabric switch-id 1011
ip arp synchronize
interface Vlan1
ip address 10.1.1.3/24
hsrp version 2
hsrp 1
preempt
ip 10.1.1.1
no shutdown
interface Vlan11
ip address 10.1.11.3/24
ip proxy-arp
hsrp version 2
hsrp 11
preempt
ip 10.1.11.1
no shutdown
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet3/5
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
shutdown
interface Ethernet3/6
description : F1 port - peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
shutdown
interface Ethernet3/1
description : link to N5548UP-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/2
description : link to N5548UP-2
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/3
description : link to N5548UP-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet3/4
description : link to N5548UP-4
switchport
switchport mode fabricpath
no shutdown
fabricpath switch-id 2
fabricpath domain default
root-priority 254
N7K2-3#
feature lacp
feature vpc
install feature-set fabricpath
feature-set fabricpath
hostname N7K2-3
interface mgmt0
ip address 172.21.55.185/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
interface Ethernet8/1
description To 5548-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/2
description To 5548-2
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/3
description To 5548-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/4
description To 5548-4
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/5
description To N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/6
description To N7K2-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/7
description To N7K1-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet8/8
description To N7K1-3
switchport
switchport mode fabricpath
no shutdown
fabricpath domain default
fabricpath switch-id 22
N5548UP-2#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.172/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 112
peer-keepalive destination 172.21.55.171 source 172.21.55.172 vrf management
reload restore
role priority 2048
fabric switch-id 1112
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/9
description : uplink to N7K1-3
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/10
description : uplink to N7K2-3
switchport
switchport mode fabricpath
no shutdown
fex 101
pinning max-links 1
description "FEX-101"
interface port-channel101
description FEX-101
vpc 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/17
description FEX-101
switchport mode fex-fabric
fex associate 101
channel-group 101
fex 102
pinning max-links 1
description "FEX-102"
interface port-channel102
description FEX-102
vpc 102
switchport mode fex-fabric
fex associate 102
interface Ethernet1/18
description FEX-102
switchport mode fex-fabric
fex associate 102
channel-group 102
interface Ethernet101/1/1
switchport mode trunk
fabricpath switch-id 12
N5548UP-4#
feature vpc
feature lacp
feature fex
install feature-set fabricpath
feature-set fabricpath
interface mgmt0
ip address 172.21.55.174/24
vrf context management
ip route 0.0.0.0/0 172.21.55.254
vlan 1,10-19,100,200,1010-1599
mode fabricpath
spanning-tree vlan 1,10-19,100,200,1010-1599 priority 8192
vpc domain 134
peer-keepalive destination 172.21.55.173 source 172.21.55.174 vrf management
reload restore
role priority 2048
fabric switch-id 1134
interface port-channel 1
description VPC Peer-Link
switchport
switchport mode fabricpath
vpc peer-link
spanning-tree port type network
interface Ethernet1/3
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/4
description peer-link member
switchport
switchport mode fabricpath
channel-group 1 mode active
no shutdown
interface Ethernet1/9
description : uplink to N7K1-1
switchport
switchport mode fabricpath
no shutdown
interface Ethernet1/10
description : uplink to N7K2-1
switchport
switchport mode fabricpath
no shutdown
fex 103
pinning max-links 1
description "FEX-103"
interface port-channel103
description FEX-103
vpc 103
switchport mode fex-fabric
fex associate 103
interface Ethernet 1/17
description FEX-103
fex associate 103
switchport mode fex-fabric
channel-group 103
fex 104
pinning max-links 1
description "FEX-104"
interface port-channel104
description FEX-104
vpc 104
switchport mode fex-fabric
fex associate 104
interface Ethernet 1/18
description FEX-104
fex associate 104
switchport mode fex-fabric
channel-group 104
interface Ethernet104/1/1
switchport mode trunk
fabricpath switch-id 14