THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense.
The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy. If it is not installed in accordance with Cisco's installation instructions, it may cause interference with radio and television reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation. However, there is no guarantee that interference will not occur in a particular installation.
You can determine whether your equipment is causing interference by turning it off. If the interference stops, it was probably caused by the Cisco equipment or one of its peripheral devices. If the equipment causes interference to radio or television reception, try to correct the interference by using one or more of the following measures:
Turn the television or radio antenna until the interference stops.
Move the equipment to one side or the other of the television or radio.
Move the equipment farther away from the television or radio.
Plug the equipment into an outlet that is on a different circuit from the television or radio. (That is, make certain the equipment and the television or radio are on circuits controlled by different circuit breakers or fuses.). Modifications to this product not authorized by Cisco Systems, Inc. could void the FCC approval and negate your authority to operate the product.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright Ó 1981, Regents of the University of California.
Network Time Protocol (NTP). Copyright Ó 1992, David L. Mills. The University of Delaware makes no representations about the suitability of this software for any purpose.
Point-to-Point Protocol. Copyright Ó 1989, Carnegie-Mellon University. All rights reserved. The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission.
The Cisco implementation of TN3270 is an adaptation of the TN3270, curses, and termcap programs developed by the University of California, Berkeley (UCB) as part of the UCB's public domain version of the UNIX operating system. All rights reserved. Copyright Ó 1981-1988, Regents of the University of California.
Cisco incorporates Fastmac and TrueView software and the RingRunner chip in some Token Ring products. Fastmac software is licensed to Cisco by Madge Networks Limited, and the RingRunner chip is licensed to Cisco by Madge NV. Fastmac, RingRunner, and TrueView are trademarks and in some jurisdictions registered trademarks of Madge Networks Limited. Copyright Ó 1995, Madge Networks Limited. All rights reserved.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED "AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PRACTICAL PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
AccessPath, AtmDirector, Browse with Me, CCIP, CCSI, CD-PAC, CiscoLink, the Cisco Powered Network logo, Cisco Systems Networking Academy, the Cisco Systems Networking Academy logo, Cisco Unity, Fast Step, Follow Me Browsing, FormShare, FrameShare, IGX, Internet Quotient, IP/VC, iQ Breakthrough, iQ Expertise, iQ FastTrack, the iQ logo, iQ Net Readiness Scorecard, MGX, the Networkers logo, ScriptBuilder, ScriptShare, SMARTnet, TransPath, Voice LAN, Wavelength Router, and WebViewer, Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherSwitch, FastHub, FastSwitch, GigaStack, IOS, IP/TV, LightStream, MICA, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, RateMUX, Registrar, SlideCast, StrataView Plus, Stratm, SwitchProbe, TeleRouter, and VCO are trademarks or registered trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries.
All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0110R).
INTELLECTUAL PROPERTY RIGHTS:
THIS DOCUMENT CONTAINS VALUABLE TRADE SECRETS AND CONFIDENTIAL INFORMATION OF CISCO SYSTEMS, INC. AND IT'S SUPPLIERS, AND SHALL NOT BE DISCLOSED TO ANY PERSON, ORGANIZATION, OR ENTITY UNLESS SUCH DISCLOSURE IS SUBJECT TO THE PROVISIONS OF A WRITTEN NON-DISCLOSURE AND PROPRIETARY RIGHTS AGREEMENT OR INTELLECTUAL PROPERTY LICENSE AGREEMENT APPROVED BY CISCO SYSTEMS, INC. THE DISTRIBUTION OF THIS DOCUMENT DOES NOT GRANT ANY LICENSE IN OR RIGHTS, IN WHOLE OR IN PART, TO THE CONTENT, THE PRODUCT(S), TECHNOLOGY OF INTELLECTUAL PROPERTY DESCRIBED HEREIN.
Copyright Ó 2012, Cisco Systems, Inc.
All rights reserved.
COMMERCIAL IN CONFIDENCE.
Introduction
Document Purpose and Usage
The purpose of this document is to recommend a migration strategy for Cisco® Catalyst® 6500 Series Switches migrating from the Cisco Catalyst 6500 Series Supervisor Engine 720 to the Cisco Catalyst 6500 Series Supervisor Engine 2T in customer production deployments. It is intended to be used by customers and engineers in the field carrying out the migration.
The document includes all the hardware and software prerequisites, end-of-sale/end-of-support deadlines, and detailed migration steps.
The intent is to test a methodology to conduct this upgrade with a minimal downtime.
It also provides a test plan for testing traffic behavior during the upgrade.
About the Authors and Contributors
Faraz Siddiqui
Faraz Siddiqui is a network consulting engineer with Cisco on the Unified Infrastructure team. He has contributed to several documents, white papers, and technical tips related to the data center and campus switching. Faraz has a B.Engg degree in computer and information systems engineering from NED University of Engg and Tech, Pakistan, and an MS degree from Wichita State University. He has been with Cisco Advanced Services since 2011.
Azeem Suleman, Double CCIE 23427
Azeem Suleman is a solutions architect with Cisco on the Unified Infrastructure team. He has written several documents, white papers, and technical tips related to the data center and security. He has been a representative to the Architecture Board and has been designing large-scale enterprise networks for more than a decade.
Azeem has a BS degree in computer engineering from SSUET, Pakistan, and an MS degree from the University of Texas at Austin. He has published in the IEEE Conference publication and has been with Cisco Advanced Services since 2009.
Contributors
• Imran Moulvi network consulting engineer with Cisco on Unified Infrastructure Team
• Vinay Suvarna: network consulting engineer with Cisco on Unified Infrastructure Team
Technical Reviews Conducted by Industry Experts
• Talha Hashmi: manager with Cisco on Unified Infrastructure Architecture Team
• Vivek Baveja: technical marketing engineer with Cisco in Catalyst 6500 Enterprise switching group
• Rolando Salinas: technical marketing engineer with Cisco in Catalyst 6500 Enterprise switching group
Overview
The Supervisor Engine 2T (Supervisor 2-Terabit) is the next-generation supervisor engine of the Cisco Catalyst 6500 family. It is a high-performance platform that increases performance by a factor of 3, scale by a factor of 4, and services above and beyond those of the previous Supervisor Engine 720. With the support of the virtual switching system (VSS), the platform allows two 2-Tbps switches to combine into a 4-Tbps virtual switch.
The Supervisor Engine 2T is a major upgrade to the most widely deployed modular switching platform in the campus and data center networking segment. The new Supervisor Engine 2T's Policy Feature Card 4, or PFC4, increases NetFlow capacity and monitoring capabilities and has a new ternary content-addressable memory (TCAM) design offering improved access control lists (ACLs), quality of service (QoS) design options, encryption security, and many other features.
Benefits to Customers Migrating to Supervisor Engine 2T
• Backward compatibility with older generation of supervisors and line cards
• Scalability and performance improvements such as distributed forwarding (dCEF) (720 mpps) with the fourth-generation policy feature card (PFC4)
• Support for future 40-Gbps module and shipping nonblocking 10-Gbps modules
• New applications and services with hardware-accelerated virtual private LAN service (VPLS) for network virtualization
• Ability to take advantage of integrated connectivity management processor (CMP) for improved out-of-band management
Customer Challenge: Reduce planned and unplanned network downtime.
Supervisor Engine 2T offers:
• 4-Tb VSSs
• VSS: Improves operational efficiency by simplifying network and reducing switch management overhead by 50%
• Maintains high availability during software upgrade using eFSU (< 200ms downtime)
• Generic Online Diagnostics (GOLD): Proactive, on-demand, and scheduled diagnostics framework to detect hardware issues
High-Level Description of Supervisor Engine 2T
The Supervisor Engine 2T is made up of four main physical components:
• The baseboard
• The 5th-generation multilayer switching feature card (MSFC5)
• The 4th-generation policy feature card (PFC4)
• The 2-Tbps fabric connection daughter card (FCDC)
The supervisor baseboard forms the foundation upon which many of the purpose-built daughter cards and other components are placed. It houses a multitude of various application-specific integrated circuits (ASICs), including the ASIC complex that makes up the primary 2-Tb (2080-Gbps) crossbar switch fabric, as well as the port ASICs that control the front panel 10 Gigabit Ethernet and Gigabit Ethernet ports.
The MSFC5 is a daughter card that holds the CPU complex, which serves as the control plane for the switch. The control plane handles the processing of all software-related features. One major difference from earlier versions of the MSFC is that this version combines what were previously two separate CPU complexes into one.
The PFC4 is another daughter card that incorporates a special set of ASICs and memory blocks, which provide hardware-accelerated data-plane services for packets traversing the switch.
The 2-Tbps switch fabric provides 26 dedicated 20-Gbps or 40-Gbps channels to support the new 6513-E chassis (in addition to all existing E-series chassis models). On the Supervisor Engine 720, the switch fabric supported 18 fabric channels, which were used to provide two fabric channels per slot on all slots (with the exception being the 6513 chassis). With the new 6513-E chassis, the 2T switch fabric is capable of supporting dual fabric channels for all line-card slots (slots 7 and 8 are reserved for the active and standby supervisors).
Table 1. Baseboard Features
Feature
Description
Switch fabric type
2080-Gbps (2-Tbps) crossbar switch fabric
Forwarding engine daughter card
PFC4 or PFC4XL
CPU daughter card
MSFC5
Uplink ports
2 x 10GE (X2 optic support)
3 x GE (SFP support)
USB ports
2 x USB (1 x Type A and 1 x Type B)
Management ports
Serial console port (RJ-45)
Connectivity management processor Ethernet port (RJ-45)
Management LED
Blue beacon LED
Media slot
Compact flash slot (Type II)
Forwarding performance
Up to 60 mpps for L2, IPv4, and Multiprotocol Label Switching (MPLS) traffic
Up to 30 mpps for IPv6 traffic
Why Now
• End of support for Cisco Catalyst 6500 Series Supervisor Engines 1A and 2
• End of software maintenance for Cisco Catalyst 6500 Series Supervisor Engine 720-3A
• Upcoming end of support for select Cisco Catalyst 6100, 6200, 6300, and 6500 Series Line Cards
• Upcoming end of support for non-E-series chassis
• Large installed base consisting of Supervisor Engine 1A and Supervisor Engine 2
End-of-Sale/Support Chart for Supervisor Engines
Table 2. End-of-Sale Dates for Supervisor Engines and Chassis
Following are some focus areas to be considered during migration from the Supervisor Engine 720 or earlier to the Supervisor Engine 2T.
1. Chassis
The Supervisor Engine 2T is designed to operate in any E-series 6500 chassis. The Supervisor Engine 2T will not be supported in any of the earlier non-E-series chassis.
Supported Chassis
The Cisco Supervisor Engine 2T is supported only in following E-series chassis:
For the E-series chassis, a corresponding E-series fan (or high-speed [HS] fan) for that chassis is required to support the Supervisor Engine 2T. While the 2500W power supply is the minimum sized power supply that must be used for a 6-slot, 9-slot, or 13-slot chassis supporting the Supervisor Engine 2T, the current minimum "shipping" power supply that is supported is 3000W power supply.
The 6503-E requires a 1400W power supply, and the 6504-E requires the 2700W power supply when a Supervisor Engine 2T is used in each chassis. Either AC or DC power supply options can be used.
Table 3. Supported Chassis, Fan, and Power Supply for Supervisor Engine 2T
Supported Chassis
Supported Fan
Supported Power Supply
6503-E
WS-C6503-E-FAN
PWR-1400-AC
6504-E
FAN-MOD4-HS
PWR-2700-AC/4
PWR-2700-DC/4
6506-E
WS-C6506-E-FAN
WS-CAC-2500W: now end of sale
WS-CDC-2500W
WS-CAC-3000W
WS-CAC-4000W-US
WS-CAC-4000-INT
PWR-4000-DC
WS-CAC-6000W
PWR-6000-DC
WS-CAC-8700W
6509-E
WS-C6509-E-FAN
WS-CAC-2500W: now end of sale
WS-CDC-2500W
WS-CAC-3000W
WS-CAC-4000W-US
WS-CAC-4000-INT
PWR-4000-DC
WS-CAC-6000W
PWR-6000-DC
WS-CAC-8700W
6509-V-E
WS-C6509-V-E-FAN
WS-CAC-2500W: now end of sale
WS-CDC-2500W
WS-CAC-3000W
WS-CAC-4000W-US
WS-CAC-4000-INT
PWR-4000-DC
WS-CAC-6000W
PWR-6000-DC
WS-CAC-8700W
6513-E
WS-C6513-E-FAN
WS-CDC-2500W
WS-CAC-3000W
WS-CAC-4000W-US
WS-CAC-4000-INT
PWR-4000-DC
WS-CAC-6000W
PWR-6000-DC
WS-CAC-8700W
3. Uplinks
The Supervisor Engine 2T has three 1G ports, all supporting Small Form-Factor Pluggable (SFP) transceiver options, including fiber and copper. The Supervisor Engine 2T is also equipped with two 10GB ports supporting IEEE 802.1ae MACsec link encryption. In comparison, the Supervisor Engine 720 has two 1-GB SFP ports and one copper 10/100-Mb port.
4. Line Cards
The Supervisor Engine 2T provides backward compatibility with the existing WS-X6700 series line cards (with the exception of the WS-X6708-10G, which will be replaced by the new WS-X6908-10G), as well as the following select WS-X6100 series line cards only:
• WS-X6148A-RJ-45
• WS-X6148-FE-SFP
• WS-X6148A-GE-TX
• WS-X6148A-GE-45AF
• WS-X6148E-GE-AT
• WS-X6148A-45AF
It is important to note that these line cards are not supported in a Cisco VSS-enabled system. There is no support for the WS-X62xx, WS-X63xx, WS-X64xx, or WS-X65xx line cards.
There is no compatibility between the Supervisor Engine 2T and earlier generations of distributed forwarding cards (that is, DFC, DFC2, or DFC3x). The DFC is used to accelerate forwarding performance for the system as a whole and uses the same forwarding architecture that is found on the PFC. The PFC4 architecture introduces a number of changes that differentiates it significantly in operation from earlier PFC/DFC models.
These changes require that only DFC4 can interoperate with the PFC4. Any existing WS-X67xx line cards that have a DFC3x will have to be upgraded with the new DFC4. WS-X67xx line cards with a DFC4 installed, will function in distributed dCEF720 mode.
Note: Any newly purchased WS-X6700 series line cards that are shipped with a DFC4 or DFC4XL preinstalled have been renamed as WS-X6800 series line cards to clearly separate the performance differences.
Note: Because of compatibility issues, the WS-X6708-10GE-3C/3CXL cannot be inserted in a Supervisor Engine 2T system and must be upgraded to the new WS-X6908-10GE-2T/2TXL.
The Supervisor Engine 2T line-card support also introduces the new WS-X6900 series line cards. These line cards support dual 40-Gbps fabric channel connections and operate in distributed dCEF2T mode.
Cisco 6700 Series line cards can be upgraded from CFC/DFC3C/DFC3CXL to DFC4 and DFC4XL This upgrade is supported on a 2T system. The DFC4 versions used for the upgrade are:
• WS-F6K-DFC4-A
• WS-F6K-DFC4-AXL
The line cards maintain their characteristics when they move from existing systems to systems with the Cisco Supervisor Engine 2T. For example, the current 6716-10G-3C line card with 4:1 oversubscription in the current systems will still have 4:1 oversubscription in a system deployed with the Cisco Supervisor Engine 2T.
Table 4. Supervisor Engine 2T-Compatible Line Cards
Line-Card Family
Line Card
Line-Card Description
6900 Series Line Cards (dCEF2T)
WS-X6908-10G-2T
WS-X6908-10G-2TXL
8-port 10GE (DFC4/DFC4XL)
WS-X6904-40G-2T
WS-X6904-40G-2TXL
4-port 40GE or 16-port 10GE (DFC4/DFC4XL)
6800 Series Line Cards (dCEF720)
WS-X6816-10T-2T
WS-X6816-10T-2TXL
16-port 10G Base-T (DFC4/DFC4XL)
WS-X6848-GE-TX-2T
WS-X6848-GE-TX-2TXL
48-port 10/100/1000 RJ-45 (DFC4/DFC4XL)
WS-X6848-SFP-2T
WS-X6848-SFP-2TXL
48-port GE SFP (DFC4/DFC4XL)
WS-X6824-SFP-2T
WS-X6824-SFP-2TXL
24-port GE SFP (DFC4/DFC4XL)
6700 Series Line Cards (CEF720)
WS-X6704-10GE
4-port 10GE (CFC only)
WS-X6748-GE-TX
48-port 10/100/1000 (CFC only)
WS-X6748-SFP
48-port GE SFP (CFC only)
WS-X6724-SFP
24-port GE SFP (CFC only)
6100 Series Line Cards (classic)
WS-X6148E-GE-45AT
48-port 10/100/1000 and PoEP (bus only)
Table 5. DFC4 Field-Upgradable Line Cards
Line Card
Line-Card Description
WS-X6716-10GE
16-port 10GE (DFC3/DFC3XL)
WS-X6716-10T
16-port 10G Base-T (DFC3/DFC3XL)
WS-X6724-SFP
24-port GE SFP (DFC3/DFC3XL or CFC)
WS-X6748-SFP
48-port GE SFP (DFC3/DFC3XL or CFC)
WS-X6748-GE-TX
48-port 10/100/1000 (DFC3/DFC3XL or CFC)
WS-X6704-10GE
4-port 10GE (DFC3/DFC3XL or CFC)
Table 6. PFC/DFC Interoperability Matrix
PFC3A
PFC3B
PFC3BXL
PFC3C
PFC3CXL
PFC4
PFC4XL
DFC3A
√
PFC3B operates as PFC3A
PFC3BXL operates as PFC3A
PFC3C operates as PFC3A
PFC3CXL operates as PFC3A
Not compatible
Not compatible
DFC3B
DFC3B operates as DFC3A
√
PFC3BXL operates as PFC3B
PFC3C operates as PFC3B
PFC3CXL operates as PFC3B
Not compatible
Not compatible
DFC3BXL
DFC3BXL operates as DFC3A
DFC3BXL operates as DFC3B
√
DFC3BXL operates as DFC3B and PFC3C operates as PFC3B
PFC3CXL operates as PFC3BXL
Not compatible
Not compatible
DFC3C
DFC3C operates as DFC3A
DFC3C operates as DFC3B
PFC3BXL operates as PFC3B and DFC3C operates as DFC3B
√
PFC3CXL operates as PFC3C
Not compatible
Not compatible
DFC3CXL
DFC3CXL operates as DFC3A
DFC3CXL operates as DFC3B
DFC3CXL operates as DFC3BXL
DFC3CXL operates as DFC3C
√
Not compatible
Not compatible
DFC4
Not compatible
Not compatible
Not compatible
Not compatible
Not compatible
√
PFC4XL operates as PFC4
DFC4XL
Not compatible
Not compatible
Not compatible
Not compatible
Not compatible
DFC4XL operates as DFC4
√
Supervisor and Line-Card Allocation in E Chassis
Table 7. Allowed Supervisor and Line-Card Slot Allocation by Chassis Type
Cisco Catalyst 6503-E
Cisco Catalyst 6504-E
Cisco Catalyst 6506-E
Cisco Catalyst 6509-E
Cisco Catalyst 6509-V-E
Cisco Catalyst 6513-E
Slot 1
Supervisor or line card
Supervisor or line card
Line card
Line card
Line card
Line card
Slot 2
Supervisor or line card
Supervisor or line card
Line card
Line card
Line card
Line card
Slot 3
Line card
Line card
Line card
Line card
Line card
Line card
Slot 4
Line card
Line card
Line card
Line card
Line card
Slot 5
Supervisor or line card
Supervisor or line card
Supervisor or line card
Line card
Slot 6
Supervisor or line card
Supervisor or line card
Supervisor or line card
Line card
Slot 7
Line card
Line card
Supervisor or line card*
Slot 8
Line card
Line card
Supervisor or line card*
Slot 9
Line card
Line card
Line card
Slot 10
Line card
Slot 11
Line card
Slot 12
Line card
Slot 13
Line card
5. Memory
The current 256-MB and 512-MB compact flash memory option in the Cisco Supervisor Engine 720 is not supported by the Cisco Supervisor Engine 2T, which supports only 1-GB (MEM-C6K-CPTFL1GB) and 2-GB (MEM-C6K-CPTFL2GB) compact flash memory. Both Supervisor Engine 2T lite and XL versions have 2-GB DRAM as the default. Both can be upgraded to 4 GB with an additional 2-GB DRAM purchase. The memory configuration for 4 GB is 2 GB plus 2 GB. A 4-GB DRAM memory option is available as well.
The following service modules are supported on the Supervisor Engine 2T:
• Cisco Catalyst 6500 Series Firewall Services Module (FWSM)
• Cisco Adaptive Security Appliances (ASA) service module
• Cisco Application Control Engine ACE 20 and ACE 30 (with updated firmware)
• Cisco Wireless Services Module (WiSM) and WiSM-2
Table 9. Supported Service Modules
Part Number
Product Description
ACE20/ACE 30
Cisco Catalyst 6500 Application Control Engine 20/Application Control Engine 30 Hardware
NAM-1
Cisco Catalyst 6500 Network Analysis Module 1
NAM-2
Cisco Catalyst 6500 Network Analysis Module 2
FWSM
Cisco Catalyst 6500 Firewall Service Module
WISM
Cisco Catalyst 6500 Wireless Service Module
WISM2
Cisco Catalyst 6500 Wireless Services Module 2
7. Software
Cisco Supervisor Engine 2T will be supported with Cisco IOS® Software Release 15.0 SY. The following four varieties are available for Cisco Supervisor Engine 2T systems:
• IP Base
IP BASE FULL ENCRYPT s2t54-ipbasek9-mz.SPA.150-1.SY1.bin Minimum memory: DRAM 2048 MB, flash 1024 MB Size: 78.78 MB (82,603,832 bytes)
• IP Services
IP SERV FULL ENCRYPT s2t54-ipservicesk9-mz.SPA.150-1.SY1.bin Minimum memory: DRAM 2048 MB, flash 1024 MB Size: 81.84 MB (85,814,640 bytes)
• Advanced IP Services
ADVANCED IP SERVICES FULL ENCRYPT s2t54-advipservicesk9-mz.SPA.150-1.SY1.bin Minimum memory: DRAM 2048 MB, flash 1024 MB Size: 85.28 MB (89,418,128 bytes)
• Advanced Enterprise Services
ADV ENT SERV FULL ENCRYPT s2t54-adventerprisek9-mz.SPA.150-1.SY1.bin Minimum memory: DRAM 2048 MB, flash 1024 MB Size: 86.85 MB (91,068,320 bytes)
Migration Test
Test Scope
This chapter contains a description of the primary areas tested, what tests were performed, and traffic profiles used for the testing done for this migration test.
Test Objectives
The test objective is to perform migration of the Supervisor Engine 720 to the Supervisor Engine 2T in a production environment. Testing of the Cisco Catalyst 6500 Series platform with associated line cards and supervisors will be implemented to demonstrate the steps involved to migrate to the next-generation Supervisor Engine 2T in a campus distribution and campus core layer.
Multiple network components are configured and enabled during the migration. These components include Border Gateway Protocol (BGP), Enhanced Interior Gateway Routing Protocol (EIGRP), and Hot Standby Router Protocol (HSRP).
In standalone topology, the access layer is attached to the distribution layer using redundant Layer 2 links in active-active configuration (that is, VLANs are load-balanced on both links). The distribution layer has two Cisco Catalyst 6500 Series Switches connected back to back through a port channel, and Layer 3 links are dual-homed to the core layer for better redundancy and load-balancing.
In VSS topology, the access layer is dual-attached to the distribution layer through multichassis EtherChannel (MEC). The distribution layer has two Cisco Catalyst 6500 Series Switches connected back to back through a port channel, and Layer 3 links are dual-homed to the core layer for better redundancy and load-balancing.
Spirent is used to generate inter-VLAN and intra-VLAN traffic. Protocol interface and devices are use to generate Address Resolution Protocol (ARP)/MAC traffic.
For live migration, the latest Cisco IOS Software has been used:
• Release 12.2 SXJ for the Supervisor Engine 720
• Release 15.0(1) SY1 for the Supervisor Engine 2T
The chassis used for this document has 9 slots (6509-E chassis) equipped with:
• 1 service module (ACE20)
• 2 line cards (WS-6748 and WS-6908)
• 2 supervisors (both Supervisor Engine 720s will be replaced with Supervisor Engine 2Ts)
Case 1: Migration of Supervisor Engine 720 to Supervisor Engine 2T in Standalone Mode (Single Supervisor)
Testing Overview
This case intends to verify:
• Functionality of Cisco Catalyst 6500 pair, with single Supervisor Engine 720 in each chassis, after migrating Supervisor Engine 720 to Supervisor Engine 2T
• Network resiliency and convergence behavior with different failover scenarios
• Steps involved in upgrading from Supervisor Engine 720 to Supervisor Engine 2T
Testing Topology
The test topology contains two Cisco Catalyst 6500 switches used at the distribution layer, two Cisco 4948s acting as core, one Cisco 3750 acting as service provider router, and one Cisco 4948 acting as access. Rapid Spanning Tree Protocol (RSTP) is used for loop avoidance in the network.
Spirent Scale
Layer 2
Layer 3
VLAN usage
50 @ core
EtherSVI
50
MAC addresses
Hosts/VLAN
Total transmitting hosts
Total receiving hosts
Traffic volume
5000
100
5000
1
1 Gbps
HSRP
50 groups
ARPs
5000
Steady-State Topology and Traffic Flows
HSRP Status
SW1
Cat6K-agg1#sh standby internal summary
Disable Init Learn Listen Speak Standby Active
Configured 0 0 0 0 0 0 50
Learnt 0 0 0 0 0 0 0
SW2-
Cat6K-agg2#sh standby internal summary
Disable Init Learn Listen Speak Standby Active
Configured 0 0 0 0 0 50 0
Learnt 0 0 0 0 0 0 0
Layer 2 Configuration
Host Configurations
VLAN Summary
Cat6K-agg1#sh vlan summary
Number of existing VLANs : 55
Number of existing VTP VLANs : 55
Number of existing extended VLANS : 0
Cat6K-agg2#show vlan summary
Number of existing VLANs : 55
Number of existing VTP VLANs : 55
Number of existing extended VLANS : 0
MAC Address Table Count
Cat6K-agg1#sh mac address-table count
MAC Entries for all vlans :
Dynamic Address Count: 5050
Static Address (User-defined) Count: 105
Total MAC Addresses In Use: 5155
Total MAC Addresses Available: 98304
Cat6K-agg2#sh mac address-table count
MAC Entries for all vlans :
Dynamic Address Count: 50
Static Address (User-defined) Count: 55
Total MAC Addresses In Use: 105
Total MAC Addresses Available: 98304
ARP
Cat6K-agg1#show ip arp summary
105 IP ARP entries, with 0 of them incomplete
Cat6K-agg2#show ip arp summary
55 IP ARP entries, with 0 of them incomplete
Results from Spirent During Steady State
Step 1. Migration of Secondary Switch (Cat6K-agg2)
The secondary switch needs to be migrated first for minimal disruption of the traffic and to make sure that there are no unexpected traffic flows.
1. Shut all the active interfaces on the secondary switch. The traffic will be diverted to the second switch based on HSRP.
Cat6K-agg2#sh cdp neighbor
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
D - Remote, C - CVTA, M - Two-port Mac Relay
Device ID Local Intrfce Holdtme Capability Platform Port ID
SJSPSWPE01 Gig 5/1 141 R S I WS-C6503- Gig 3/12
4948-access Gig 2/48 122 R S I WS-C4948- Gig 1/48
4948-core-2 Gig 2/3 169 R S I WS-C4948- Gig 1/2
4948-core-1 Gig 2/4 135 R S I WS-C4948- Gig 1/4
Cat6K-agg1 Ten 1/2 167 R S I WS-C6509- Ten 1/2
Cat6K-agg1 Ten 1/3 177 R S I WS-C6509- Ten 1/3
Cat6K-agg2#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Cat6K-agg2(config)#int range gi2/48,gi2/3 - 4,te1/2 - 3
Cat6K-agg2(config-if-range)#shut
Cat6K-agg2(config-if-range)#
The screenshot is taken from the Spirent tool when the links on the secondary switch are shut down. As can be seen, there is no effect on traffic after shutdown of interfaces.
2. Insert a 1-GB or 2-GB external compact flash in the Supervisor Engine 720. As indicated earlier, the Supervisor Engine 720 comes with a maximum of 512 MB of internal flash memory; external flash memory is required to copy the required Supervisor Engine 2T image.
3. Copy the running config and new software image for the Supervisor Engine 2T on the compact flash.
4. Remove the Supervisor Engine 720, unsupported line cards, service modules, and power supplies from non-E chassis.
5. Rack and stack the E-series chassis.
6. Insert the new Supervisor Engine 2T with 1-GB compact flash that has the supported Cisco IOS Software image into the supervisor slot with supported line cards and turn on the switch power supplies.
• The new Supervisor Engine 2T will come up in rommon mode.
• Boot the Supervisor Engine 2T with the new software image using rommon CLI.
rommon>boot disk0:s2txx_new_sup2t_image
• The average time to bring up the new supervisor is 3 to 4 minutes depending upon the number of modules present in the chassis.
7. When the Supervisor Engine 2T boots up with supported software, restore the config from bootflash or disk0:
"copy disk0:saved_config system:running_config"
8. Since the software code is backward compatible, the switch will retain the configs, and any old CLIs will be converted into new commands.
0 output buffer failures, 0 output buffers swapped out
The snapshot of the Spirent tool shows that there is no effect on the traffic flowing across Switch 1 during Switch 2 migration.
Step 3. Migration of Primary Switch (Cat6K-agg1)
Before the primary switch is upgraded, production traffic needs to be forwarded over to the secondary switch (Cat6K-agg2).
1. Shut all active interfaces on the primary switch.
2. When the interfaces on the primary switch are disabled, a 20-second traffic drop is observed, essentially because of ARP learning, HSRP, and spanning tree recalculations, and then traffic fails over to the backup path.
When Switch 1 is being upgraded with the Supervisor Engine 2T, the traffic flow is as follows.
3. Repeat the same steps to upgrade the Supervisor Engine 720 to Supervisor Engine 2T in the primary switch. Insert a 1-GB or 2-GB external compact flash in the Supervisor Engine 720. (As indicated earlier, the Supervisor Engine 720 comes with a maximum of 512 MB of internal flash memory; external flash memory is required to copy the required Supervisor Engine 2T image.)
4. Copy the running config and new software image for the Supervisor Engine 2T on the compact flash.
5. Remove the Supervisor Engine 720, unsupported line cards, service modules, and power supplies from non-E chassis.
6. Rack and stack the E-series chassis.
7. Insert the new Supervisor Engine 2T with 1-GB compact flash that has the supported Cisco IOS Software image into the supervisor slot with supported line cards and turn on the switch power supplies.
• The new Supervisor Engine 2T will come up in rommon mode.
• Boot the Supervisor Engine 2T with new software image using rommon CLI.
rommon>boot disk0:s2txx_new_sup2t_image
• The average time to bring up the new supervisor is 3 to 4 minutes depending upon the number of modules present in the chassis.
8. When the Supervisor Engine 2T boots up with supported software, restore the config from bootflash or disk0:
"copy disk0:saved_config system:running_config"
9. Since the software code is backward compatible, the switch will retain the configs, and old CLIs will be converted into new commands.
• When the config is restored, there is no effect on traffic flowing from Switch 1.
10. Unshut the interfaces.
Cat6K-agg1(configs)#int range gi2/48,gi2/3 - 4,te1/2 - 3
Cat6K-agg1(configs-if-range)#no shut
Cat6K-agg1(configs-if-range)#
11. When the configuration is loaded and the traffic falls back to the primary switch, traffic loss of 20 seconds is observed for spanning tree and HSRP recalculations.
Step 4. Supervisor Engine 2T Verification in Primary Switch
1 EARL8 Distributed Forwardin WS-F6K-DFC4-E SAL16095K3L 1.2 Ok
2 Centralized Forwarding Card WS-F6700-CFC SAL10381WPG 3.0 Ok
5 EARL8 MAKALU-LITE VS-F6K-PFC4 SAL161172EG 1.2 Ok
5 CPU Daughterboard VS-F6K-MSFC5 SAL16106JQ0 1.4 Ok
Mod Online Diag Status
---- -------------------
1 Pass
2 Pass
5 Pass
7 Pass
8 Not Applicable
9 Pass
Cat6K-agg1#
Cat6K-agg1#sh cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
D - Remote, C - CVTA, M - Two-port Mac Relay
Device ID Local Intrfce Holdtme Capability Platform Port ID
4948-access Gig 2/48 156 R S I WS-C4948- Gig 1/47
4948-core-2 Gig 2/3 154 R S I WS-C4948- Gig 1/1
4948-core-1 Gig 2/4 154 R S I WS-C4948- Gig 1/3
Cat6K-agg2 Ten 1/2 131 R S I WS-C6509- Ten 1/2
Cat6K-agg2 Ten 1/3 139 R S I WS-C6509- Ten 1/3
Cat6K-agg1#
Case 2: Migration of Supervisor Engine 720 to Supervisor Engine 2T in Standalone Mode (Dual Supervisor, High Availability)
Testing Overview
This case intends to verify:
• Functionality of Cisco Catalyst 6500 pair with dual Supervisor Engine 720s in each chassis after migrating Supervisor Engine 720 to Supervisor Engine 2T
• Network resiliency and convergence behavior with different failover scenarios
• Steps involved in upgrading from Supervisor Engine 720 to Supervisor Engine 2T
Testing Topology
The test topology contains two Cisco Catalyst 6500 switches used at the distribution layer, two Cisco 4948s acting as core, one Cisco 3750 acting as service provider router, and one Cisco 4948 acting as access. RSTP is used for loop avoidance in the network.
1 FRU type (0x6004, 0xEA(234) WS-F6K-DFC4-E SAL16095K3L 1.2 PwrDown
2 Centralized Forwarding Card WS-F6700-CFC SAL1207GEH3 4.0 Ok
5 Policy Feature Card 3 WS-F6K-PFC3BXL SAD1125072U 1.8 Ok
5 MSFC3 Daughterboard WS-SUP720 SAD11260535 2.6 Ok
Mod Online Diag Status
---- -------------------
1 Not Applicable
2 Pass
5 Pass
6 Not Applicable
Cat6K-agg2#
Results from Spirent During Steady State
Step 1. Upgrade of Secondary Switch (Cat6K-agg2)
The secondary switch needs to be migrated first for minimal disruption of the traffic and to make sure that there are no unexpected traffic flows.
1. Shut all the active interfaces on the secondary switch. The traffic will be diverted to the second switch based on HSRP.
Cat6K-agg2(config)#int range gi2/48,gi2/3 - 4,te1/2 - 3
Cat6K-agg2(config-if-range)#shut
The screenshot is taken from the Spirent tool when the links on the secondary switch are shut down. As can be seen, there is no effect on traffic after shutdown of interfaces.
2. Insert a 1-GB or 2-GB external compact flash in the Supervisor Engine 720. (As indicated earlier, the Supervisor Engine 720 comes with a maximum of 512 MB of internal flash memory; external flash memory is required to copy the required Supervisor Engine 2T image.)
3. Copy the running config and new software image for Supervisor Engine 2T on the compact flash.
4. Remove both Supervisor Engine 720s, unsupported line cards, service modules, and power supplies from non-E chassis.
5. Rack and stack the E-series chassis.
6. Insert the new Supervisor Engine 2T in both supervisor slots with 1-GB compact flash that has the supported Cisco IOS Software image in the supervisor slots with supported line cards and turn on the switch power supplies.
• The new Supervisor Engine 2Ts will come up in rommon mode.
• Boot both Supervisor Engine 2Ts with new software image using rommon CLI.
rommon>boot disk0:s2txx_new_sup2t_image
• The average time to bring up the new supervisor is 3 to 4 minutes depending upon the number of modules present in the chassis.
7. Both the Supervisor Engine 2Ts will boot with supported software in high availability (active/standby state). Restore the config from bootflash or disk0:
"copy disk0:saved_config system:running_config"
8. Since the software code is backward compatible, the switch will retain the configs, and old CLIs will be converted into new commands.
1 Distributed Forwarding Card WS-F6K-DFC4-E SAL16095R3F 1.2 Ok
2 Centralized Forwarding Card WS-F6700-CFC SAL10381WPG 3.0 Ok
5 Policy Feature Card 4 VS-F6K-PFC4 SAL161172EG 1.2 Ok
5 CPU Daughterboard VS-F6K-MSFC5 SAL16106JQ0 1.4 Ok
6 Policy Feature Card 4 VS-F6K-PFC4 SAL16010C7B 1.1 Ok
6 CPU Daughterboard VS-F6K-MSFC5 SAL16020TKS 1.3 Ok
Mod Online Diag Status
---- -------------------
1 Pass
2 Pass
5 Pass
6 Pass
7 Pass
8 Not Applicable
9 Pass
Step 3. Migration of Primary Switch (Cat6K-agg1)
Before the primary switch is upgraded, traffic needs to be shifted over to the secondary switch (Switch 2).
1. Shut all active interfaces on the primary switch.
2. When the interfaces on the primary switch are disabled, a 20-sec traffic drop is observed, essentially because of ARP learning, HSRP, and spanning tree recalculations, and then traffic fails over to the backup path.
During the time Switch 1 is being upgraded, the traffic flows from Switch 2 using HSRP virtual IP address.
3. Repeat the same steps to upgrade the Supervisor Engine 720s to Supervisor Engine 2Ts in the primary switch (Switch 1). Insert a 1-GB or 2-GB external compact flash in the Supervisor Engine 720. (As indicated earlier, the Supervisor Engine 720 comes with a maximum of 512 MB of internal flash memory; external flash memory is required to copy the required Supervisor Engine 2T image.)
4. Copy the running config and new software image for Supervisor Engine 2T on the compact flash.
5. Remove both Supervisor Engine 720s, unsupported line cards, service modules. and power supplies from non-E chassis.
6. Rack and stack the E-series chassis.
7. Insert the new Supervisor Engine 2T in both supervisor slots with 1-GB compact flash that has the supported Cisco IOS Software image in the supervisor slots with supported line cards and turn on the switch power supplies.
• The new Supervisor Engine 2Ts will come up in rommon mode.
• Boot both Supervisor Engine 2Ts with new software image using rommon CLI.
rommon>boot disk0:s2txx_new_sup2t_image
• The average time to bring up the new supervisor is 3 to 4 minutes depending upon the number of modules present in the chassis.
8. Both the Supervisor Engine 2Ts will boot with supported software in high availability (active/standby state). Restore the config from bootflash or disk0:
"copy disk0:saved_config system:running_config"
9. Since the software code is backward compatible, the switch will retain the configs, and old CLIs will be converted into new commands.
• When the config is restored, there is no effect on traffic flowing from Switch 1.
10. Unshut the interfaces.
Cat6K-agg1(configs)#int range gi2/48,gi2/3 - 4,te1/2 - 3
Cat6K-agg1(configs-if-range)#no shut
Cat6K-agg1(configs-if-range)#
11. When the configuration is loaded and the traffic falls back to the primary switch, traffic loss of 20 seconds is observed for spanning tree and HSRP recalculations.
0 output buffer failures, 0 output buffers swapped out
Cat6K-agg1#
Cat6K-agg1# sh int gi2/48 | i rate
Queueing strategy: fifo
30 second input rate 86493000 bits/sec, 84467 packets/sec
30 second output rate 48000 bits/sec, 85 packets/sec
Cat6K-agg1#
Case 3: Migration of Supervisor Engine 720 to Supervisor Engine 2T in VSS Mode
Testing Overview
This case intends to verify:
• Functionality of Cisco Catalyst 6500 pair in VSS mode with single Supervisor Engine 720 in each chassis after migrating to Supervisor Engine 2T
• Network resiliency and convergence behavior with different failover scenarios
• Steps involved in upgrading from Supervisor Engine 720 to Supervisor Engine 2T
Testing Topology
The test topology contains two Cisco Catalyst 6500 switches used at the distribution layer in VSS mode, two Cisco 4948s acting as core, one Cisco 3750 acting as service provider router, and one Cisco 4948 acting as access. RSTP is used for loop avoidance in the network.
Layer 2
Layer 3
VLAN usage
50 @ core
EtherSVI
50
MAC addresses
Hosts/VLAN
Total transmitting hosts
Total receiving hosts
Traffic volume
5000
100
5000
1
1 Gbps
HSRP
None
ARPs
5000
Steady-State Topology and Traffic Flows
Layer 2 Configuration
Host Configurations
Verification of Switches in VSS Mode
Cat6K-agg1#sh switch virtual
Switch mode : Virtual Switch
Virtual switch domain number : 10
Local switch number : 1
Local switch operational role: Virtual Switch Active
Peer switch number : 2
Peer switch operational role : Virtual Switch Standby
Cat6K-agg1#
Cat6K-agg1#sh switch virtual redundancy
My Switch Id = 1
Peer Switch Id = 2
Last switchover reason = none
Configured Redundancy Mode = sso
Operating Redundancy Mode = sso
Switch 1 Slot 5 Processor Information :
-----------------------------------------------
Current Software state = ACTIVE
Uptime in current state = 19 hours, 24 minutes
Image Version = Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(33)SXJ2, RELEASE SOFTWARE (fc4)
The secondary switch needs to be migrated first for minimal disruption of the traffic and to make sure that there are no unexpected traffic flows.
1. Shut all the active interfaces on the secondary switch, including the virtual switch link (VSL). This will be done from the Switch 1 console itself since they are now a part of VSS.
Cat6K-agg1(config)# int range ethernet 2/2/48,gig2/2/3,Gig 2/2/4,Gig 2/2/11
Cat6K-agg1(config-if-range)#shut
*Aug 31 17:15:10.973: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 2.2.1.1 (GigabitEthernet2/2/3) is down: interface down
*Aug 31 17:15:11.441: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 2.2.2.1 (GigabitEthernet2/2/4) is down: interface down
VSL Link Shut
Cat6K-agg1(config-if-range)#interface range te 2/5/4-5
Cat6K-agg1(config-if-range)#shut
WARNING: You are shutting down one or more VSL interfaces.
If all VSL interfaces are down, connectivity between active
and standby switch (if present) will be lost and would also result
in two active switches. Traffic disruption will occur, and possible
configuration mismatch between the switches can happen.
Do you want to proceed? [yes/no]: yes
*Aug 31 17:18:41.469: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON: Te1/5/4: Link down
*Aug 31 17:18:41.961: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON: Te1/5/5: Link down
*Aug 31 17:18:41.961: %VSLP-SW1_SP-2-VSL_DOWN: Last VSL interface Te1/5/5 went down
*Aug 31 17:18:41.981: %VSLP-SW1_SP-2-VSL_DOWN: All VSL links went down while switch is in ACTIVE role
As can be seen in the graph, there is a brief packet loss, and the traffic on the receiving port drops for a small interval, but the network doesn't go down completely.
Port Name
Rx Rate (bps)
Tx Rate (bps)
Dropped Count (Frames)
Dropped Frame Percent
Port //12/1
167,560
172,968
0
0
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
Port //12/1
167,560
172,968
0
0
Port //12/1
167,416
172,968
64
0.08
Port //12/1
167,560
172,968
0
0
2. Insert a 1-GB or 2-GB external compact flash in the Supervisor Engine 720. (As indicated earlier, the Supervisor Engine 720 comes with a maximum of 512 MB of internal flash memory; external flash memory is required to copy the required Supervisor Engine 2T image.)
3. Back up the running config and new software image for the Supervisor Engine 2T on the compact flash.
4. Remove both Supervisor Engine 720s, unsupported line cards, service modules, and power supplies from non-E chassis.
5. Rack and stack the E-series chassis.
6. Insert the new Supervisor Engine 2T in supervisor slot with 1-GB compact flash that has the supported Cisco IOS Software image in the supervisor slots with supported line cards and turn on the switch power supplies.
• The new Supervisor Engine 2T will come up in rommon mode.
• Boot Supervisor Engine 2T with new software image using rommon CLI.
rommon>boot disk0:s2txx_new_sup2t_image
• The average time to bring up the new supervisor is 3 to 4 minutes depending upon the number of modules present in the chassis.
7. Restore the config from bootflash or disk0: When restoring the config, be sure to leave out any VSS-related configs (remove VSS-related config). Also make sure the VSL link-associated ports are disabled.
"copy disk0:saved_config system:running_config"
8. Once the config is restored, make sure the port connecting to downstream devices with MEC is configured for port channel with the right protocol. These ports will stay down even after the port channel protocol is configured; this is expected behavior. Also enable the other ports that were disabled earlier; leave the VSL link disabled.
Cat6K-agg1#sh int g2/48
GigabitEthernet2/48 is up, line protocol is down (notconnect)
Hardware is C6k 1000Mb 802.3, address is 001f.6cf6.528f (bia 001f.6cf6.528f)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT
input flow-control is off, output flow-control is on
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
During the time Switch 2 is being upgraded, the traffic flows from Switch 1, as can be seen in the following figure.
Step 2. Migration of Primary Switch (Cat6k-agg1)
Before upgrading the primary switch, traffic needs to be shifted over to the secondary switch (Switch 2).
1. Shut all active interfaces on the primary switch (Switch 1), including VSL link. Now when the interfaces on the primary switch are disabled, the link on the secondary will come up.
Cat6K-agg1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Cat6K-agg1(config)#interface range gigabitEthernet 1/2/48,gig1/2/3,Gig 1/2/4,Gig 1/2/11
Cat6K-agg1(config-if-range)#shut
Cat6K-agg1(config-if-range
SW2 Interface is active
Cat6K-agg1#sh int g2/48
GigabitEthernet2/48 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 001f.6cf6.528f (bia 001f.6cf6.528f)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 137/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT
input flow-control is off, output flow-control is on
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
2. At this point, traffic loss of around 33 seconds is observed because of the STP convergence and EtherChannel forming.
3. Repeat the same steps to upgrade the Supervisor Engine 720s to Supervisor Engine 2Ts in the primary switch. Insert a 1-GB or 2-GB external compact flash in the Supervisor Engine 720. (As indicated earlier, the Supervisor Engine 720 comes with a maximum of 512 MB of internal flash memory; external flash memory is required to copy the required Supervisor Engine 2T image.)
4. Back up the running config and new software image for Supervisor Engine 2T on the compact flash.
5. Remove both Supervisor Engine 720s, unsupported line cards, service modules, and power supplies from non-E chassis.
6. Rack and stack the E-series chassis.
7. Insert the new Supervisor Engine 2T in supervisor slot with 1-GB compact flash that has the supported Cisco IOS Software image in the supervisor slots with supported line cards and turn on the switch power supplies.
• The new Supervisor Engine 2T will come up in rommon mode.
• Boot Supervisor Engine 2T with new software image using rommon CLI.
rommon>boot disk0:s2txx_new_sup2t_image
• The average time to bring up the new supervisor is 3 to 4 minutes depending upon the number of modules present in the chassis.
8. When the switch boots up, make sure all interfaces, including the VSL, are shut. Restore the config from bootflash or disk0, including the VSS-related configs. This will cause the switch to reload and convert to VSS.
"copy disk0:saved_config system:running_config"
9. Since the software code is backward compatible, the switch will retain the configs, and any old CLIs will be converted into new commands.
Once the config is restored, make sure the port connecting to downstream devices with MEC is configured for port channel with the right protocol. These ports will stay down even after the port channel protocol is configured; this is expected behavior.
Cat6K-agg1#sh int g1/2/48
GigabitEthernet1/2/48 is up, line protocol is down (notconnect)
Hardware is C6k 1000Mb 802.3, address is 0019.56c3.f30f (bia 0019.56c3.f30f)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT
input flow-control is off, output flow-control is on
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Now disable the interfaces on the secondary switch. When the interfaces on the secondary switch are disabled, the link on the primary will come up, and the traffic from the downstream access switch will switch over to the primary switch.
Note: As we are using Spirent to generate traffic for this test setup, the drop in traffic is because of the manual ARP refresh in Spirent. In a real-world scenario, you might see some drop because of this scenario.
On Switch 2
Cat6K-agg2(config)#int range gi2/3-4,gi2/48
Cat6K-agg2(config-if-range)#shut
Cat6K-agg2(config-if-range)#
*Sep 4 23:05:50.895: %DUAL-5-NBRCHANGE: EIGRP-IPv4:(543) 1: Neighbor 2.2.1.1 (GigabitEthernet2/3) is down: interface down
*Sep 4 23:05:50.895: %DUAL-5-NBRCHANGE: EIGRP-IPv4:(543) 1: Neighbor 2.2.2.1 (GigabitEthernet2/4) is down: interface down
*Sep 4 14:43:42.295: %EC-5-UNBUNDLE: Interface Gi1/48 left the port-channel Po5
4948-access#
*Sep 4 14:43:46.187: %EC-5-BUNDLE: Interface Gi1/47 joined port-channel Po5
0 output buffer failures, 0 output buffers swapped out
Cat6K-agg1#
At this point we see a traffic loss of 30 seconds because of ARP learning.
Now reenable the VSL link on Switch 1 and convert Switch 2 to VSS mode.
Cat6K-agg1(config-if-range)#int po5
Cat6K-agg1(config-if)#no shut
Cat6K-agg1(config-if)#int ran te1/5/4-5
Cat6K-agg1(config-if-range)#no sh
Cat6K-agg1(config-if-range)#
When Switch 2 boots up with VSL link disabled, there should not be any traffic loss.
Now enable the VSL link first. As can be seen, when the VSL link is enabled, the system detects dual active and goes into recovery mode.
Cat6K-agg1#
*Sep 4 23:29:15.839: %VSLP-SW2-5-RRP_MSG: Role change from Active to Standby and hence need to reload
*Sep 4 23:29:15.839: %VSLP-SW2-5-RRP_UNSAVED_CONFIG: Ignoring system reload since there are unsaved configurations.
Please save the relevant configurations
*Sep 4 23:29:15.839: %VSLP-SW2-5-RRP_MSG: Use 'redundancy reload shelf' to bring this switch to its preferred STANDBY role
*Sep 4 23:29:15.843: %DUAL_ACTIVE-SW2-1-RECOVERY: Dual-active condition detected: Starting recovery-mode, all non-VSL and non-excluded interfaces have been shut down
Cat6K-agg1(recovery-mode)#
Traffic still keeps running through SW1. Issue redundancy reload shelf to reload this switch again so it comes up as standby switch.
Cat6K-agg1(recovery-mode)#redundancy reload shelf
System configuration has been modified. Save? [yes/no]: yes
When Switch 2 comes up in VSS mode, the config is synchronized between the two switches, and redundancy establishes. No traffic disruption is seen during this process.
Cat6K-agg1#show switch virtual redundancy
My Switch Id = 1
Peer Switch Id = 2
Last switchover reason = none
Configured Redundancy Mode = sso
Operating Redundancy Mode = sso
Switch 1 Slot 5 Processor Information :
-----------------------------------------------
Current Software state = ACTIVE
Uptime in current state = 2 hours, 41 minutes
Image Version = Cisco IOS Software, s2t54 Software (s2t54-ADVIPSERVICESK9-M), Version 12.2(50)SY, RELEASE SOFTWARE (fc3)
Finally, enable the interfaces of Switch 2 connecting to upstream and downstream devices. Configure EtherChannel protocol on ports connecting to MEC devices.
Cat6K-agg1#sh etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator
M - not in use, no aggregation due to minimum links not met
m - not in use, port not aggregated due to minimum links not met
Cisco Catalyst 6500 Line Card and PoE Daughter Cards
Cisco Catalyst 6500 Line Card
WS-X6148-GE-45AF
WS-X6148X2-45AF
WS-X6148-RJ21V
WS-X6148-RJ45V
WS-X6148V-GE-TX
WS-X6348-RJ21V
WS-X6348-RJ45V
WS-F6K-VPWR
WS-F6K-VPWR-GE
WS-X6148E-GE-45AT
Summarized Test Results and Recommendations
This chapter summarizes the test results and provides recommendation based on those results.
Case
Scenario
Shifting/Isolating Traffic
Effect on Traffic
Traffic Affected (in Secs)
1
Secondary switch upgrade in single sup (standalone)
Yes, by shutting down the uplinks connected to upstream/downstream switches and Switch 1
None
0
Primary switch upgrade in single sup (standalone)
Yes, by shutting down the uplinks connected to upstream/downstream switches and Switch 2
Effect when primary is shut down for RSTP and HSRP recalculations
30 sec
2
Secondary switch upgrade in dual sup (standalone)
Yes, by shutting down the uplinks connected to upstream/downstream switches and Switch 1
None
0
Primary switch upgrade in dual sup (standalone)
Yes, by shutting down the uplinks connected to upstream/downstream switches and Switch 2
Effect when primary is shut down for RSTP and HSRP recalculations
30 sec
3
Secondary switch upgrade in VSS mode
Yes, by shutting down the uplinks connected to upstream/downstream switches and Switch 1
Minimal packet loss was observed because of RSTP and ARP calculations when packets were going through secondary VSS chassis
10 sec
There seems to be partial packet loss
Primary switch upgrade in VSS mode
Yes, by shutting down the uplinks connected to upstream/downstream switches and Switch 2 and converting to VSS mode
Effect when primary is shut down for RSTP calculations/ARP synchronization and reconfiguring VSS
30 sec
Partial traffic loss
Recommendations
• It was found that upgrading Cisco Catalyst 6500 from Supervisor Engine 720 to Supervisor Engine 2T was straightforward and added significant value in the areas of MACsec encryption, improved ACL capabilities, and IPv4/IPv6/MPLS/VPLS/VSS throughput performance.
• In addition, it was observed that the Supervisor Engine 2T supports existing service models such as network analysis module (NAM), wireless services module (WiSM), Application Control Engine (ACE20 required firmware upgrade), Firewall Service Module (FWSM), 6148A-GE and 6148E-GE with PoE/PoE+, 6724-SFP line cards 6704, and 6716 line cards after a trivial DFC3 to DFC4 daughter card swap.
• It was also found that line cards can be swapped and upgraded while the Supervisor Engine 2T is operational, avoiding off-hour scheduled downtime. In addition, the existing interface transceivers SFP and X2 being used in a Cisco Catalyst 6500 Series Supervisor Engine 720 can be reused with the Supervisor Engine 2T.
• It was observed that Supervisor Engine 720 Cisco IOS Software configurations may be copied and migrated to a Supervisor Engine 2T using a flash drive successfully upon boot up.
• Lab testing shows that migration between two different generation of supervisors (Supervisor Engine 720 to Supervisor Engine 2T) and new generation line cards caused minimal disruption during upgrade. However, it is recommended to upgrade the supervisors during change management windows to avoid any production traffic loss.
• It is recommended to perform the migration in steps, which should include hardware, software, and the actual migration plan.
