Managing Switch Stacks

Finding Feature Information

Your software release may not support all the features documented in this module. For the latest caveats and feature information, see Bug Search Tool and the release notes for your platform and software release. To find information about the features documented in this module, and to see a list of the releases in which each feature is supported, see the feature information table at the end of this module.

Use Cisco Feature Navigator to find information about platform support and Cisco software image support. To access Cisco Feature Navigator, go to http://www.cisco.com/go/cfn. An account on Cisco.com is not required.

Prerequisites for Switch Stacks

All the switches in the switch stack need to be running the same license level as the active switch. For information about license levels, see the System Management Configuration Guide (Catalyst 3650 Switches).

All switches in the switch stack need to be running compatible software versions.

A StackWise adapter must be installed in the stacking port to enable stacking. For switch stack hardware considerations, see the Catalyst 3650 Switch Hardware Installation Guide.

Restrictions for Switch Stacks

The following are restrictions for your switch stack configuration:

  • Switch stacks running the LAN Base license level do not support Layer 3 features.

  • A switch stack can have up to nine stacking-capable switches connected through their StackWise-160 ports.

  • You cannot have a switch stack containing a mix of Catalyst 3850 and Catalyst 3650 switches.

  • You cannot have a switch stack containing a mix of different license levels.

Information About Switch Stacks

Switch Stack Overview

A switch stack can have up to nine stacking-capable switches connected through their StackWise-160 ports. The stack members work together as a unified system. Layer 2 and Layer 3 protocols present the entire switch stack as a single entity to the network.

A switch stack always has one active switch and one standby switch. If the active switch becomes unavailable, the standby switch assumes the role of the active switch, and continues to the keep the stack operational.

The active switch controls the operation of the switch stack, and is the single point of stack-wide management. From the active switch, you configure:

  • System-level (global) features that apply to all stack members

  • Interface-level features for each stack member

The active switch contains the saved and running configuration files for the switch stack. The configuration files include the system-level settings for the switch stack and the interface-level settings for each stack member. Each stack member has a current copy of these files for back-up purposes.

Supported Features in a Switch Stack

The system-level features supported on the active switch are supported on the entire switch stack.

Encryption Features

If the active switch is running the cryptographic universal software image (supports encryption), the encryption features are available on the switch stack.

StackWise-160

The stack members use the StackWise-160 technology to work together as a unified system. Layer 2 and Layer 3 protocols support the entire switch stack as a single entity in the network.


Note

Switch stacks running the LAN Base image do not support Layer 3 features.

StackWise-160 has a stack bandwidth of 160 Gbps, and uses stateful switchover (SSO) to provide resiliency within the stack. The stack behaves as a single switching unit that is managed by an active switch elected by the member switches. The active switch automatically elects a standby switch within the stack. The active switch creates and updates all the switching, routing and wireless information and constantly synchronizes that information with the standby switch. Access points continue to remain connected during an active-to-standby switchover unless the access point is directly connected to the active switch. In this case the access point will lose power and reboot. A working stack can accept new members or delete old ones without service interruption.

Switch Stack Membership

A standalone device is a device stack with one stack member that also operates as the active switch. You can connect one standalone device to another to create a device stack containing two stack members, with one of them as the active switch. You can connect standalone devicees to an existing device stack to increase the stack membership.

Hello messages are sent and received by all stack members.
  • If a stack member does not respond, that member is removed from the stack.

  • If the standby device does not respond, a new standby device is elected.

  • If the active device does not respond, the standby device becomes the active device.

In addition, keepalive messages are sent and received between the active and standby devicees.
  • If the standby device does not respond, a new standby device is elected.

  • If the active device does not respond, the standby device becomes the active device.

Changes to Switch Stack Membership

If you replace a stack member with an identical model, the new switch functions with exactly the same configuration as the replaced switch, assuming that the new switch (referred to as the provisioned switch) is using the same member number as the replaced switch.

The operation of the switch stack continues uninterrupted during membership changes unless you remove the active switch or you add powered-on standalone switches or switch stacks.

  • Adding powered-on switches (merging) causes all switches to reload and elect a new active switch from among themselves. The newly elected active switch retains its role and configuration. All other switches retain their stack member numbers and use the stack configuration of the newly elected active switch.

  • Removing powered-on stack members causes the switch stack to divide (partition) into two or more switch stacks, each with the same configuration. This can cause:
    • An IP address conflict in your network. If you want the switch stacks to remain separate, change the IP address or addresses of the newly created switch stacks.

    • A MAC address conflict between two members in the stack. You can use the stack-mac update force command to resolve the conflict.

If a newly created switch stack does not have an active switch or standby switch, the switch stack will reload and elect a new active switch.


Note

Make sure that you power off the switches that you add to or remove from the switch stack.

After adding or removing stack members, make sure that the switch stack is operating at full bandwidth (160 Gbps). Press the Mode button on a stack member until the Stack mode LED is on. The last two right port LEDs on all switches in the stack should be green. Depending on the switch model, the last two right ports are 10-Gigabit Ethernet ports or small form-factor pluggable (SFP) module ports (10/100/1000 ports). If one or both of these LEDs are not green on any of the switches, the stack is not operating at full bandwidth.


If you remove powered-on members but do not want to partition the stack:

  • Power off the switches in the newly created switch stacks.

  • Reconnect them to the original switch stack through their stack ports.

  • Power on the switches.

For cabling and power considerations that affect switch stacks, see the Catalyst 3650 Switch Hardware Installation Guide .

Stack Member Numbers

The stack member number (1 to 9) identifies each member in the Device stack. The member number also determines the interface-level configuration that a stack member uses. You can display the stack member number by using the show switch EXEC command.

A new, out-of-the-box Device (one that has not joined a Device stack or has not been manually assigned a stack member number) ships with a default stack member number of 1. When it joins a Device stack, its default stack member number changes to the lowest available member number in the stack.

Stack members in the same Device stack cannot have the same stack member number. Every stack member, including a standalone Device, retains its member number until you manually change the number or unless the number is already being used by another member in the stack.

  • If you manually change the stack member number by using the switch current-stack-member-number renumber new-stack-member-number command, the new number goes into effect after that stack member resets (or after you use the reload slot stack-member-number privileged EXEC command) and only if that number is not already assigned to any other members in the stack. Another way to change the stack member number is by changing the Device_NUMBER environment variable.

    If the number is being used by another member in the stack, the Device selects the lowest available number in the stack.

    If you manually change the number of a stack member and no interface-level configuration is associated with that new member number, that stack member resets to its default configuration.

    You cannot use the switch current-stack-member-number renumber new-stack-member-number command on a provisioned Device. If you do, the command is rejected.

  • If you move a stack member to a different Device stack, the stack member retains its number only if the number is not being used by another member in the stack. If it is being used, the Device selects the lowest available number in the stack.

  • If you merge Device stacks, the Device that join the Device stack of a new active switch select the lowest available numbers in the stack.

As described in the hardware installation guide, you can use the Device port LEDs in Stack mode to visually determine the stack member number of each stack member.

In the default mode Stack LED will blink in green color only on the stack master. However, when we scroll the Mode button to Stack option - Stack LED will glow green on all the stack members.

When mode button is scrolled to Stack option, the switch number of each stack member will be displayed as LEDs on the first five ports of that switch. The switch number is displayed in binary format for all stack members. On the switch, the amber LED indicates value 0 and green LED indicates value 1.

Example for switch number 5 (Binary - 00101):

First five LEDs will glow in below color combination on stack member with switch number 5.

  • Port-1 : Amber

  • Port-2 : Amber

  • Port-3 : Green

  • Port-4 : Amber

  • Port-5 : Green

Similarly first five LEDs will glow in amber or green, depending on the switch number on all stack members.


Note

  • If we connect a Horizontal stack port to a normal network port on other end, stack port transmission/reception will be disabled within 30 seconds if there are no SDP packet received from the other end.

  • Stack port will not go down but only transmission/reception will be disabled. The log message shown below will be displayed on the console. Once the peer end network port is converted to stack port, transmission/reception on this stack port will be enabled.

%STACKMGR-4-HSTACK_LINK_CONFIG: Verify peer stack port setting for hstack StackPort-1 switch 5 (hostname-switchnumber)

Stack Member Priority Values

A higher priority value for a stack member increases the probability of it being elected active switch and retaining its stack member number. The priority value can be 1 to 15. The default priority value is 1. You can display the stack member priority value by using the show switch EXEC command.


Note

We recommend assigning the highest priority value to the device that you prefer to be the active switch. This ensures that the device is reelected as the active switch if a reelection occurs.


To change the priority value for a stack member, use the switch stack-member-number priority new priority-value command. For more information, see the “Setting the Stack Member Priority Value” section.

The new priority value takes effect immediately but does not affect the current active switch. The new priority value helps determine which stack member is elected as the new active switch when the current active switch or the switch stack resets.

Switch Stack Bridge ID and MAC Address

A switch stack is identified in the network by its bridge ID and, if it is operating as a Layer 3 device, its router MAC address. The bridge ID and router MAC address are determined by the MAC address of the active switch.

If the active switch changes, the MAC address of the new active switch determines the new bridge ID and router MAC address.

If the entire switch stack reloads, the switch stack uses the MAC address of the active switch.

Persistent MAC Address on the Switch Stack

You can use the persistent MAC address feature to set a time delay before the stack MAC address changes. During this time period, if the previous active switch rejoins the stack, the stack continues to use its MAC address as the stack MAC address, even if the switch is now a stack member and not an active switch. If the previous active switch does not rejoin the stack during this period, the switch stack takes the MAC address of the new active switch as the stack MAC address. By default, the stack MAC address will be the MAC address of the first active switch, even if a new active switch takes over.

You can also configure stack MAC persistency so that the stack MAC address never changes to the new active switch MAC address.

Active and Standby Switch Election and Reelection

All stack members are eligible to be the active switch or the standby switch. If the active switch becomes unavailable, the standby switch becomes the active switch.

An active switch retains its role unless one of these events occurs:

  • The switch stack is reset.

  • The active switch is removed from the switch stack.

  • The active switch is reset or powered off.

  • The active switch fails.

  • The switch stack membership is increased by adding powered-on standalone switches or switch stacks.

The active switch is elected or reelected based on one of these factors and in the order listed:

  1. The switch that is currently the active switch.

  2. The switch with the highest stack member priority value.


    Note

    We recommend assigning the highest priority value to the switch that you prefer to be the active switch. This ensures that the switch is reelected as active switch if a reelection occurs.


  3. The switch with the shortest start-up time.

  4. The switch with the lowest MAC address.


Note

The factors for electing or reelecting a new standby switch are same as those for the active switch election or reelection, and are applied to all participating switches except the active switch.

After election, the new active switch becomes available after a few seconds. In the meantime, the switch stack uses the forwarding tables in memory to minimize network disruption. The physical interfaces on the other available stack members are not affected during a new active switch election and reset.

When the previous active switch becomes available, it does not resume its role as the active switch.

If you power on or reset an entire switch stack, some stack members might not participate in the active switch election. Stack members that are powered on within the same 2-minute timeframe participate in the active switch election and have a chance to become the active switch. Stack members that are powered on after the 120-second timeframe do not participate in this initial election and become stack members. For powering considerations that affect active-switch elections, see the switch hardware installation guide.

As described in the hardware installation guide, you can use the ACTV LED on the switch to see if the switch is the active switch.

Switch Stack Configuration Files

The active switch has the saved and running configuration file for the switch stack. The standby switch automatically receives the synchronized running configuration file. Stack members receive synchronized copies when the running configuration file is saved into the startup configuration file. If the active switch becomes unavailable, the standby switch takes over with the current running configuration.

The configuration files record these settings:

  • System-level (global) configuration settings such as IP, STP, VLAN, and SNMP settings that apply to all stack members

  • Stack member interface-specific configuration settings that are specific for each stack member


Note

The interface-specific settings of the active switch are saved if the active switch is replaced without saving the running configuration to the startup configuration.

A new, out-of-box device joining a switch stack uses the system-level settings of that switch stack. If a device is moved to a different switch stack before it is powered on, that device loses its saved configuration file and uses the system-level configuration of the new switch stack. If the device is powered on as a standalone device before it joins the new switch stack, the stack will reload. When the stack reloads, the new device may become the active switch, retain its configuration and overwrite the configuration files of the other stack members.

The interface-specific configuration of each stack member is associated with the stack member number. Stack members retain their numbers unless they are manually changed or they are already used by another member in the same switch stack. If the stack member number changes, the new number goes into effect after that stack member resets.

  • If an interface-specific configuration does not exist for that member number, the stack member uses its default interface-specific configuration.

  • If an interface-specific configuration exists for that member number, the stack member uses the interface-specific configuration associated with that member number.

If you replace a failed member with an identical model, the replacement member automatically uses the same interface-specific configuration as the failed device. You do not need to reconfigure the interface settings. The replacement device (referred to as the provisioned device) must have the same stack member number as the failed device.

You back up and restore the stack configuration in the same way as you would for a standalone device configuration.

Offline Configuration to Provision a Stack Member

You can use the offline configuration feature to provision (to supply a configuration to) a new switch before it joins the switch stack. You can configure the stack member number, the switch type, and the interfaces associated with a switch that is not currently part of the stack. The configuration that you create on the switch stack is called the provisioned configuration. The switch that is added to the switch stack and that receives this configuration is called the provisioned switch.

You manually create the provisioned configuration through the switch stack-member-number provision type global configuration command. You must change the stack-member-number on the provisioned switch before you add it to the stack, and it must match the stack member number that you created for the new switch on the switch stack. The switch type in the provisioned configuration must match the switch type of the newly added switch. The provisioned configuration is automatically created when a switch is added to a switch stack and when no provisioned configuration exists.

When you configure the interfaces associated with a provisioned switch, the switch stack accepts the configuration, and the information appears in the running configuration. However, as the switch is not active, any configuration on the interface is not operational and the interface associated with the provisioned switch does not appear in the display of the specific feature. For example, VLAN configuration information associated with a provisioned switch does not appear in the show vlan user EXEC command output on the switch stack.

The switch stack retains the provisioned configuration in the running configuration whether or not the provisioned switch is part of the stack. You can save the provisioned configuration to the startup configuration file by entering the copy running-config startup-config privileged EXEC command. The startup configuration file ensures that the switch stack can reload and can use the saved information whether or not the provisioned switch is part of the switch stack.

Effects of Adding a Provisioned Switch to a Switch Stack

When you add a provisioned Device to the switch stack, the stack applies either the provisioned configuration or the default configuration. This table lists the events that occur when the switch stack compares the provisioned configuration with the provisioned switch.

Table 1. Results of Comparing the Provisioned Configuration with the Provisioned Switch

Scenario

Result

The stack member numbers and the Device types match.

  1. If the stack member number of the provisioned switch matches the stack member number in the provisioned configuration on the stack, and

  2. If the Device type of the provisioned switch matches the Device type in the provisioned configuration on the stack.

The switch stack applies the provisioned configuration to the provisioned switch and adds it to the stack.

The stack member numbers match but the Device types do not match.

  1. If the stack member number of the provisioned switch matches the stack member number in the provisioned configuration on the stack, but

  2. The Device type of the provisioned switch does not match the Device type in the provisioned configuration on the stack.

The switch stack applies the default configuration to the provisioned switch and adds it to the stack.

The provisioned configuration is changed to reflect the new information.

The stack member number is not found in the provisioned configuration.

The switch stack applies the default configuration to the provisioned switch and adds it to the stack.

The provisioned configuration is changed to reflect the new information.

The stack member number of the provisioned switch is not found in the provisioned configuration.

The switch stack applies the default configuration to the provisioned switch and adds it to the stack.

If you add a provisioned switch that is a different type than specified in the provisioned configuration to a powered-down switch stack and then apply power, the switch stack rejects the (now incorrect) switch stack-member-number provision type global configuration command in the startup configuration file. However, during stack initialization, the nondefault interface configuration information in the startup configuration file for the provisioned interfaces (potentially of the wrong type) is executed. Depending on the differences between the actual Device type and the previously provisioned switch type, some commands are rejected, and some commands are accepted.


Note

If the switch stack does not contain a provisioned configuration for a new Device, the Device joins the stack with the default interface configuration. The switch stack then adds to its running configuration with a switch stack-member-number provision type global configuration command that matches the new Device. For configuration information, see the Provisioning a New Member for a Switch Stack section.


Effects of Replacing a Provisioned Switch in a Switch Stack

When a provisioned switch in a switch stack fails, it is removed from the stack, and is replaced with another Device, the stack applies either the provisioned configuration or the default configuration to it. The events that occur when the switch stack compares the provisioned configuration with the provisioned switch are the same as those when you add a provisioned switch to a stack.

Effects of Removing a Provisioned Switch from a Switch Stack

If you remove a provisioned switch from the switch stack, the configuration associated with the removed stack member remains in the running configuration as provisioned information. To completely remove the configuration, use the no switch stack-member-number provision global configuration command.

Upgrading a Switch Running Incompatible Software

The auto-upgrade and auto-advise features enable a switch with software packages that are incompatible with the switch stack to be upgraded to a compatible software version so that it can join the switch stack.

Auto-Upgrade

The purpose of the auto-upgrade feature is to allow a switch to be upgraded to a compatible software image, so that the switch can join the switch stack.

When a new switch attempts to join a switch stack, each stack member performs compatibility checks with itself and the new switch. Each stack member sends the results of the compatibility checks to the active switch, which uses the results to determine whether the switch can join the switch stack. If the software on the new switch is incompatible with the switch stack, the new switch enters version-mismatch (VM) mode.

If the auto-upgrade feature is enabled on the existing switch stack, the active switch automatically upgrades the new switch with the same software image running on a compatible stack member. Auto-upgrade starts a few minutes after the mismatched software is detected before starting.

Auto-upgrade is disabled by default.

Auto-upgrade includes an auto-copy process and an auto-extract process.

  • Auto-copy automatically copies the software image running on any stack member to the new switch to automatically upgrade it. Auto-copy occurs if auto-upgrade is enabled, if there is enough flash memory in the new switch, and if the software image running on the switch stack is suitable for the new switch.


    Note

    A switch in VM mode might not run all released software. For example, new switch hardware is not recognized in earlier versions of software.
  • Automatic extraction (auto-extract) occurs when the auto-upgrade process cannot find the appropriate software in the stack to copy to the new switch. In that case, the auto-extract process searches all switches in the stack for the bin file needed to upgrade the switch stack or the new switch. The bin file can be in any flash file system in the switch stack or in the new switch. If a bin file suitable for the new switch is found on a stack member, the process extracts the file and automatically upgrades the new switch.

The auto-upgrade feature is not available in bundle mode. The switch stack must be running in installed mode. If the switch stack is in bundle mode, use the software expand privileged EXEC command to change to installed mode.

You can enable auto-upgrade by using the software auto-upgrade enable global configuration command on the new switch. You can check the status of auto-upgrade by using the show running-config privileged EXEC command and by checking the Auto upgrade line in the display.

You can configure auto-upgrade to upgrade the new switch with a specific software bundle by using the software auto-upgrade source url global configuration command. If the software bundle is invalid, the new switch is upgraded with the same software image running on a compatible stack member.

When the auto-upgrade process is complete, the new switch reloads and joins the stack as a fully functioning member. If you have both stack cables connected during the reload, network downtime does not occur because the switch stack operates on two rings.

For more information about upgrading a switch running incompatible software see the Cisco IOS File System, Configuration Files, and Bundle Files Appendix, Cisco IOS XE Release 3SE (Catalyst 3650 Switches).

Auto-Advise

The auto-advise feature is triggered when:

  • The auto-upgrade feature is disabled.

  • The new switch is in bundle mode and the stack is in installed mode. Auto-advise displays syslog messages about using the software auto-upgrade privileged EXEC command to change the new switch to installed mode.

  • The stack is in bundle mode. Auto-advise displays syslog messages about booting the new switch in bundle mode so that it can join the stack.

  • An auto-upgrade attempt fails because the new switch is running incompatible software. After the switch stack performs compatibility checks with the new switch, auto-advise displays syslog messages about whether the new switch can be auto-upgraded.

Auto-advise cannot be disabled. It does not give suggestions when the switch stack software and the software of the switch in version-mismatch (VM) mode do not contain the same license level.

Examples of Auto-Advise Messages
Auto-Upgrade Is Disabled and Incompatible Switch Attempting to Join: Example

This sample auto-advise output shows the system messages displayed when the auto-upgrade feature is disabled and an incompatible switch 1 tries to join the switch stack:

*Oct 18 08:36:19.379: %INSTALLER-6-AUTO_ADVISE_SW_INITIATED: 2 installer:  Auto advise initiated for switch 1
*Oct 18 08:36:19.380: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  Searching stack for software to upgrade switch 1
*Oct 18 08:36:19.382: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  Switch 1 with incompatible software has been
*Oct 18 08:36:19.382: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  added to the stack. The software running on
*Oct 18 08:36:19.382: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  all stack members was scanned and it has been
*Oct 18 08:36:19.382: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  determined that the 'software auto-upgrade'
*Oct 18 08:36:19.382: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  command can be used to install compatible
*Oct 18 08:36:19.382: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  software on switch 1.

Auto-Upgrade is Disabled and New Switch is in Bundle Mode: Example

This sample auto-advise output shows the system messages displayed when auto-upgrade is disabled and a switch running in bundle mode tries to join the stack that is running in installed mode:

*Oct 18 11:09:47.005: %INSTALLER-6-AUTO_ADVISE_SW_INITIATED: 2 installer:  Auto advise initiated for switch 1
*Oct 18 11:09:47.005: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  Switch 1 running bundled software has been added
*Oct 18 11:09:47.005: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  to the stack that is running installed software.
*Oct 18 11:09:47.005: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  The 'software auto-upgrade' command can be used to
*Oct 18 11:09:47.005: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  convert switch 1 to the installed running mode by
*Oct 18 11:09:47.005: %INSTALLER-6-AUTO_ADVISE_SW: 2 installer:  installing its running software.

Switch Stack Management Connectivity

You manage the switch stack and the stack member interfaces through the active switch. You can use the CLI, SNMP, and supported network management applications such as CiscoWorks. You cannot manage stack members on an individual Device basis.


Note

Use SNMP to manage network features across the stack that are defined by supported MIBs. The switch does not support MIBs to manage stacking-specific features such as stack membership and election.

Connectivity to the Switch Stack Through an IP Address

The switch stack is managed through a single IP address. The IP address is a system-level setting and is not specific to the active switch or to any other stack member. You can still manage the stack through the same IP address even if you remove the active switch or any other stack member from the stack, provided there is IP connectivity.


Note

Stack members retain their IP addresses when you remove them from a switch stack. To avoid a conflict by having two devices with the same IP address in your network, change the IP addresses of any Device that you remove from the switch stack.


For related information about switch stack configurations, see the Switch Stack Configuration Files section.

Connectivity to the Switch Stack Through Console Ports or Ethernet Management Ports

You can connect to the active switch by using one of these methods:

  • You can connect a terminal or a PC to the active switch through the console port of one or more stack members.

  • You can connect a PC to the active switch through the Ethernet management ports of one or more stack members. For more information about connecting to the switch stack through Ethernet management ports, see the Using the Ethernet Management Port section.

You can connect to the active switch by connecting a terminal or a PC to the stack master through the console port of one or more stack members.

Be careful when using multiple CLI sessions to the active switch. Commands that you enter in one session are not displayed in the other sessions. Therefore, it is possible that you might not be able to identify the session from which you entered a command.

We recommend using only one CLI session when managing the switch stack.

How to Configure a Switch Stack

Enabling the Persistent MAC Address Feature


Note

When you enter the command to configure this feature, a warning message appears with the consequences of your configuration. You should use this feature cautiously. Using the old active switch MAC address elsewhere in the same domain could result in lost traffic.

Follow these steps to enable persistent MAC address:

SUMMARY STEPS

  1. enable
  2. configure terminal
  3. stack-mac persistent timer [0 | time-value ]
  4. end
  5. copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose
Step 1

enable

Example:


Device> enable

Enables privileged EXEC mode. Enter your password if prompted.

Step 2

configure terminal

Example:


Device# configure terminal

Enters the global configuration mode.

Step 3

stack-mac persistent timer [0 | time-value ]

Example:

Device(config)# stack-mac persistent timer 7

Enables a time delay after an active-switch change before the stack MAC address changes to that of the new active switch. If the previous active switch rejoins the stack during this period, the stack uses that MAC address as the stack MAC address.

  • Enter the command with no value or with a value of 0 to continue using the MAC address of the current active switch indefinitely.

  • Enter a time-value from 1 to 60 minutes to configure the time period before the stack MAC address changes to the new active switch.

    The stack MAC address of the previous active switch is used until the configured time period expires.

Step 4

end

Example:


Device(config)# end

Returns to privileged EXEC mode.

Step 5

copy running-config startup-config

Example:


Device# copy running-config startup-config 

(Optional) Saves your entries in the configuration file.

What to do next

Use the no stack-mac persistent timer global configuration command to disable the persistent MAC address feature.

Assigning a Stack Member Number

This optional task is available only from the active switch.

Follow these steps to assign a member number to a stack member:

SUMMARY STEPS

  1. enable
  2. configure terminal
  3. switch current-stack-member-number renumber new-stack-member-number
  4. end
  5. reload slot stack-member-number
  6. show switch
  7. copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose
Step 1

enable

Example:


Device> enable

Enables privileged EXEC mode. Enter your password if prompted.

Step 2

configure terminal

Example:


Device# configure terminal

Enters the global configuration mode.

Step 3

switch current-stack-member-number renumber new-stack-member-number

Example:

Device(config)# switch 3 renumber 4

Specifies the current stack member number and the new stack member number for the stack member. The range is 1 to 9.

Specifies the current stack member number and the new member number for the stack member. The range is 1 to 2.

You can display the current stack member number by using the show switch user EXEC command.

Step 4

end

Example:


Device(config)# end

Returns to privileged EXEC mode.

Step 5

reload slot stack-member-number

Example:

Device# reload slot 4

Resets the stack member.

Step 6

show switch

Example:

showDevice

Verify the stack member number.

Step 7

copy running-config startup-config

Example:


Device# copy running-config startup-config 

(Optional) Saves your entries in the configuration file.

Setting the Stack Member Priority Value

This optional task is available only from the active switch.

Follow these steps to assign a priority value to a stack member:

SUMMARY STEPS

  1. enable
  2. switch stack-member-number priority new-priority-number
  3. end
  4. show switch stack-member-number
  5. copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose
Step 1

enable

Example:


Device> enable

Enables privileged EXEC mode. Enter your password if prompted.

Step 2

switch stack-member-number priority new-priority-number

Example:

Device# switch 3 priority 2

Specifies the stack member number and the new priority for the stack member. The stack member number range is 1 to 9. The priority value range is 1 to 15.

You can display the current priority value by using the show switch user EXEC command.

The new priority value takes effect immediately but does not affect the current active switch. The new priority value helps determine which stack member is elected as the new active switch when the current active switch or switch stack resets.

Step 3

end

Example:

Device# end

Returns to priviledge EXEC mode.

Step 4

show switch stack-member-number

Example:

Device# show switch

Verify the stack member priority value.

Step 5

copy running-config startup-config

Example:


Device# copy running-config startup-config 

(Optional) Saves your entries in the configuration file.

Provisioning a New Member for a Switch Stack

This optional task is available only from the active switch.

SUMMARY STEPS

  1. show switch
  2. configure terminal
  3. switch stack-member-number provision type
  4. end
  5. copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose
Step 1

show switch

Example:

Device# show switch

Displays summary information about the switch stack.

Step 2

configure terminal

Example:

Device#  configure terminal

Enters global configuration mode.

Step 3

switch stack-member-number provision type

Example:

Device(config)# switch 3 provision WS-xxxx

Specifies the stack member number for the preconfigured switch. By default, no switches are provisioned.

For stack-member-number , the range is 1 to 9. Specify a stack member number that is not already used in the switch stack. See Step 1.

For type , enter the model number of a supported switch that is listed in the command-line help strings.

Step 4

end

Example:

Device(config)# end

Returns to privileged EXEC mode.

Step 5

copy running-config startup-config

Example:

Device# copy running-config startup-config

(Optional) Saves your entries in the configuration file.

Removing Provisioned Switch Information

Before you begin, you must remove the provisioned switch from the stack. This optional task is available only from the active switch.

SUMMARY STEPS

  1. configure terminal
  2. no switch stack-member-number provision
  3. end
  4. copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose
Step 1

configure terminal

Example:

Device# configure terminal

Enters global configuration mode.

Step 2

no switch stack-member-number provision

Example:

Device(config)# no switch 3 provision

Removes the provisioning information for the specified member.

Step 3

end

Example:

Device(config)# end

Returns to privileged EXEC mode.

Step 4

copy running-config startup-config

Example:

Device# copy running-config startup-config

(Optional) Saves your entries in the configuration file.

Example

If you are removing a provisioned switch in a stack with this configuration:

  • The stack has four members

  • Stack member 1 is the active switch

  • Stack member 3 is a provisioned switch

and want to remove the provisioned information and to avoid receiving an error message, you can remove power from stack member 3, disconnect the StackWise-160 cables between the stack member 3 and switches to which it is connected, reconnect the cables between the remaining stack members, and enter the no switch stack-member-number provision global configuration command.

Displaying Incompatible Switches in the Switch Stack

SUMMARY STEPS

  1. show switch

DETAILED STEPS

Command or Action Purpose

show switch

Example:

Device# show switch

Displays any incompatible switches in the switch stack (indicated by a 'Current State' of 'V-Mismatch'). The V-Mismatch state identifies the switches with incompatible software. The output displays Lic-Mismatch for switches that are not running the same license level as the active switch.

For information about managing license levels, see the System Management Configuration Guide (Catalyst 3650 Switches).

Upgrading an Incompatible Switch in the Switch Stack

SUMMARY STEPS

  1. software auto-upgrade
  2. copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose
Step 1

software auto-upgrade

Example:

Device# software auto-upgrade

Upgrades incompatible switches in the switch stack, or changes switches in bundle mode to installed mode.

Step 2

copy running-config startup-config

Example:

Device# copy running-config startup-config

(Optional) Saves your entries in the configuration file.

Troubleshooting the Switch Stack

Temporarily Disabling a Stack Port

If a stack port is flapping and causing instability in the stack ring, to disable the port, enter the switch stack-member-number stack port port-number disable privileged EXEC command. To reenable the port, enter the switch stack-member-number stack port port-number enable command.


Note

Be careful when using the switch stack-member-number stack port port-number disable command. When you disable the stack port, the stack operates at half bandwidth.

A stack is in the full-ring state when all members are connected through the stack ports and are in the ready state.

The stack is in the partial-ring state when the following occurs:

  • All members are connected through their stack ports but some are not in the ready state.

  • Some members are not connected through the stack ports.

SUMMARY STEPS

  1. switch stack-member-number stack port port-number disable
  2. switch stack-member-number stack port port-number enable

DETAILED STEPS

  Command or Action Purpose
Step 1

switch stack-member-number stack port port-number disable

Example:

Device# switch 2 stack port 1 disable

Disables the specified stack port.

Step 2

switch stack-member-number stack port port-number enable

Example:

Device# switch 2 stack port 1 enable

Reenables the stack port.

When you disable a stack port and the stack is in the full-ring state, you can disable only one stack port. This message appears:

Enabling/disabling a stack port may cause undesired stack changes. Continue?[confirm] 

When you disable a stack port and the stack is in the partial-ring state, you cannot disable the port. This message appears:

Disabling stack port not allowed with current stack configuration.

Reenabling a Stack Port While Another Member Starts

Stack Port 1 on Switch 1 is connected to Port 2 on Switch 4. If Port 1 is flapping, you can disable Port 1 with the switch 1 stack port 1 disable privileged EXEC command. While Port 1 on Switch 1 is disabled and Switch 1 is still powered on, follow these steps to reenable a stack port:

Procedure


Step 1

Disconnect the stack cable between Port 1 on Switch 1 and Port 2 on Switch 4.

Step 2

Remove Switch 4 from the stack.

Step 3

Add a switch to replace Switch 4 and assign it switch-number 4.

Step 4

Reconnect the cable between Port 1 on Switch 1 and Port 2 on Switch 4 (the replacement switch).

Step 5

Reenable the link between the switches. Enter the switch 1 stack port 1 enable privileged EXEC command to enable Port 1 on Switch 1.

Step 6

Power on Switch 4.



Caution

Powering on Switch 4 before enabling the Port 1 on Switch 1 might cause one of the switches to reload.

If Switch 4 is powered on first, you might need to enter the switch 1 stack port 1 enable and the switch 4 stack port 2 enable privileged EXEC commands to bring up the link.


Monitoring the Device Stack

Table 2. Commands for Displaying Stack Information

Command

Description

show switch

Displays summary information about the stack, including the status of provisioned switches and switches in version-mismatch mode.

show switch stack-member-number

Displays information about a specific member.

show switch detail

Displays detailed information about the stack.

show switch neighbors

Displays the stack neighbors.

show switch stack-ports [summary ]

Displays port information for the stack.

show redundancy

Displays the redundant system and the current processor information. The redundant system information includes the system uptime, standby failures, switchover reason, hardware, configured and operating redundancy mode. The current processor information displayed includes the active location, the software state, the uptime in the current state and so on.

show redundancy state

Displays all the redundancy states of the active and standby devices.

Configuration Examples for Switch Stacks

Switch Stack Configuration Scenarios

Most of these switch stack configuration scenarios assume that at least two device are connected through their StackWise-160 ports.

Table 3. Configuration Scenarios

Scenario

Result

Active switch election specifically determined by existing active switches

Connect two powered-on switch stacks through the StackWise-160 ports.

Only one of the two active switches becomes the new active switch.

Active switch election specifically determined by the stack member priority value

  1. Connect two switches through their StackWise-160 ports.

  2. Use the switch stack-member-number priority new-priority-number global configuration command to set one stack member with a higher member priority value.

  3. Restart both stack members at the same time.

The stack member with the higher priority value is elected active switch.

Active switch election specifically determined by the configuration file

Assuming that both stack members have the same priority value:

  1. Make sure that one stack member has a default configuration and that the other stack member has a saved (nondefault) configuration file.

  2. Restart both stack members at the same time.

The stack member with the saved configuration file is elected active switch.

Active switch election specifically determined by the MAC address

Assuming that both stack members have the same priority value, configuration file, and feature set, restart both stack members at the same time.

The stack member with the lower MAC address is elected active switch.

Stack member number conflict

Assuming that one stack member has a higher priority value than the other stack member:

  1. Ensure that both stack members have the same stack member number. If necessary, use the switch current-stack-member-number renumber new-stack-member-number global configuration command.

  2. Restart both stack members at the same time.

The stack member with the higher priority value retains its stack member number. The other stack member has a new stack member number.

Add a stack member

  1. Power off the new switch.

  2. Through their StackWise-160 ports, connect the new switch to a powered-on switch stack.

  3. Power on the new switch.

The active switch is retained. The new switch is added to the switch stack.

Active switch failure

Remove (or power off) the active switch.

One of the remaining stack members becomes the new stack master. All other stack members in the stack remain as stack members and do not reboot.

Add more than nine stack members

  1. Through their StackWise-160 ports, connect ten device.

  2. Power on all device.

Two device become active switches. One active switch has nine stack members. The other active switch remains as a standalone device.

Use the Mode button and port LEDs on the device to identify which device are active switches and which device belong to each active switch.

Enabling the Persistent MAC Address Feature: Example

This example shows how to configure the persistent MAC address feature for a 7-minute time delay and to verify the configuration:

Device(config)# stack-mac persistent timer 7
		WARNING: The stack continues to use the base MAC of the old Master
		WARNING: as the stack MAC after a master switchover until the MAC
		WARNING: persistency timer expires. During this time the Network
		WARNING: Administrators must make sure that the old stack-mac does
		WARNING: not appear elsewhere in this network domain. If it does,
		WARNING: user traffic may be blackholed.
		Device(config)# end
		Device# show switch
		Switch/Stack Mac Address : 0016.4727.a900
		Mac persistency wait time: 7 mins
		                                           H/W   Current
		Switch#  Role   Mac Address     Priority Version  State 
		----------------------------------------------------------
		*1       Active 0016.4727.a900     1      P2B     Ready
 
		

Provisioning a New Member for a Switch Stack: Example

This example shows how to provision a switch with a stack member number of 2 for the switch stack. The show running-config command output shows the interfaces associated with the provisioned switch:

Device(config)# switch 2 provision switch_PID
		Device(config)# end
		Device# show running-config | include switch 2
		!
		interface GigabitEthernet2/0/1
		!
		interface GigabitEthernet2/0/2
		!
		interface GigabitEthernet2/0/3
		<output truncated>

		

show switch stack-ports summary Command Output: Example

Only Port 1 on stack member 2 is disabled.

Device# show switch stack-ports summary
		Device#/  Stack   Neighbor   Cable    Link   Link   Sync      #         In  
		 Port#     Port              Length    OK   Active   OK    Changes   Loopback
		          Status                                          To LinkOK          
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1     OK         3      50 cm     Yes    Yes    Yes        1        No 
		  1/2     Down      None    3 m       Yes    No     Yes        1        No 
		  2/1     Down      None    3 m       Yes    No     Yes        1        No 
		  2/2     OK         3      50 cm     Yes    Yes    Yes        1        No 
		  3/1     OK         2      50 cm     Yes    Yes    Yes        1        No 
		  3/2     OK         1      50 cm     Yes    Yes    Yes        1        No 

		
Table 4. show switch stack-ports summary Command Output

Field

Description

Switch#/Port#

Member number and its stack port number.

Stack Port Status

Status of the stack port.

  • Absent—No cable is detected on the stack port.

  • Down—A cable is detected, but either no connected neighbor is up, or the stack port is disabled.

  • OK—A cable is detected, and the connected neighbor is up.

Neighbor

Switch number of the active member at the other end of the stack cable.

Cable Length

Valid lengths are 50 cm, 1 m, or 3 m.

If the switch cannot detect the cable length, the value is no cable. The cable might not be connected, or the link might be unreliable.

Link OK

Whether the stack cable is connected and functional. There may or may not be a neighbor connected on the other end.

The link partner is a stack port on a neighbor switch.

  • No—There is no stack cable connected to this port or the stack cable is not functional.

  • Yes—There is a functional stack cable connected to this port.

Link Active

Whether a neighbor is connected on the other end of the stack cable.

  • No—No neighbor is detected on the other end. The port cannot send traffic over this link.

  • Yes—A neighbor is detected on the other end. The port can send traffic over this link.

Sync OK

Whether the link partner sends valid protocol messages to the stack port.

  • No—The link partner does not send valid protocol messages to the stack port.

  • Yes—The link partner sends valid protocol messages to the port.

# Changes to LinkOK

The relative stability of the link.

If a large number of changes occur in a short period of time, link flapping can occur.

In Loopback

Whether a stack cable is attached to a stack port on the member.

  • No—At least one stack port on the member has an attached stack cable.

  • Yes—None of the stack ports on the member has an attached stack cable.

Software Loopback: Examples

In a stack with three members, stack cables connect all the members:

Device# show switch stack-ports summary
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync   #Changes   In
            Status            Length    OK    Active  OK     To LinkOK  Loopback         
		--------  ------  --------  --------  ----  ------  ----  ---------   --------
		  1/1     OK         3      50 cm     Yes    Yes    Yes        1        No 
		  1/2     OK         2      3 m       Yes    Yes    Yes        1        No 
		  2/1     OK         1      3 m       Yes    Yes    Yes        1        No 
		  2/2     OK         3      50 cm     Yes    Yes    Yes        1        No 
		  3/1     OK         2      50 cm     Yes    Yes    Yes        1        No 
		  3/2     OK         1      50 cm     Yes    Yes    Yes        1        No 
		
		

If you disconnect the stack cable from Port 1 on Switch 1, these messages appear:

01:09:55: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 3 has changed to state DOWN
		01:09:56: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state DOWN
		
		Device# show switch stack-ports summary 
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback            
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1     Absent    None    No cable  No     No     No         1        No 
		  1/2     OK         2      3 m       Yes    Yes    Yes        1        No 
		  2/1     OK         1      3 m       Yes    Yes    Yes        1        No 
		  2/2     OK         3      50 cm     Yes    Yes    Yes        1        No 
		  3/1     OK         2      50 cm     Yes    Yes    Yes        1        No 
		  3/2     Down      None    50 cm     No     No     No         1        No 
		
		

If you disconnect the stack cable from Port 2 on Switch 1, the stack splits.

Switch 2 and Switch 3 are now in a two-member stack connected through stack cables:

Device# show sw stack-ports summary 
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback  
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  2/1     Down      None    3 m       No     No     No         1        No 
		  2/2     OK         3      50 cm     Yes    Yes    Yes        1        No 
		  3/1     OK         2      50 cm     Yes    Yes    Yes        1        No 
		  3/2     Down      None    50 cm     No     No     No         1        No 
		
		

Switch 1 is a standalone switch:

Device# show switch stack-ports summary 
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback  
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1     Absent    None    No cable  No     No     No         1        Yes
		  1/2     Absent    None    No cable  No     No     No         1        Yes

Software Loopback with Connected Stack Cables: Examples

  • On Port 1 on Switch 1, the port status is Down, and a cable is connected.

    On Port 2 on Switch 1, the port status is Absent, and no cable is connected.

    Device# show switch stack-ports summary
    		Device#
     Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
                Status            Length    OK    Active  OK    To LinkOK  Loopback  
    		--------  ------  --------  --------  ---- ------  ----  ---------  --------
    		  1/1     Down      None    50 Cm      No     No     No       1          No   
    		  1/2     Absent    None    No cable   No     No     No       1          No
    		
    		
  • In a physical loopback, a cable connects both stack ports on a switch. You can use this configuration to test
    • Cables on a switch that is running properly

    • Stack ports with a cable that works properly

    Device# show switch stack-ports summary
    		Device#
     Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
                Status            Length    OK    Active  OK    To LinkOK  Loopback  
    		--------  ------  --------  --------  ----  ------  ----  ---------  --------
    		  2/1       OK       2      50 cm     Yes    Yes    Yes       1         No
    		  2/2       OK       2      50 cm     Yes    Yes    Yes       1         No
    		
    		

    The port status shows that

    • Switch 2 is a standalone switch.

    • The ports can send and receive traffic.

Software Loopback with no Connected Stack Cable: Example

Device# show switch stack-ports summary
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback  
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1     Absent    None    No cable   No     No     No       1         Yes   
		  1/2     Absent    None    No cable   No     No     No       1         Yes

		

Finding a Disconnected Stack Cable: Example

Stack cables connect all stack members. Port 2 on Switch 1 connects to Port 1 on Switch 2.

This is the port status for the members:

Device# show switch stack-ports summary
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback  
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1     OK         2      50 cm     Yes    Yes    Yes        0        No
		  1/2     OK         2      50 cm     Yes    Yes    Yes        0        No
		  2/1     OK         1      50 cm     Yes    Yes    Yes        0        No
		  2/2     OK         1      50 cm     Yes    Yes    Yes        0        No
		
		

If you disconnect the cable from Port 2 on Switch 1, these messages appear:

%STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN
		
		%STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state DOWN
		
		

This is now the port status:


		Device# show switch stack-ports summary
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback  
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1       OK       2      50 cm     Yes    Yes    Yes        1        No  
		  1/2     Absent    None    No cable   No     No     No        2        No   
		  2/1      Down     None    50 cm      No     No     No        2        No   
		  2/2       OK       1      50 cm     Yes    Yes    Yes        1        No   
		
		

Only one end of the cable connects to a stack port, Port 1 on Switch 2.

  • The Stack Port Status value for Port 2 on Switch 1 is Absent, and the value for Port 1 on Switch 2 is Down.

  • The Cable Length value is No cable.

Diagnosing the problem:

  • Verify the cable connection for Port 2 on Switch 1.

  • Port 2 on Switch 1 has a port or cable problem if
    • The In Loopback value is Yes.

    or

    • The Link OK, Link Active, or Sync OK value is No.

Fixing a Bad Connection Between Stack Ports: Example

Stack cables connect all members. Port 2 on Switch 1 connects to Port 1 on Switch 2.

This is the port status:

Device# show switch stack-ports summary
		Device#
 Sw#/Port#  Port    Neighbor  Cable     Link  Link    Sync  #Changes   In
            Status            Length    OK    Active  OK    To LinkOK  Loopback  
		--------  ------  --------  --------  ----  ------  ----  ---------  --------
		  1/1       OK       2      50 cm     Yes    Yes    Yes       1          No   
		  1/2      Down     None    50 cm      No     No     No       2          No   
		  2/1      Down     None    50 cm      No     No     No       2          No   
		  2/2       OK       1      50 cm     Yes    Yes    Yes       1          No
		
		

Diagnosing the problem:

  • The Stack Port Status value is Down.

  • Link OK, Link Active, and Sync OK values are No.

  • The Cable Length value is 50 cm. The switch detects and correctly identifies the cable.

The connection between Port 2 on Switch 1 and Port 1 on Switch 2 is unreliable on at least one of the connector pins.

Additional References for Switch Stacks

Related Documents

Related Topic Document Title

Cabling and powering on a switch stack.

Catalyst 3650 Switch Hardware Installation Guide

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2960cx_3650cx/hardware/installation/guide/b_2960cx-3560cx_hig.html

Error Message Decoder

Description Link

To help you research and resolve system error messages in this release, use the Error Message Decoder tool.

https://www.cisco.com/cgi-bin/Support/Errordecoder/index.cgi

Standards and RFCs

Standard/RFC Title

None

MIBs

MIB MIBs Link

All supported MIBs for this release.

To locate and download MIBs for selected platforms, Cisco IOS releases, and , use Cisco MIB Locator found at the following URL:

http://www.cisco.com/go/mibs

Technical Assistance

Description Link

The Cisco Support website provides extensive online resources, including documentation and tools for troubleshooting and resolving technical issues with Cisco products and technologies.

To receive security and technical information about your products, you can subscribe to various services, such as the Product Alert Tool (accessed from Field Notices), the Cisco Technical Services Newsletter, and Really Simple Syndication (RSS) Feeds.

Access to most tools on the Cisco Support website requires a Cisco.com user ID and password.

http://www.cisco.com/support