Guest

Cisco Catalyst 3750 Series Switches

Troubleshooting Switch Stacks

  • Viewing Options

  • PDF (444.4 KB)
  • Feedback
Troubleshooting Switch Stacks

Table Of Contents

Troubleshooting Switch Stacks

Preventing Traffic Disruptions

Problem: Switch Does Not Join the Stack

Resolving Switch Compatibility Issues

Fixing Incompatible Cisco IOS Versions and Switch-Stack Protocol Versions

Resolving Software License Compatibility Issues

Resolving Incompatible Configurations

Fixing SDM Template Configuration Incompatibilities (Catalyst 3750G-12S Switches)

Resolving StackWise Connection Problems

Fixing Problems with StackWise Interfaces and Cables

Resolving Stack Partitions

Joining a Stack: Typical Sequence States and Rules

Using the Configuration Manager

Using the Stack Manager

Reviewing How the Stack Master Is Elected and Port Numbers Are Assigned

Reviewing How the Stack Master is Rebooted or Disconnected

Understanding Switch Hardware Priority and Stack Manager Version

Problem: Stack Ring Is Not Operating at Full Bandwidth

Resolving Bandwidth Issues (Catalyst 3750 and Catalyst 3750-E Switches)

Using LEDs to Verify StackWise Link Connections

Problem: StackWise Port is Flapping

Resolving Version Mismatches and Copying Files

Upgrading Catalyst 3750 and Catalyst 3750-E Switch Stacks


Troubleshooting Switch Stacks


Preventing Traffic Disruptions

Problem: Switch Does Not Join the Stack

Problem: Stack Ring Is Not Operating at Full Bandwidth

Problem: StackWise Port is Flapping

Resolving Version Mismatches and Copying Files

Upgrading Catalyst 3750 and Catalyst 3750-E Switch Stacks

Preventing Traffic Disruptions

Before you connect a switch to or disconnect a switch from a stack, power it off. Connecting or disconnecting the StackWise cables when switches are powered on can cause switch reloads or temporary traffic loss in the active switches.

Problem: Switch Does Not Join the Stack

Resolving Switch Compatibility Issues

Resolving StackWise Connection Problems

Joining a Stack: Typical Sequence States and Rules

Resolving Switch Compatibility Issues

If a switch does not join a stack, look for incompatibilities between the stack master and member switches in relation to the Cisco IOS version, the license levels, and the configuration settings on the switches.

Fixing Incompatible Cisco IOS Versions and Switch-Stack Protocol Versions

Resolving Software License Compatibility Issues

Resolving Incompatible Configurations

Fixing SDM Template Configuration Incompatibilities (Catalyst 3750G-12S Switches)

Fixing Incompatible Cisco IOS Versions and Switch-Stack Protocol Versions

A new switch might not join a stack when Cisco IOS versions or stack protocol versions differ. Review these points about Cisco IOS versions and stack protocol versions:

Switches with the same Cisco IOS version also have the same stack protocol version and are compatible in the same switch stack.

Switches with the same Cisco IOS version as the stack master can join the stack.

Switches with different Cisco IOS versions can have different stack protocol versions.

Switches with the same Cisco IOS version but different major version numbers are incompatible and cannot function in the same switch stack.

The stack master reports the reason for any incompatibility.

The Cisco IOS software version running on a switch that you plan to add to a switch stack must either be the same as or compatible with on the master switch version. After you connect a new switch to a switch stack, enter the show version user EXEC command to see the version information, the switch types, and the active Cisco IOS version for each switch:

Stack# show version
Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 30    WS-C3750E-24PD     12.2(46)SE      	 C3750E-UNIVERSAL-M
     2 28    WS-C3750E-24PS     12.2(46)SE      	 C3750E-UNIVERSAL-M
     3 54    WS-C3750E-48TD     12.2(46)SE      	 C3750E-UNIVERSAL-M
...
output truncated

If stack members are running incompatible Cisco IOS versions, software version incompatibilities called mismatches occur. If you have a switch that cannot join a switch stack, enter the show switch user EXEC command to see whether the state of the switch is in Version Mismatch (see Switch 3):

Stack# show switch
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 0018.ba60.de00     15     1       Ready
 2       Member 0018.ba60.ce00     14     1       Ready
 3       Member 0016.9d0c.7500     1      2       Version Mismatch

To see which software version switch 3 is running, enter the remote 3 show version command.

If the stack master detects that the software on a new switch does not match, it upgrades (or downgrades) the new switch with the active image that is running on the switch stack by using the automatic upgrade (auto-upgrade) and the automatic advise (auto-advise) features.

Switches with the same major version number but a different minor version number are considered partially compatible. When you connect a partially compatible switch to a stack, the new switch might enter version-mismatch mode and not join the stack as a functional member. The current state in this case is also Version Mismatch, which you can see by entering the show switch user EXEC command:

Stack# show switch
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 0018.ba60.de00     15       1     Ready
 2       Member 0018.ba60.ce00     14       1     Ready
 3       Member 0016.9d0c.7500     1        2     Version Mismatch

See the "Resolving Version Mismatches and Copying Files" section and the "Understanding Switch Hardware Priority and Stack Manager Version" section for more information.

Resolving Software License Compatibility Issues

Mismatched or incompatible software license levels might prevent a Catalyst 3750E switch from joining a stack. These license levels are also called sets, or feature sets:

IP base (ipbase)

IP services (ipservices)

Advanced IP services (advanced ipservices)

Each member switch must have a compatible feature license. Otherwise, you need to upgrade the license level and then reload the switch.

To review the installed license in a member switch, enter these commands. In this example, member switches have the same license level:

Stack-1# show license all switch 1
License Store: Primary License Storage
StoreIndex: 0   Feature: advipservices 
        License Type: Permanent
        License State: Active, In Use
        
Stack-1# show license all switch 2
License Store: Primary License Storage
StoreIndex: 0   Feature: advipservices 
        License Type: Permanent
        License State: Active, In Use

A switch can have up to three installed licenses, but only one is active. Use the show version user EXEC command to see the active licenses. This example is part of the output message for one stack:

Stack-1# show version
Switch 03
---------
License Level                   : advipservices
License Type                    : Permanent
Next reboot licensing Level     : advipservices
Switch 04
---------
License Level                   : advipservices
License Type                    : Permanent
Next reboot licensing Level     : advipservices

Reload the switch after you upgrade the license level. See the appropriate software configuration guide for more information on license administration.


Note Activating a specific feature set on Catalyst 3750 switches requires a specific Cisco IOS version in each switch. These switches do not use licenses to activate features.


Resolving Incompatible Configurations

A switch previously used as a standalone or in a different stack might have different configuration settings. Return the switch to its default values before you connect it to a new stack.

When you add a switch to a stack, the stack master automatically changes the configuration of the new switch. The master switch revises the new member switch port numbers to conform to the current port numbering sequence in the stack. Any existing port-level configuration in the newly-added switch is automatically cleared or updated.

If you add a switch that was a member of one stack to a different stack, the switch port numbers might be inappropriate. For example, instead of the Ethernet ports being numbered with the default values of Fa1/0/1 or G1/0/1, the port numbering might start with Fa2/0/1 or G2/0/1. If the port numbers begin with G2/0/1, for example, enter the switch renumber global configuration command to return switch ports to the default numbering values as shown in this example:

Switch(config)# switch 2 renumber 1

When you reboot the switch, all port-level configurations are removed, and port numbering begins with Fa1/0/1 or G1/0/1.

To prevent incompatible configurations, follow these steps on a previously configured switch that you plan to add to the stack:

1. Delete the config.text file and vlan.dat files from flash memory.

2. Set all options such as the SDM template and port numbers to their default values.

3. Install the same Cisco IOS version that is running in the stack.

4. Power off the switch.

5. Connect the StackWise cables.

6. Power on the switch.

Fixing SDM Template Configuration Incompatibilities (Catalyst 3750G-12S Switches)

The master switch automatically corrects SDM template mismatches in Catalyst 3750 and 3750E switches when you connect one of them to a stack. A configuration mismatch might occur if the switch was a Catalyst 3750G-12S switch configured with a nondefault SDM template before you connect it to a stack.

To display the configured SDM template on a switch, enter the show sdm prefer privileged EXEC command.

To change a Catalyst 3750G-12S switch SDM template to the default setting before you connect it to a stack, enter the sdm prefer default global configuration command.

See the appropriate software configuration guide for additional information on SDM templates.

Resolving StackWise Connection Problems

These sections discuss how to resolve problems with StackWise connections:

Fixing Problems with StackWise Interfaces and Cables

Resolving Stack Partitions

When you connect a switch to a stack, make sure you properly connect both StackWise cables. It is possible for a switch to join a stack when only one StackWise cable is connected to another active stack member, in which case the stack operates at less than maximum ring bandwidth.

An open ring operates at half the available bandwidth. After you connect both StackWise cables of the new switch and it has joined the ring, full ring bandwidth is restored.

Consider these precautions when you connect StackWise cables:

Loose connector retainer screws allow the connector to move and possibly disconnect. Always tighten the retainer screws when you test a StackWise cable. A connector or retainer screw might appear to be fully engaged, but if it is not adequately tightened, port flapping or a no-link-present state might occur.

If you use a screwdriver to tighten the retainer screws, avoid excessive force. Tighten the screws only until the connector is fully engaged. Tighten each of the retainer screws to the same degree.

When you troubleshoot suspected cable connection problems, use the show switch user EXEC command to review the status of switches in the stack. In this example, you can see that switch #3 is in the Provisioned state, which means you need to investigate why. See the descriptions in Table 1.

Table 1 Current State Output and Descriptions

Current State
Description

Ready

The switch is fully operational.

Progressing

The stack master is communicating with the new switch joining the stack.

Provisioned

Ports are assigned in the stack running-configuration, but no switch with the port numbers Fa3/0/1, G3/0/1, and so on, is detected in the stack. If a switch that previously joined a stack is removed, the port numbers remain in the running-configuration, and the missing switch is shown as Provisioned.

The Provisioned state might be caused by a switch that is no longer connected to the stack or be the result of manually preprovisioning the related ports. See the Catalyst 3750 or the Catalyst 3750-E switch software configuration guide for more information on preprovisioning switches in a stack.


Fixing Problems with StackWise Interfaces and Cables

A defective StackWise interface might prevent a switch from joining a stack and can also cause stack port flapping. The best way to test a StackWise interface is by using a known good StackWise cable to connect the two StackWise interfaces. The switch should report a good connection when you enter the show switch detail user EXEC command.

StackWise cable connection problems are usually caused by a loose connection, not a defect in the cable or connector. If two StackWise cables are connected to another stack member, the switch should join the stack even if one of the cables is defective. If you connect only one StackWise cable to a switch and the connection is unreliable, that switch might not join the stack.

The best way to test a StackWise cable that you suspect is defective is to replace it with a known-good cable. If replacing the cable solves the problem, the replaced cable is possibly defective. However, an unreliable connection that was corrected when you replaced the cable might also have been the cause.

If you do not have replacement cable, use these steps to verify whether a cable or a stack member port is faulty:

1. Exchange the cable with another cable in the stack. If you must connect the cable to active, powered-on switches, wait 60 seconds for the stack to reconverge and stabilize after you connect the cable. Use the show switch user EXEC command to identify active stack members and stack member ports.

2. Make sure that you tighten the retainer screws when you test a StackWise cable or port. Loose retainer screws are a common source of trouble. Avoid overtightening the retainer screws, and tighten both of the retainer screws to the same degree.

You can also test a StackWise cable or interface by using only one cable and one switch. Connect the StackWise cable to both StackWise interfaces on the same switch (see Figure 1).

Figure 1 Using One Cable and Switch to Test a StackWise Cable or Interface

Boot or reboot the switch. This message appears:

SM: Detected stack cables at PORT1 PORT2

The message does not conclusively mean that there are no problems with the interfaces or cable. However, the cable and the member ports are probably good.

If a StackWise cable is connected to both StackWise ports on the switch and this message appears, either the cable is defective or StackWise port 2 is defective:

SM: Detected stack cables at PORT1

Enter the show switch detail user EXEC command to review the status of specific StackWise interfaces:

The switches in this example are active stack members, but the ring is incomplete. This example is typical of a missing cable between two switches. After you connect a known-good cable between switch 1 and switch 3, the stack ports become active and show a status of OK. This situation might be a switch hardware problem, but before you replace a switch, verify that the StackWise cables are correctly connected; that is, make sure the retainer screws are equally tightened.

This shows the results of an added StackWise cable between switch 1, port 1 and switch 3, port 2. All StackWise ports are active, and the ring is complete. Full stack bandwidth is available.

The retainer screw shown on the top of the connection shown in Figure 2 is not fully engaged, which can result in the connector working itself loose. The best practice is to finger-tighten both retainer screws to the same degree.

Figure 2 Example of Loose Retainer Screw

Resolving Stack Partitions

An improperly connected ring, a member switch failure, or a faulty StackWise cable connection can cause a stack partition. The stack separates into two independent stacks, each with a stack master. A stack partition usually changes configuration and port numbering. Each stack might also have the same IP address assigned to a switch virtual interface (SVI), which results in a duplicate IP address.

Joining a Stack: Typical Sequence States and Rules

Using the Configuration Manager

Using the Stack Manager

Reviewing How the Stack Master Is Elected and Port Numbers Are Assigned

Reviewing How the Stack Master is Rebooted or Disconnected

Understanding Switch Hardware Priority and Stack Manager Version

This example shows a typical sequence when adding a switch to a stack:

This sequence takes several minutes. Continue to enter the show switch user EXEC command to see the current status.

These are other states that you might see when a switch is joining a stack:

Feature Mismatch. The transition would be Progressing —> Feature Mismatch.

Version Mismatch. The transition would be Progressing —> Version Mismatch.

After an automatic Cisco IOS upgrade corrects a detected Cisco IOS feature mismatch, the upgraded switch reloads before becoming a fully operational stack member.

Using the Configuration Manager

The configuration manager is a Cisco IOS control program that runs on the stack master. The configuration manager performs these functions:

Maintains the running and saved configuration files and distributes them from the stack master to the member switches. The complete stack entity has one saved and one running configuration. All the switches in the stack use the stack-master running configuration.

Keeps a copy of the startup and running configuration on all member switches. Any stack member can then become the new stack master.

Synchronizes the running configuration to all member switches.

When the stack master is powered off or rebooted, the newly elected stack master runs the configuration.

Using the Stack Manager

The Stack Manager is a Cisco IOS program that runs on the stack master switch and performs these tasks:

Discovers other switches in the stack (stack members).

Adds new switches to the stack after discovery.

Removes a disconnected switch from the stack. The output of the show switch user EXEC command shows the switch as provisioned instead of ready.

Elects a stack master during stack boot-up or when you disconnect the working stack master from the stack.

Resolves switch number conflicts when a new switch joins. Assigns a switch number and port numbers to the new switch and reboots it to effect the changes.

Communicates with other switches by using stack messages and gathers and stores information from stack messages.

Manages a stack merge when you connect two independent stacks into a single, larger stack. Renumbers some of the switches and selects one stack master.

Reviewing How the Stack Master Is Elected and Port Numbers Are Assigned

All member switches participate in the election of a stack master if they all boot-up within the election window (20 to 60 seconds). The minimum time depends on which Cisco IOS version is running on the switches. This message appears:

Waiting for Master Election 

Any switches that you connect to the stack after the election process completes are considered stragglers and do not participate in the stack master election. These are the procedural rules for electing a stack master:

Valid switch numbers are 1 through 9. The port numbers of the member switches begin with the switch number, such as Fa9/0/1 or G2/0/1, and so forth, depending on whether the ports are FastEthernet or GigabitEthernet. For example, switch 5 has port numbers beginning with Fa5/0/1 or G5/0/1, depending on the type of switch.

Switch numbering does not reflect the physical location of the stack members.

Switch numbers are sticky, that is, each switch keeps the same switch number after it reboots as a stack member even if it is no longer in a stack. If necessary, you can renumber a switch using the switch current-stack-member-number renumber new-stack-member-number global configuration command. This command renumbers switch 5 to switch 1, and changes port numbers from Fa5/0/1 (or G5/0/1) to Fa1/0/1 or (G1/0/1):

Switch(config)# switch 5 renumber 1

The stack master resolves any switch number conflicts, or the two switches involved in the conflict resolve it if a stack master is not yet elected. The renumbered switch reloads.

Member switches wait for the master to report a complete initialization, then this message appears:

Waiting for Master Ready

To identify the stack master:

Enter the show switch user EXEC command.

Look at the front panel Master (MSTR) LED. It is solid green only for the switch that is the stack master.

An active switch connected to a fully operational stack becomes a stack member. It receives the running configuration from the stack master. When you disconnect a stack member, the switch numbers and port numbers of the remaining stack members do not change, and the switches do not reload.

Reviewing How the Stack Master is Rebooted or Disconnected

A new stack master is elected when you:

Power off, disconnect, or reboot the current stack master.

Power on all switches in a new stack at the same time.

Reboot all switches after you manually change the stack member priorities by using the CLI.

Connect a powered-on stack to another powered-on stack merging two separate stacks merge into one.

These rules determine switch priority:

1. You can manually set the switch priority when stack master election or re-election occurs. You can set a priority of 1 to 15; 15 is the highest priority. This command sets switch 1 to priority 15, and it will be the stack master when you reload the stack if no other switches are also set to priority 15:

Stack(config)# switch 1 priority 15

You can then assign a lower priority to the remaining switches in the stack, or they can remain at default. A priority 15 switch overrides these selection decisions:

2. The switch with the highest hardware and software priority is the stack master if you do not set the priority. If all switches have the same hardware and software priority, this procedure occurs:

3. The switch that is not using the default configuration becomes the stack master. The stack manager renumbers and reboots the switches that are using a default configuration (no config.text file in flash memory).

4. The switch with the lowest MAC address.

Stack master and switch priority conflicts are resolved in this order:

1. The switch that is already master remains unchanged.

2. The switch (or stack) whose master has a higher user-configured priority.

3. The switch (or stack) that is not using the default configuration.

4. The switch (or stack) with the more recent license or image version.

5. The switch (or stack) with the higher hardware priority or hardware version.

6. The stack with the most members (if two stacks are being connected).

7. The switch with the lowest MAC addresses.

Understanding Switch Hardware Priority and Stack Manager Version

Stack hardware priority is a number associated with a specific switch model. Two different switch models can have the same hardware priority. Higher priority numbers increase the probability of a switch being elected the stack master.

The hardware priority is sent in Stack Manager messages and has a value between 15 and 0. (When you manually assign the priority, the range is 15 to 1.) For example, a Catalyst 3750-E switch typically has a default hardware priority of 5. The Catalyst 3750 switches typically have a default hardware priority of 2.

Switch hardware priority is different from the switch hardware version. The hardware version number is associated with the switch model. Different switch models can have same hardware version if they support the same system level features. (A higher version number does not always mean more capability.) This number is not used in stack master election.

This example shows the hardware priority and the hardware version:

Stack2# show switch
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 0018.ba60.de00     15     1       Ready

The stack manager version is also called the image version and has major and minor version numbers as shown in this example:

Main problems related to minor version number mismatches:

The switch cannot join stack.

The switch joins the stack but has a feature mismatch.

To resolve these problems, install the correct Cisco IOS version in the switch with the mismatch.

In some cases, the minor version is different in member switches, which might result in a version mismatch. With a new Cisco IOS version, the minor version might increase.

This example shows the Cisco IOS version running on each switch in a stack:

Stack# show version
Switch Ports Model              SW Version          SW Image
------ ----- -----              ----------          ----------
*    1 30    WS-C3750E-24PD     12.2(46)SE    	 C3750E-UNIVERSAL-M
     2 30    WS-C3750E-24PD     12.2(46)SE    	 C3750E-UNIVERSAL-M
     

See the "Resolving Switch Compatibility Issues" section and the "Resolving Version Mismatches and Copying Files" section for more information.

Problem: Stack Ring Is Not Operating at Full Bandwidth

To operate at full bandwidth, a stack requires a closed ring. If only one StackWise cable is connected to the first and last switches in the stack, or if the ring is incomplete, less than full bandwidth is achieved.

These issues can cause an incomplete ring, in order of probability from high to low:

A bad connection between the StackWise cable connector and a switch chassis connector

A defective StackWise cable

A defective StackWise interface on a switch

A missing StackWise cable in the installation

These issues might cause a bad connection with an otherwise good cable:

The retainer screws on the connector are loose, which might not be obvious. Loose screws might allow the connector to move and possibly disconnect. The disconnection might appear to be either permanent or intermittent.

The retainer screws are not evenly tightened, which might cause a constant or an intermittent disconnection.

See the "Fixing Problems with StackWise Interfaces and Cables" section and the "Resolving StackWise Connection Problems" section for more information.

See these sections:

Resolving Bandwidth Issues (Catalyst 3750 and Catalyst 3750-E Switches)

Using LEDs to Verify StackWise Link Connections

Resolving Bandwidth Issues (Catalyst 3750 and Catalyst 3750-E Switches)

To troubleshoot problems associated with a stack ring not operating at full bandwidth, you need to know what bandwidth you can expect.

This is the maximum theoretical ring speed (bandwidth) for a Catalyst 3750 switch:

32 Gb/s with a full or complete StackWise ring

16 Gb/s with an incomplete or open ring, only one StackWise cable on two switches in the ring

This is the maximum theoretical ring speed (bandwidth) for a Catalyst 3750-E switch:

64 Gb/s with a full or complete StackWise ring of Catalyst 3750-E switches.

32 Gb/s with a full or complete StackWise ring, and a mix of Catalyst 3750 and Catalyst 3750-E switches. A Catalyst 3750 in a Catalyst 3750-E stack results in a maximum ring speed of 32 Gb/s.

16 Gb/s with an incomplete StackWise ring, and a mix of Catalyst 3750 and Catalyst3750-E switches.

To display the active Cisco IOS versions and switch models in a stack, enter the show version user EXEC command:

Stack# show version
Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
*    1 30    WS-C3750E-24PD     12.2(46)SE      		 C3750E-UNIVERSALK9-M
     2 28    WS-C3750G-24PS     12.2(46)SE      	 C3750-IPSERVICESK9-M
     3 54    WS-C3750E-48TD     12.2(46)SE      	 C3750E-UNIVERSALK9-M
...
output truncated

To display information related to the stack member or the switch stack, enter the show switch user EXEC command:

Stack# show switch
Switch/Stack Mac Address : 0018.ba60.de00
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 0018.ba60.de00     15     1       Ready
 2       Member 0016.47f4.e300     14     0       Ready
 3       Member 0018.ba60.c700     13     1       Ready

This example shows a mixed stack of Catalyst 3750-E and 3750G switches. All switches have joined the stack, but not all are running the same Cisco IOS version. All switches in a stack should be running the same Cisco IOS version (or equivalent Cisco IOS version in the case of mixed Catalyst 3750-E and 3750 switch stacks).

Enter the show switch stack-ring speed user EXEC command to see the StackWise bandwidth:

Use the show switch detail user EXEC command to see the status of all StackWise interfaces in the stack. The switches in this example are all active stack members, but the ring is incomplete:

A StackWise cable has been added between switch 1, port 1 and switch 3, port 2. All StackWise ports are connected and active, and the ring is complete. Full stack bandwidth is available. This is an excerpt of the output from the show switch detail user EXEC command:

Using LEDs to Verify StackWise Link Connections

You can test StackWise ring bandwidth by using the switch front-panel LEDs:

1. Press the Mode button on a stack member until the Stack mode LED is on.

2. Verify that the LEDs of the last two ports on the right of the switch front panel are green. Depending on the switch model, the last two ports on the right of the front panel are either 10/100/1000 ports or SFP module ports. If one or both LEDs are not green, the stack is not operating at full bandwidth.

Problem: StackWise Port is Flapping

These conditions might cause StackWise port flapping:

An unreliable or intermittent (bad) connection between a StackWise cable connector and the switch stack port connector

A defective StackWise cable

A defective StackWise interface on a switch

When one or more StackWise ports are flapping, the source of the problem is probably an unreliable connection between a StackWise cable and a StackWise switch port.

If you suspect StackWise port flapping, first verify that the retainer screws are adequately tightened and fully seated against the connector shell. Make sure both retainer screws are tightened the same amount. In some cases you need to disconnect and reconnect the cable while you are troubleshooting, but do not test the connection unless the retainer screws are firmly holding the connector.

An unreliable StackWise connection might produce a similar error message:

#
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state DOWN
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state DOWN
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 1 has changed to state DOWN

-- or --

#
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 4 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 5 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 4 has changed to state DOWN
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 5 has changed to state DOWN
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 4 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 4 has changed to state DOWN
SUPQ-4-CPUHB_RECV_STARVE: Still seeing receive queue stuck after throttling 
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 5 has changed to state UP
STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 5 has changed to state DOWN

An unreliable cable connection caused both of these patterns. Neither the StackWise cable nor the StackWise port was defective. A bad connection between the StackWise cable and the stack connector on the switch caused the problem.

A problem at either end of a StackWise link might cause port flapping at the other end of the link.

See the "Fixing Problems with StackWise Interfaces and Cables" section and "Resolving StackWise Connection Problems" section.

Resolving Version Mismatches and Copying Files

The remote file system runs on the stack master. You can use it to:

Access file systems and copy files from one switch to another in the stack.

Use the archive download-sw privileged EXEC command to simultaneously update the Cisco IOS image in all switches.

Review these points about stack administration and Cisco IOS upgrades:

You can access the CLI from any switch console port in the stack, but all commands are sent to the stack master. This feature is sometimes called a unified console.

The stack master processes the CLI commands for all switches in the stack.

All output to the console from any switch in the stack is redirected to the stack master console.

When version mismatches occur on a member switch, you can review the contents of flash memory in the mismatched switch. This command shows the contents of flash memory for switch 3 in the stack:

Switch# dir flash3:

If you previously upgraded the switch with the archive download-sw privileged EXEC command, the Cisco IOS file is in a subdirectory in flash memory. You can display the contents by using this command:

Switch# dir flash3:/c3750e-universal-mz.122-25.SE

An automatic upgrade can occur in most cases when a Cisco IOS version that is running on the stack master is not the same as the Cisco IOS version on the switch in mismatched mode. The automatic upgrade requires the Cisco IOS tar file of the current image to be available to any of the stack members. The tar file is stored in the flash memory on one of the switches in the stack, preferably the stack master.

If the tar file of the current image is not available, we recommend that you download a compatible image by using the appropriate commands and TFTP.

The automatic upgrade and auto-advise features do not work if the stack master and the switch in mismatched mode run different feature sets (IP Services and IP Base, for example). Starting with Cisco IOS Release 12.2(35)SE, automatic upgrade supports upgrades between cryptographic and noncryptographic images of the same feature set.

If automatic upgrade is not functional, you can manually copy a Cisco IOS file from one switch to another by using this command:

Stack# copy flash1:c3750e-universal-mz.122-35.SE1.bin flash3:

This example copies the Cisco IOS file from flash memory in switch 1 to the flash memory in switch 3. If the bin file is in a subdirectory, this is the appropriate command:

Stack# copy flash1:/c3750e-universal-mz.122-35.SE1/c3750e-universal-mz.122-35.SE1.bin 
flash3:

After you copy the new Cisco IOS file to the switch that is in version mismatch mode, you can delete the mismatched Cisco IOS file. Use this command to delete the unneeded subdirectory from switch 3 flash memory:

Stack# remote command 3 delete /recursive /force flash:/c3750e-universal-mz.122-25.SE

Next, reboot the upgraded switch to load and run the compatible Cisco IOS version:

Stack# reload slot 3

After the switch reloads and loads the compatible Cisco IOS version, it joins the stack.

Upgrading Catalyst 3750 and Catalyst 3750-E Switch Stacks

This section includes some useful options for upgrading switch stacks. See the software configuration guide for routine software upgrade procedures for switches in a stack.

When both a Catalyst 3750-E switch and a Catalyst 3750 switch are in the same mixed stack, you can simultaneously upgrade both by using this command syntax:

stack# archive download-sw tftp://server/3750E.tar tftp://server/3750.tar

This might be a typical command string:

stack# archive download-sw tftp://192.168.1.1/3750E.tar tftp://192.168.1.1/3750.tar

You might have to specify the TFTP path if you do not copy the Cisco IOS file from the default TFTP directory on the TFTP server:

stack# archive download-sw tftp://server/path/3750E.tar tftp://server/path/3750.tar

If you do not need the HTML files, you can use the command option archive download-sw /image to install only the bin file to the flash memory in the Cisco IOS directory of each stack member. By eliminating the HTML files, the Catalyst 3750 switch uses less flash memory, and installation time is faster. Cases might occur where insufficient space is available in the flash memory to extract and install a complete tar file, Use the command option archive download-sw /image to save flash memory by installing only the IOS bin file in flash memory:

stack# archive download-sw /image tftp:// ...

This command might be useful when you upgrade from older Cisco IOS versions in a mixed stack:

stack# archive download-sw /allow-feature-upgrade tftp:// . . . 

This is a typical combination:

stack# archive download-sw /image /allow tftp:// . . . 

You can also copy a Cisco IOS bin file to any specific switch in the stack, and then reload that switch. This example shows upgrading switch #2 in the stack:

Switch-1# copy tftp flash2:
Address or name of remote host [ ]? ip address
Source filename [ ]? /path/filename
Switch# config t
Switch(config)# boot system switch 2 filename
Switch(config)# end
switch# reload slot 2
Proceed with reload? [confirm]

If the bin file is in a directory path in flash memory, use this command:

Switch(config)# boot system switch 2 directory/filename.bin

Always use the archive download-sw command to upgrade Cisco IOS in any switch because using that command automatically sets the correct boot variable. As an alternative when necessary, you can copy a Cisco IOS bin file to any specific switch in the stack and then reload that switch.

When you copy a bin file to flash memory, the boot variable is not set. Upgrading a switch by using the archive download-sw /image command sets the correct filename in the boot variable.

See the appropriate switch software configuration guide for more information about stack upgrades.