Ethernet Spanning Tree Limitations: E Series Card

Introduction

This document clarifies some of the spanning tree rules, and describes how the rules impact VLAN assignment. This document does not intend to be a complete guide of spanning tree and Ethernet circuit provisioning on the ONS 15454. Instead, this document:

• Explains the reasons that cause certain VLAN assignments to fail.

• Provides recommendations that you can use to better design networks. The recommendations enable you to consider the spanning tree limitations when you plan and implement circuits.

• Suggests a workaround in case you encounter the spanning tree constraints when you modify or create circuits.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Cisco ONS 15454

• Spanning Tree Protocol (STP)

Components Used

The information in this document is based on these software and hardware versions:

• Cisco ONS 15454 version 4.6.x and later

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Background Information

The primary function of the Spanning Tree Algorithm (STA) is to cut the loops that redundant links create in bridged networks. When STP detects multiple paths between network hosts, STP blocks the ports until only one path exists.

STA is enabled by default on the optical interfaces of the ONS 15454. You can also configure STA on the front ports of Ethernet cards.

Spanning tree rules on the ONS 15454 do not allow you to create new circuits or modify existing circuits if you do not respect certain VLAN assignment constraints. However, the rules do not prevent some circuit configurations that can lead to improperly designed networks. You must bear those configurations in mind when you design your network.

VLAN Assignment Failures

Problem Description

The spanning tree software on the ONS 15454 runs on the Timing, Communications and Control (TCC), which is a shared resource.

Note: This document uses TCC generically to refer to all variations of the card.

Each node can have a maximum of eight spanning tree instances. In order to minimize the number of spanning tree instances per node, you can map spanning tree instances on a circuit basis instead of VLAN basis. A circuit can map to only one spanning tree instance. You can assign a set of VLANs to a circuit.

The ONS 15454 software also supports these features:

• Automatic generation of spanning tree instances

• Circuits with VLANs that partially overlap

• Facility to collapse the spanning tree

In order to support these features, and also because you map spanning tree instances on a circuit basis, these checks are applicable when you create or modify a circuit:

• The VLAN set of the new or modified circuit must match the VLAN sets of other existing circuits.

• If the VLAN set of the new or modified circuit overlaps with the VLAN set of an existing circuit, both circuits use the same spanning tree instance.

• If the VLAN set of the new or modified circuit overlaps with the VLAN sets of other existing circuits that run the same spanning tree, all the circuits use the same spanning tree instance.

• If the VLAN set of the new or modified circuit overlaps with the VLAN sets of other existing circuits that run different spanning tree instances, the VLAN assignment fails.

Table 1 shows an example of successful VLAN assignments:

Table 1 – Successful VLAN Assignment

Circuit	VLAN Set	Comments	Spanning Tree Instance
C1	10, 20	New Spanning Tree Instance	STP 1
C2	30	New Spanning Tree Instance	STP 2
C3	20, 40	Since 20 matches 20 in C1, same spanning tree instance as C1.	STP 1
C4	30, 50	Since 30 matches 30 in C2, same spanning tree instance as C2.	STP 2
C5	60	New Spanning Tree Instance	STP 3
C6	30, 50, 70	30 and 50 match 30 and 50 in C4, same spanning tree instance as C4	STP 2

Table 2 illustrates a simple case of VLAN assignment failure:

Table 2 – VLAN Assignment Failure

Circuit	VLAN Set	Comments	Spanning Tree Instance
C1	10	New Spanning Tree Instance	STP 1
C2	20	New Spanning Tree Instance	STP 2
C3	10, 20	10 matches 10 in C1, and 20 matches 20 in C2. C1 and C2 belong to different spanning tree instances. Therefore, VLAN assignment fails.	Failure

The VLAN assignment in the second example fails because C3 matches the VLAN sets of C1 and C2 but C1 and C2 run different spanning tree instances.

When the VLAN assignment fails during circuit creation, a "VLAN/Spanning Tree Violation" error appears (see Figure 1).

Figure 1 – VLAN/Spanning Tree Violation

Similarly, when VLAN assignment fails while you try to edit a circuit, an error message appears (see Figure 2).

Figure 2 – Unable to Assign VLAN Set

Recommendation

As a result of the restriction mentioned in the Problem Description section, be very careful about the order in which you add circuits with VLAN sets that overlap. In order to avoid constraints later, Cisco recommends that you plan the VLAN assignment so that you first add the circuits with larger VLAN sets, which have a higher chance of overlap. This way, if you add a circuit with an overlapping VLAN set subsequently, the circuit collapses into the same spanning tree.

Consider the example in Table 2. Cisco recommends that you provision C3 first, and then provision C1 and C2. Alternatively, you can provision the circuits in the order C3-C2-C1, which has the same effect. See Table 3 for details.

Table 3 – Recommended Order to Provision the Circuits

Circuit	VLAN Set	Comments	Spanning Tree Instance
C3	10,20	New Spanning Tree Instance	STP 1
C1	10	10 matches 10 in C3, same spanning tree instance as C3.	STP 1
C2	20	20 matches 20 in C3, same spanning tree instance as C3	STP1

The same logic is applicable when you apply spanning tree to the front ports of Ethernet cards.

Workaround for Circuits Provisioned in Incorrect Order

Use this workaround to avoid the VLAN assignment error when you need to modify circuits that you have not provisioned in the recommended order: assign phantom VLANs to the existing circuits.

Phantom VLANs refer to unused VLANs that do not carry traffic. Addition of phantom VLANs forces the spanning tree to collapse into the same instance. Consider the network design carefully to ensure that you do not incorrectly block any span. Based on the complexity and design of the network, traffic hits are sometimes unavoidable.

A typical example, where two VLANs must collapse into the same spanning tree, is a “dumbbell” scenario. In a dumbbell scenario, you use a linear configuration to join two rings with two VLANs, for example, V10 and V20. In order to avoid loops, before you add a circuit that joins the two rings, ensure that circuits on each node collapse into the same spanning tree.

Figure 3 – The Dumbbell Scenario

For example, assume that the initial VLAN assignment on Node 1 is as shown here:

• C1: V10 STP 1

• C2: V20 STP 2

Here is a possible workaround:

1. Add a phantom VLAN (V99) to C1.

   • C1: V10, V99 STP 1

   • C2: V20 STP2

2. Add a phantom VLAN (V99) to C2.

   • C1: V10, V99 STP 1

   • C2: V20, V99 STP 1

3. Add new circuit C3 with VLANs V10 and V20.

   1. C1: V10, V99 STP 1

   2. C2: V20, V99 STP 1

   3. C3: V10, V20, V99 STP1

4. Remove the phantom VLAN from C1 and C2.

   1. C1: V10 STP 1

   2. C2: V20 STP 1

   3. C3: V10, V20 STP1

   Figure 3 represents the final VLAN topology.

Invalid Circuit Configurations

Successful circuit creation or modification means that the VLAN assignment passes the per-circuit-spanning-tree mapping rule, but does not guarantee that the circuit configuration is valid. Even though you collapse a spanning tree, you cannot cure an improperly designed network. Here are some scenarios that explain this point.

Scenario 1

This first scenario consists of two nodes, Node 1 and Node 2, with two circuits C1 and C2. Circuit C1 carries VLANs V10 and V20, and circuit C2 carries VLAN V20 (see Figure 4). A loop is present in the domain of V20, but the domain of V10 has no loop. However, one of the spans is blocked because the circuits collapse into one spanning tree. Here are the factors that determine which one of the spans is blocked:

• MAC addresses of the back-end ports

• Circuit size

• Order of creation of the circuits

If circuit C1 happens to be blocked, V10 traffic does not flow. Therefore, this network design is not valid under spanning tree limitations.

Figure 4 – Invalid Configuration: Scenario 1

Scenario 2

The second scenario consists of two nodes, Node 1 and Node 2, and three circuits C1, C2 and C3. Here, you create the circuits in the correct order (see Table 2), so that circuit provisioning succeeds, and all circuits are in the same spanning tree. Circuit C1 carries VLANs V10 and V20, C2 carries VLAN V10, and C3 carries VLAN V20 (see Figure 5).

Assume that the spanning tree parameters are just right, which can happen in some situations, for example, when C1 is wider than the other circuits. C2 and C3 are blocked, and all traffic flows between Node 1 and Node 2. If you subsequently remove C1, circuits C2 and C3 continue to run the same spanning tree. After the deletion of C1, either VLAN V10 or VLAN V20 is blocked. Again, this network design is not valid under spanning tree limitations.

Figure 5 – Invalid Configuration: Scenario 2

Scenario 3

This example consists of a four-node system with two circuits. Circuit C1 carries VLANs V10 and V20 while C2 carries VLAN V10, V20 and V30. Both circuits run the same spanning tree instance, because the VLAN sets of both circuits overlap. The V10 and V20 domains contain a loop. Therefore, one of the spans is blocked. If the blocked span is C1, all VLANs flow. This configuration appears fine, but the problem is that no protection is available for V30; if the C2 span fails, V10 and V20 flow over C1, but there is no path for V30.

Figure 6 – Invalid Configuration: Scenario 3

Point-to-Point Unstitched Circuits

When you collapse the spanning tree, you encounter issues with point-to-point circuits that span the same set of nodes but on different “Unstitched” cards. In the “Unstitched” mode, which is also known as “Single-card EtherSwitch”, each card remains a single switching entity within the ONS 15454. However, if two circuits that span different “Unstitched” cards use the same VLAN ID, the circuits still collapse into the same spanning tree instance, and one of them is blocked. Figure 7 illustrates this problem.

Figure 7 – Example for Point-to-Point Unstitched Circuits

In this example, C2 is blocked, and so, no traffic flows between Router 3 and Router 4. In order to overcome this issue, Cisco introduced the per-circuit turn-off feature (also known as “VLAN reuse”) in ONS 15454 version 3.3 and later. This feature allows you to disable or enable STP on a single circuit basis. When you disable STP, multiple point-to-point circuits that use different “Unstitched” cards can use the same VLAN ID without being blocked.

In order to disable the Spanning Tree, ensure that you do not check the Enable Spanning Tree check box in the Circuit Creation screen (see the red rectangle in Figure 8).

Figure 8 – Circuit Creation: Disable Spanning Tree

Spanning Tree Assignment Display

Complete these steps in order to display the spanning tree assignments through CTC:

1. Log into the Cisco Transport Controller (CTC).

   Figure 9 – Spanning Tree Assignment

   

2. Click Maintenance (see arrow A in Figure 9).

3. Click Ether Bridge (see arrow B in Figure 9).

4. Click Circuits (see arrow C in Figure 9).

   The display includes the Type, Circuit Name/Port, STP ID and VLANs.

Related Information

• Technical Support & Documentation - Cisco Systems