Guest

Support

Configuration Synchronization Operations

  • Viewing Options

  • PDF (575.6 KB)
  • Feedback
Configuration Synchronization Operations

Table Of Contents

Configuration Synchronization Operations

Overview

Benefits of Configuration Synchronization

Requirements

Guidelines and Limitations

vPC Configurations

Cisco Fabric Services Over IP

Switch Profiles

Switch Profile Commands

User-Based Access Controls

Verification Checks

Mutual Exclusion Check

Merge Check

Commit

Buffering

Import

Configuration Synchronization Best Practices

Configuration Examples

Configuring a vPC Topology Using Configuration Synchronization

Active/Active FEX Topology Examples

Dual-Homed FEX Topology (Active/Active FEX Topology)

New Deployments in an Active/Active FEX Topology

Existing Deployment with an Active/Active FEX Topology

Straight-Through Topology Examples

Switch vPC Topology and Straight-Through FEX Topologies (Host vPC)

New Deployment in a vPC Topology and Straight-Through FEX Topology

Existing Deployments in a vPC Topology and Straight-Through FEX Topology

Reloading a Cisco Nexus 5000 Series Switch

Synchronizing the Peer Switches After a Switch Reload

vPC Peer-Link Failures

mgmt0 Interface Connectivity is Lost

Rollback Failures with Conditional Features

Channel Group Failures

At-A-Glance Configuration Modes

Terminology


Configuration Synchronization Operations


This chapter provides information about configuration synchronization operations in Virtual Port Channel (vPC) topologies.

This chapter includes the following sections:

Overview

Configuration Synchronization Best Practices

Configuration Examples

At-A-Glance Configuration Modes

Terminology

Overview

Some Cisco NX-OS software features require consistent configurations across Cisco Nexus 5000 Series switches in the network. For example, vPC topologies require identical configurations on peer switches. As a result, you, as the network administrator, must repeat configurations on both peer switches. This process, which can cause errors due to misconfigurations or omissions, can result in additional service disruptions because of mismatched configurations. Configuration synchronization eliminates these problems by allowing you to configure one switch and automatically synchronize the configuration on the peer switch.

In a vPC topology, an EtherChannel can be formed across two physical switches and vPCs can be connected to any networking device or end host. Because each Cisco Nexus 5000 Series switch forms an EtherChannel bundle to a downstream device, each Cisco Nexus 5000 Series switch must have some matching parameters. You can use a vPC consistency check to verify that both Cisco Nexus 5000 Series switches have the same configuration (Type 1 or Type 2). If they do not match, depending on whether it is a global (for example, spanning-tree port mode), a port-level (for example, speed, duplex, or channel-group type), or even a port-channel interface, the vPC can go into a suspended state or a VLAN can go into a blocking state on both peer switches. As a result, you must ensure that the configuration from one switch is copied identically to the peer switch.

Configuration synchronization allows you to synchronize the configuration between a pair of switches in a network. You use a switch profile to create a configuration file that you can apply locally and you use it to synchronized the configuration to its peer. Configuration synchronization and vPCs are two independent features and configuration synchronization does not eliminate vPC consistency checks. The checks will continue. If there is a configuration mismatch, the vPC can still go into a suspended state. One important benefit of configuration synchronization is that it eliminates the need to manually repeat the same configuration on both switches.

This section includes the following topics:

Benefits of Configuration Synchronization

Requirements

Guidelines and Limitations

Cisco Fabric Services Over IP

Switch Profiles

User-Based Access Controls

Verification Checks

Commit

Buffering

Import

Benefits of Configuration Synchronization

Configuration synchronization benefits are as follows:

Provides a mechanism to synchronize configuration from one switch to another switch.

Merges configurations when connectivity is established between peers.

Allows you to choose which configuration is synchronized.

Provides mutual exclusion for commands.

Provides verify and commit Cisco NX-OS commands.

Supports existing session and port profile functionality.

Provides an import command to migrate existing vPC configurations to a switch profile.

Supports Gigabit Expansion Module (GEM) and Fabric Extender (FEX) pre-provisioning.

Requirements

The requirements for configuration synchronization are as follows:

Cisco NX-OS Release 5.0(2)N1(1) or a later release.

Cisco Fabric Services over IP (CFSoIP) enabled on each peer, including an identical CFSoIP multicast address.

Identical switch profiles on each switch.

Configured peer IP addresses.

Guidelines and Limitations

The guidelines for configuration synchronization are as follows:

You must configure the following interfaces in a switch profile:

Port-channel interfaces

Ports that are not channel-group members

You must configure all port-channel members outside the switch profile in configuration terminal mode.

You must follow configurations in a specified order.

Depending on the type of vPC topology (active/active or straight-through) and the type of configuration that is needed (port channel, nonport channel, FEX, QoS, and so on), you must use the switch profile mode or the configuration terminal mode. See the "At-A-Glance Configuration Modes" section to identify what mode is used for different types of configurations.

Configuration synchronization has the following configuration limitations:

FCoE in vPC Topologies—FCoE configurations are not supported in switch profiles because configurations are typically different on peer switches. If you enable FCoE on a vPC peer switch, you must not configure the port channel in the switch profile.

Feature Commands—The feature feature name commands that enable a conditional feature are not supported in switch profiles. You should independently configure these commands on each peer switch in configuration terminal mode.

Configuration Rollback and Conditional Features—With configuration synchronization, when a conditional feature is present in a checkpoint and not in the running configuration, a configuration rollback to that checkpoint fails. The workaround is to reconfigure the conditional feature ("feature xyz") before the configuration rollback is executed. This workaround also applies to the vpc domain command and the peer-keepalive command in vpc-domain mode.

vPC Configurations

Configuration synchronization requires two Cisco Nexus 5000 Series peer switches that are configured in a vPC topology. Figure 4-1 shows a vPC topology configured on two Cisco Nexus 5000 Series switches (N5k-1 and N5k-2).

Figure 4-1 vPC Topology

To configure vPC on two Cisco Nexus 5000 Series switches, follow these steps:


Step 1 Create a vPC domain and configure a vPC keepalive link.

You must create identical vPC domain IDs on both vPC peer switches.

The domain ID is used to automatically form the vPC system MAC address.

N5k-1(config)# vpc domain 10
N5k-2(config)# vpc domain 10
 
   

Step 2 Configure a vPC peer-keepalive link.

You can configure the destination IP for the peer-keepalive link that carries the keepalive messages. Optionally, you can configure other parameters for the keepalive messages.

N5k-1(config)# vpc domain 10
N5k-1(config-vpc-domain)# peer-keepalive destination 10.29.170.8 vrf management
 
   
N5k-2(config)# vpc domain 10
N5k-2(config-vpc-domain)# peer-keepalive destination 10.29.170.7 vrf management
 
   

Step 3 Create and configure a vPC peer link.

You can create a peer link by designating an EtherChannel on each switch as the peer link for the specified vPC domain. We recommend that you configure the EtherChannels that you are designating as the vPC peer link in trunk mode and that you use two ports on separate modules on each vPC peer switch for redundancy.

 
   
N5k-1(config)# interface port-channel 10
N5k-1(config-if)# vpc peer-link
 
   
N5k-1(config)# interface ethernet 1/17-18
N5k-1(config-if-range)# switchport mode trunk
N5k-1(config-if-range)# channel-group 10
 
   
N5k-2(config)# interface port-channel 10
N5k-2(config-if)# vpc peer-link
 
   
N5k-2(config)# interface ethernet 1/17-18
N5k-2(config-if-range)# switchport mode trunk
N5k-2(config-if-range)# channel-group 10
 
   


Note You can configure a vPC peer link between the two Cisco Nexus 5000 Series switches either manually on both switches or with configuration synchronization from any one of the two peer switches. For information about the configuration synchronization method to configure the vPC peer link, see the "Configuring a vPC Topology Using Configuration Synchronization" section. For additional information on the vPC feature, see Chapter 2 "Virtual Port Channel Operations" and the Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide, Release 5.0(2)N2(1) at the following URL: http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/layer2/502_n2_1/Cisco_n5k_layer2_config_gd_rel_502_N2_1_chapter8.html


Cisco Fabric Services Over IP

The Cisco Fabric Services over IP (CFSoIP) protocol transports configuration synchronization over the mgmt0 interface (mgmt virtual routing and forwarding [VRF]). You must ensure the connectivity to the mgmt0 interface. CFSoIP and Cisco Fabric Services (CFS) are different protocols. CFS runs across the peer link for a vPC. Although both protocols are based on the CFS protocol, they exchange different control packets.

To use the CFSoIP protocol for configuration synchronization, follow these steps:


Step 1 Configure the CFSoIP multicast address on each peer switch:

N5k-1# config terminal
N5k-1(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   
N5k-2# config terminal
N5k-2(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   

Step 2 Enable CFSoIP manually on each peer switch:

N5k-1(config)# cfs ipv4 distribute
 
   
N5k-2(config)# cfs ipv4 distribute
 
   

Note CFSoIP is not supported on the switch virtual interface (SVI)/default VRF.


Step 3 Establish the peer connection over the mgmt0 transport interface:

N5k-1(config)# interface mgmt 0
N5k-1(config-if)# ip address 10.29.170.7/24
N5k-1(config-if)# vrf context management
N5k-1(config-vrf)# ip route 0.0.0.0/0 10.29.170.1
 
   
N5k-2(config)# interface mgmt 0
N5k-2(config-if)# ip address 10.29.170.8/24
N5k-2(config-if)# vrf context management
N5k-2(config-vrf)# ip route 0.0.0.0/0 10.29.170.1
 
   

Switch Profiles

Beginning with Cisco NX-OS Release 5.0(2)N1(1), config-sync mode allows you to create a switch profile. A switch profile contains a predefined configuration that you can use to configure a peer switch so that both peers have the same configuration. In config-sync mode, you define the peer and the configuration in the switch profile. Peers are identified by their IP address and they are local to each switch profile. Commands entered in config-sync mode are buffered until they are committed. Configuration changes made in configuration terminal mode apply only to the local switch.

You must create an identical switch profile on each peer switch in config-sync mode. This configuration is not automatically synchronized and you must configure it on each peer switch.

To create the switch profiles, enter the following commands:

N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
N5k-1(config-sync)# sync-peers destination 10.29.170.8
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync)# sync-peers destination 10.29.170.7
 
   

Note The switch profile name must be identical on both peers. You can create only one switch profile on each peer switch.


In Cisco NX-OS Release 5.0(2)N1(1), switch profiles do not support all commands. The config-sync mode commands are limited to vPC configurations.

Switch Profile Commands

The following configuration commands are supported in a switch profile:

VLAN

ACL

Spanning-Tree Protocol (STP)

Quality of Service (QoS)

Interface configurations (Ethernet, port channel, or vPC interfaces)

The following commands are not supported in a switch profile:

Enable feature sets (you must manually enable a specific feature before relevant configurations can be added)

vPC domain

vPC peer-keepalive

FCoE

User-Based Access Controls

In Cisco NX-OS Release 5.0(2)N1(1), only the user who creates a switch profile can edit the switch profile even if another user has the same admin privileges. Beginning with Cisco NX-OS Release 5.0(2)N2(1), you can add, delete, or modify a switch profile configuration based on Role Based Access Control (RBAC) configurations. Users that have the appropriate privilege level to access the switch profile can successfully modify the switch profile and commit the configuration.

For more information on RBAC, see the Cisco Nexus 5000 Series System Management Configuration Guide.

As a network administrator, you can restrict a user from accessing a switch profile. When a restricted user has permission to access a switch profile, that user can successfully commit the switch profile on the initiating switch. However, issuing a particular command (for example, the switchport mode access command), fails or succeeds in a switch profile according to the RBAC policies and rules assigned to that user.

In addition, the same username and privilege level must exist for a successful commit on the peer switch. If the same username and privilege level does not exist on the peer switch, the commit fails. You must ensure that configuration synchronization peers have the same configured users and roles. Occasionally, the same username can exist but roles might be mismatched. Also, the same user on one peer switch could have a more restricted role on the other peer switch and in that case the commit might fail. You must configure usernames with matching roles on peer switches to avoid these problems. As a best practice, the user with the network administrator role should create the switch profile to reduce the risk of configurations failing to commit due to permission issues.

In vPC topologies, when one peer switch is running Cisco NX-OS Release 5.0(2)N1(1) and a second peer switch is running Cisco NX-OS Release 5.0(2)N2(1), a successful commit depends on which switch was used to issue the commit. On a switch running Cisco NX-OS Release Cisco NX-OS Release 5.0(2)N1(1), only the user who created the switch profile can issue the commit. On a switch running Cisco NX-OS Release 5.0(2)N2(1), users with appropriate privileges can issue the commit.

Verification Checks

To reduce the possibility of overriding switch profile configurations or configurations that are not part of a switch profile, two types of validation checks are performed:

Mutual Exclusion Check

Merge Check

Mutual Exclusion Check

The Mutual exclusion check identifies potential conflicts between a switch profile configuration and the global configurations (configurations that are not part of a switch profile). A command that is included in a switch profile cannot be configured outside of the switch profile. The same rules apply on the peer switch.

The mutual exclusion check is done locally and on the peer switch. When entering the verify or commit command, if the peer switch is reachable using the mgmt0 interface, the check is done on both the local switch and the peer switch. If the peer switch is not reachable, the check is only done on the local switch.

If the mutual exclusion check fails, you must manually correct the configuration and enter the commit command again.

The following commands are exceptions and they can exist inside and outside the switch profile without receiving a mutual exchange error:

Interface configuration commands except port-channel interfaces

Shutdown/no shut commands

System Quality of Service (QoS) (system qos command) command

For implementations including port channels, consider the following guidelines to minimize mutual exchange errors:

Port channels created in switch profile mode should not be configured using global configuration (config terminal) mode.

If a port-channel is created in global configuration mode, channel groups including member interfaces must also be created using global configuration mode.

Port-channels that are configured within switch profile mode may have members both inside and outside of a switch profile.

If you want to import a member interface to a switch profile, the port-channel that corresponds with the member interface must also be present within the switch profile.

For more information on configuring port channels, see the Cisco NX-OS Layer 2 Switching Configuration Guide. For more information on configuring switch profiles, see the Cisco NX-OS 5000 System Management Configuration Guide.

Merge Check

A Merge check is done on the peer switch to ensure that the received configuration does not conflict with the switch profile configuration that already exists on the receiving switch. If a merge check failure occurs, you must manually correct the configuration and enter the commit command again.

Commit

Use the commit command to synchronize the configuration with the peer switch and to apply the configuration locally. Configurations are stored in the buffer until the commit command is issued. A commit can be executed by a vPC primary switch or a secondary switch. The initiator is the switch on which the commit command was issued. You can enter the commit command only on one switch at any given time. If a commit is attempted while another commit is in progress, it fails and the following syslog message appears:

Failed: Session Database already locked, Verify/Commit in Progress
 
   

As a best practice, you should assign one switch as the initiator, make all configurations on that switch, and then synchronize the configuration on the peer switch to simplify the process and reduce any possible confusion.

All configuration changes (including configuration terminal mode changes for all supported commands) are prevented when a switch profile session is in progress.


Note If the peer switch is reachable and you enter the commit command, the configuration is applied locally and to its peer switch. If the commit is unsuccessful, the configuration is not applied on the local or remote switch (atomic behavior).


Commands are executed in the same order in which they are buffered. If there is an order dependency for certain commands (for example, QoS policy commands), the commands must be defined before they are applied. The order of commands can be edited in the buffer. If you are including commands that are part of a feature that requires the feature to be enabled, you must ensure that the feature is enabled and defined manually on each switch.


Note The feature command is not synchronized between peers (for example, feature vpc or feature lacp.


When you enter the commit command, the CLI prompt may not return right away. The length of time it takes to apply the configuration may be longer if the size of the configuration is very large. This operation is normal and we recommend that you do not abort the commit (by pressing Ctrl-c or Ctrl-z) because it might leave the configuration in an inconsistent state.

Buffering

The switch profile configuration is stored in a buffer until the commit command is entered. You can add, delete, or move configurations in the buffer. Once the configuration has been pushed using the commit command, it is applied to the system configuration. Use the show running command to verify that the configuration has been applied. You can also use the show running switch-profile command to specifically check what configuration was synchronized using the switch profile.

Consider the following guidelines for the configuration that is stored in the buffer:

Configurations are buffered until a successful commit.

You can add, delete, or move configurations in the buffer.

Commands are executed in the same order in which they are buffered. You can change the order inside the buffer by using the buffer-delete command or the buffer-move command.

Import

When you upgrade to Cisco NX-OS Release 5.0(2)N1(1), you have the option to enter the import command to copy supported running configuration commands to a switch profile. The switch -profile import command allows you to import the entire running configuration or you can choose specific interfaces to merge. Changes are not supported during the import process. If you add new commands in addition to the import configurations, the commit might fail. The commands remain in the buffer. You have the option to correct the buffer and enter the commit command again or abort the import mode. If you abort the import, the commands in the buffer are lost.

Configuration Synchronization Best Practices

Configuration synchronization is primarily used in vPC topologies. You should follow the best practice guidelines in this section to ensure a successful configuration synchronization.


Caution You must follow the best practices for configuration synchronization described in this chapter for Cisco NX-OS Release 5.0(2)N1(1). Failure to do so might leave configurations in an inconsistent state.

In addition to configuration synchronization, Cisco NX-OS Release 5.0(2)N1(1) introduced three additional features:

Pre provisioning—Allows you to configure offline GEM and FEX interfaces.

Port profiles—Allows you to define consistent interface configurations that are applied to multiple ports.

Port profiles apply to ports and switch profiles apply to switch configurations; they are not the same.

Port profiles are not required for a configuration synchronization but they can be included in a configuration synchronization.

Configuration Rollback—Allows you to create checkpoints of the running configuration and then perform a rollback to those checkpoints.

Use pre-provisioning, port profiles, and configuration rollbacks to enhance configuration synchronization and provide maximum benefits in a vPC topology. These features are included in the examples found in this chapter. See the Cisco Nexus 5000 Series Configuration Guides for additional information on these features.


Note These features are independent of configuration synchronization; you do not need to enable them to use configuration synchronization.


Configuration Examples

This section describes the following configuration examples:

Configuring a vPC Topology Using Configuration Synchronization

Active/Active FEX Topology Examples

Straight-Through Topology Examples

Reloading a Cisco Nexus 5000 Series Switch

vPC Peer-Link Failures

mgmt0 Interface Connectivity is Lost

Rollback Failures with Conditional Features


Note The following examples are current as of Cisco NX-OS Release 5.0(2)N1(1).


Configuring a vPC Topology Using Configuration Synchronization

In Figure 4-2, N5k-1 and N5k-2 are part of vPC Domain 10. The peer keepalive is configured over the mgmt0 interface and Ethernet 1/17-18 are bundled into P010 to form the peer link. Configuration synchronization maintains a consistent configuration on the peer switches and simplifies the switch administration in a vPC topology.

Figure 4-2 vPC Topology

Example 4-1 shows the sample running configuration required for the vPC to become operational.

Example 4-1 Running Configuration of Peer Switches in a vPC Topology

vPC Configuration for N5k-1
vPC Configuration for N5k-2
vlan 1-10
 
        
feature vpc
 
        
vpc domain 10
	peer-keepalive destination 10.29.170.8
	peer-config-check-bypass
 
        
interface port-channel10
	switchport mode trunk
	vpc peer-link
	spanning-tree port type network
 
        
interface Ethernet1/17
	switchport mode trunk
	channel-group 10
 
        
interface Ethernet1/18
	switchport mode trunk
	channel-group 10
vlan 1-10
 
        
feature vpc
 
        
vpc domain 10
	peer-keepalive destination 10.29.170.7
	peer-config-check-bypass
 
        
interface port-channel10
	switchport mode trunk
	vpc peer-link
	spanning-tree port type network
 
        
interface Ethernet1/17
	switchport mode trunk
	channel-group 10
 
        
interface Ethernet1/18
	switchport mode trunk
	channel-group 10


Note Peer-config-check-bypass is a best practice configuration for vPCs. For more information, see the Cisco Nexus 5000 Series Design Guide at the following URL: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572829-01_Design_N5K_N2K_vPC_DG.pdf


To configure the vPC topology shown in Figure 4-2, follow these steps:


Step 1 Enable the vPC feature on the peer switches.

N5k-1# config terminal
N5k-1(config-vpc-domain)# feature vpc
 
   
N5k-2# config terminal
N5k-2(config-vpc-domain)# feature vpc
 
   

Step 2 Configure the peer keepalive on both switches using the mgmt0 interface.

N5k-1(config)# vpc domain 10
N5k-1(config-vpc-domain)# peer-keepalive destination 10.29.170.8
 
   
N5k-2(config)# vpc domain 10
N5k-2(config-vpc-domain)# peer-keepalive destination 10.29.170.7
 
   

Step 3 Configure the CFSoIP multicast address on each peer switch:

N5k-1(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   
N5k-2(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   

Step 4 Enable CFSoIP on both switches.

N5k-1(config)# cfs ipv4 distribute
 
   
N5k-2(config)# cfs ipv4 distribute
 
   

Step 5 Configure the switch profile with the same name on both switches.

N5k-1(config)# config sync
N5k-1(config-sync)# switch-profile Test
 
   
N5k-2(config)# config sync
N5k-2(config-sync)# switch-profile Test
 
   

Step 6 Enter the sync peer destination command to configure both switches.

N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
N5k-1(config-sync-sp)# sync-peers destination 10.29.170.8
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync-sp)# sync-peers destination 10.29.170.7
 
   

Step 7 In switch-profile mode, create the port-channel interface for the peer link.

N5k-1(config-if)# config sync
N5k-1(config-sync)# switch-profile Test
N5k-1(config-sync-sp)# int po10
N5k-1(config-sync-sp-if)# exit
N5k-1(config-sync-sp)# commit
 
   

Step 8 In interface mode, associate the port-channel member to PO 10.

N5k-1(config)# int ether 1/17-18 
N5k-1(config-if)# channel-group 10
 
   
N5k-2(config)# int ether 1/17-18 
N5k-2(config-if)# channel-group 10
 
   

Step 9 In switch profile mode, add the appropriate configurations under the port-channel interface to form the peer link.

N5k-1(config-if)# config sync
N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# interface po10
N5k-1(config-sync-sp-if)# switchport mode trunk
N5k-1(config-sync-sp-if)# vpc peer-link 
N5k-1(config-sync-sp-if)# commit
 
   
Verification successful...
Proceeding to apply configuration. This might take a while depending on
amount of configuration in buffer.
Please avoid other configuration changes during this time.
Commit Successful
 
   

Active/Active FEX Topology Examples

This section includes the following examples:

Dual-Homed FEX Topology (Active/Active FEX Topology)

New Deployments in an Active/Active FEX Topology

Existing Deployment with an Active/Active FEX Topology

Dual-Homed FEX Topology (Active/Active FEX Topology)

Figure 4-3 shows that each FEX is dual-homed with two Cisco Nexus 5000 Series switches. The FEX-fabric interfaces for each FEX are configured as a vPC on both peer switches. The host interfaces on the FEX appear on both peer switches. If these host interfaces are bundled in a port channel, you must configure the port channel identically on both peer switches. Configuration synchronization helps keep the FEX configuration synchronized between the pair of vPC peer switches.

Figure 4-3 Dual-Homed FEX Active/Active Topology

In Figure 4-3, the vPC is already operational. FEX 100 is dual-homed to both parent switches: N5k-1 and N5k-2 on FEX-fabric interfaces Ethernet 1/1. Because the FEX is pre-provisioned, there is no existing running configuration on Ethernet 1/1.


Note A port channel within the same FEX is supported on Cisco Nexus 2200 Series Fabric Extenders.


FEX 100 is configured to have two types of host interfaces. One interface is Ethernet100/1/1, which is singly attached to a server (nonport-channel member), and the other interface is Ethernet 100/1/2-3, which is configured in a port channel to the server (port-channel member).

Example 4-2 shows the sample running configuration for the peer switches. Two types of configurations are shown:

Basic Configuration

Port profile configuration.

You can use either option or you can use both configurations together.


Note You can use port profiles to reduce operational overhead although they are not required.


Example 4-2 Running Configuration of a FEX in an Active/Active Topology for the Peer Switches

Basic Configuration—No Port Profile
Port Profile Configuration
vlan 1-10
 
        
interface port-channel100
	switchport mode fex fabric
	vpc 100
	fex associate 100
 
        
interface port-channel 200
	switchport mode trunk
	switchport trunk allowed vlan 1-5
 
        
interface Ethernet1/1
	fex associate 100
	switchport mode fex fabric
	channel-group 100
 
        
interface Ethernet100/1/1
	switchport mode trunk
	switchport trunk allowed vlan 1-10
 
        
interface Ethernet100/1/2
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 200
 
        
interface Ethernet100/1/3
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 200
 
        
vlan 1-10
 
        
port-profile type ethernet eth-profile
	switchport mode trunk
	state enabled
 
        
port-profile type port-channel pc-profile
	switchport mode trunk
	state enabled
 
        
interface port-channel100
	switchport mode fex fabric
	vpc 100
	fex associate 100
 
        
interface port-channel 200
	inherit port-profile pc-profile
	switchport trunk allowed vlan 1-5
 
        
interface Ethernet1/1
	fex associate 100
	switchport mode fex fabric
	channel-group 100
 
        
interface Ethernet100/1/1
	inherit port-profile eth-profile
	switchport trunk allowed vlan 1-10
 
        
interface Ethernet100/1/2
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 200
 
        
interface Ethernet100/1/3
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 200

New Deployments in an Active/Active FEX Topology

In a new deployment, configuration synchronization is introduced from the beginning to synchronize the configuration across peer switches. As a result, there is no existing running configuration on the FEX ports.

To configure the dual-homed FEX active/active topology shown in Figure 4-3, follow these steps:


Step 1 Configure the CFSoIP multicast address on each peer switch:

N5k-1# config terminal
N5k-1(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   
N5k-2# config terminal
N5k-2(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   

Step 2 Enable CFSoIP on both switches.

N5k-1(config)# cfs ipv4 distribute
 
   
N5k-2(config)# cfs ipv4 distribute
 
   

Step 3 Create a switch profile on both switches.

N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
N5k-1(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-2>
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-1>
 
   

Step 4 Pre-provision the FEX.


Note In a FEX active/active topology, always pre-provision the FEXs that are dual-homed inside the switch profile. This process helps configuration synchronization when the FEX is not connected to a Cisco Nexus 5000 Series switch.


N5k-1(config-sync-sp)# slot 100
N5k-1(config-sync-sp-slot)# provision model N2k-C2232P
N5k-1(config-sync-sp-slot)# exit
 
   
N5k-1(config-sync-sp-if)# sh switch-profile buffer
 
   
switch-profile  : Test
----------------------------------------------------------
Seq-no  Command
----------------------------------------------------------
1       slot 100
1.1       provision model N2K-C2232P
 
   

Step 5 Add referred global configuration to the switch profile.


Note Because interface configurations will be synchronized, all policies that are applied on the interface must be synchronized (for example, port profiles, QoS, and ACL policies).


N5k-1(config-sync-sp)# port-profile type ethernet eth-profile
N5k-1(config-sync-port-prof)# switchport mode trunk
N5k-1(config-sync-port-prof)# state enabled
 
   
N5k-1(config-sync-sp)# port-profile type port-channel pc-profile
N5k-1(config-sync-port-prof)# switchport mode trunk
N5k-1(config-sync-port-prof)# state enabled
 
   

Step 6 Configure the Ethernet interfaces (the non-port-channel members) inside the switch profile.

N5k-1(config-sync-sp)# interface Ethernet100/1/1
N5k-1(config-sync-sp-if)# inherit port-profile eth-profile
N5k-1(config-sync-sp-if)# switchport trunk allowed vlan 1-10
 
   

Step 7 Create the port-channel interface inside the switch profile.


Note You must configure port-channel interfaces in the switch profile, not in configuration terminal mode.


This example shows that port channel 100 (vPC 100) is the EtherChannel from N5k to N2k:

N5k-1(config-sync-sp)# interface Port-channel100
 
   

This example shows that port channel 200 is the EtherChannel from N2k to the end device:

N5k-1(config-sync-sp)# interface Port-channel200
 
   

Step 8 Commit the configuration inside the switch profile.

N5k-1(config-sync-sp)# commit
 
   

Step 9 Add members to the port channel in configuration terminal mode on both switches.


Note The configuration must be done on both switches in configuration terminal mode.


This example shows that N5k-1- Ethernet1/1 is a FEX-fabric member of port channel 100:

N5k-1(config)# int ether1/1
N5k-1(config-if)# channel-group 100 force
 
   

This example shows that N5k-1- Ethernet1/100/2-3 are members of port channel 200:

N5k-1(config)# interface Ethernet100/1/2-3
N5k-1(config-if-range)# channel-group 200 force
 
   

This example shows that N5k-2- Ethernet1/1 is a FEX-fabric interface that is in port channel 100:

N5k-2(config)# int ether1/1
N5k-2(config-if)# channel-group 100 force
 
   

This example shows that N5k-2- Ethernet1/100/2-3 are members of port channel 200:

N5k-2(config)# interface Ethernet100/1/2-3
N5k-2(config-if-range)# channel-group 200 force
 
   

Note In Cisco NX-OS Release 5.0(2)N2(1), if you do not use the channel-group 200 force command on the Ethernet interfaces, a problem will occur on pre-provisioned interfaces that are offline. In this example, if module 100 is offline, the configuration on PO 200 in Step 9 must be specifically configured on each member interface, in addition to the channel-group command. The channel-group 200 force command is not supported in Cisco NX-OS Release 5.0(2)N1(1) and earlier releases.


N5k-1(config)# interface Ethernet100/1/2-3
N5k-1(config-if-range)# switchport mode trunk
N5k-1(config-if-range)# switchport trunk allowed vlan 1-5
 
   
N5k-2(config)# interface Ethernet100/1/2-3
N5k-2(config-if-range)# switchport mode trunk
N5k-2(config-if-range)# switchport trunk allowed vlan 1-5
 
   

Step 10 Modify the port-channel configuration in the switch profile.

N5k-1(config-sync-sp-if)# interface Port-Channel100
N5k-1(config-sync-sp-if)# switchport mode fex-fabric 
N5k-1(config-sync-sp-if)# fex associate 100
N5k-1(config-sync-sp-if)# vpc 100
 
   
N5k-1(config-sync-sp)# interface Port-channel200
N5k-1(config-sync-sp-if)# inherit port-profile pc-profile
N5k-1(config-sync-sp-if)# switchport trunk allowed vlan 1-5
 
   

Step 11 Commit the configuration in the switch profile.

N5k-1(config-sync-sp)# commit
 
   

Existing Deployment with an Active/Active FEX Topology

In an existing deployment, the configurations are already present and configuration synchronization is used to simplify future configuration modifications.

To configure peer switches in the vPC topology shown in Figure 4-3, follow these steps:


Step 1 Configure the CFSoIP multicast address on each peer switch:

N5k-1# config terminal
N5k-1(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   
N5k-2# config terminal
N5k-2(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   

Step 2 Enable CFSoIP on both switches.

N5k-1(config)# cfs ipv4 distribute
 
   
N5k-2(config)# cfs ipv4 distribute
 
   

Step 3 Create a switch profile on both switches.

N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
 
   

Step 4 Pre-provision the FEX on both switches.


Note In a FEX active/active topology, always pre-provision the FEXs that are dual-homed inside the switch profile.


N5k-1(config-sync-sp)# slot 100
N5k-1(config-sync-sp-slot)# provision model N2k-C2232P
N5k-1(config-sync-sp-slot)# exit
 
   
N5k-2(config-sync-sp)# slot 100
N5k-2(config-sync-sp-slot)# provision model N2k-C2232P
N5k-2(config-sync-sp-slot)# exit
 
   

Step 5 Commit the configuration in the switch profile on both switches.

N5k-1(config-sync-sp)# commit
 
   
N5k-2(config-sync-sp)# commit
 
   

Step 6 Import the running configuration.

N5k-1(config-sync-sp)# import running-config
N5k-1(config-sync-sp-import)# show switch-profile Test buffer
 
   

Import the configuration to the switch profile on both switches. You can import the configuration using one of the following three methods:

Running configuration—All configurations that are allowed inside a switch profile are imported. You must remove unwanted configurations. For example, you must remove port-channel member configurations if the member interfaces do not match on the peer switches.

Interface configuration—Only specified interface configurations are imported.

Manual mode—Selected configurations are imported. If the configuration that needs to be imported is small, use the manual mode to paste the desired configuration.

Table 4-1 shows the command sequence to import the running configuration:

Table 4-1 Command Sequence to Import the Running Configuration

Sequence Number
Command
1
vlan 1-10
2
2.1
2.2
2.3
interface port-channel100
switchport mode fex-fabric
vpc 100
fex associate 100
3
3.1
3.2
interface port-channel200
switchport mode trunk
switchport trunk allowed vlan 1-5
4
4.1
4.2
4.3
interface Ethernet1/1
fex associate 100
switchport mode fex-fabric
channel-group 100
5
5.1
5.2
interface Ethernet100/1/1
switchport mode trunk
switchport trunk allowed vlan 1-10
6
6.1
6.2
6.3
interface Ethernet100/1/2
switchport mode trunk
switchport trunk allowed vlan 1-5
channel-group 200
7
7.1
7.2
7.3
interface Ethernet100/1/3
switchport mode trunk
switchport trunk allowed vlan 1-5
channel-group 200

Step 7 Remove member interfaces of PO 100 and PO 200 from the buffer.

N5k-1(config-sync-sp-import)# buffer-delete 4, 6, 7
 
   

Use the buffer-delete command to delete the unwanted configuration from the buffer.

Step 8 Commit the configuration in the switch profile on both switches.

N5k-1(config-sync-sp-import)# commit
 
   
N5k-2(config-sync-sp-import)# commit
 
   

Step 9 Add the sync peer on both switches.


Note When importing the configuration, you must use the sync-peers command after the configurations are imported independently on both switches.


N5k-1# config sync
N5k-1(config-sync)# switch-profile sp
N5k-1(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-2>
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile sp
N5k-2(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-1>
 
   


Caution When you remove a switch profile using the no switch-profile name [all-config | local-config] command, the configuration in the switch profile is immediately removed from the running configuration. This disrupts the configurations that were present in the switch profile, for example port channel and vPC configurations. For current information about this issue, refer to CSCtl87240 and CSCtl87260 in the Cisco Nexus 5000 Series Switch and Cisco Nexus 2000 Series Fabric Extender Release Notes and in the Cisco Bug Toolkit located at the following URL: http://tools.cisco.com/Support/BugToolKit/action.do?hdnAction=searchBugs.

Straight-Through Topology Examples

This section includes the following examples:

Switch vPC Topology and Straight-Through FEX Topologies (Host vPC)

New Deployment in a vPC Topology and Straight-Through FEX Topology

Existing Deployments in a vPC Topology and Straight-Through FEX Topology

Switch vPC Topology and Straight-Through FEX Topologies (Host vPC)

In Figure 4-4, the Cisco Nexus 5000 Series switch ports are directly connected to another switch or host and are configured as part of a port channel that becomes part of a vPC.

Figure 4-4 shows that vPC 20 is configured on port channel 20, which has Eth1/10 on N5k-1 and Eth2/1 on N5k-2 as members.

Figure 4-4 Switch vPC Topology

In Figure 4-5, each FEX is single-homed (straight-through FEX topology) with a Cisco Nexus 5000 Series switch. The host interfaces on this FEX are configured as port channels and those port channels are configured as vPCs.

Eth100/1/1 on N5k-1 and Eth102/1/5 on N5k-2 are configured as members of PO200 and PO200 is configured for vPC 200.

Figure 4-5 FEX Straight-Through Topology (Host vPC)

In both topologies, port channels P020 and P0200 must be configured identically on the peer switches and configuration synchronization is used to synchronize the configurations of the vPC switches.

Example 4-3 shows the sample running configuration that must be configured for the peer switches shown in the vPC topologies in Figure 4-4 and Figure 4-5.

Example 4-3 Running Configuration Example for the Nexus 5000 Series Switches in a vPC Straight-Through Topology.

Basic Configuration—No Port Profile
Port Profile Configuration
vlan 1-10
 
        
interface port-channel 20
	switchport mode trunk
	vpc 20
	switchport trunk allowed vlan 1-5
 
        
interface port-channel 200
	switchport mode trunk
	vpc 200
	switchport trunk allowed vlan 1-5
 
        
interface Ethernet1/10
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 20
 
        
interface Ethernet100/1/1
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 200
vlan 1-10
 
        
port-profile type port-channel pc-profile
	switchport mode trunk
	state enabled
 
        
interface port-channel 20
	inherit port-profile pc-profile
	vpc 20
	switchport trunk allowed vlan 1-5
 
        
interface port-channel 200
	inherit port-profile pc-profile
	vpc 200
	switchport trunk allowed vlan 1-5
 
        
interface Ethernet1/10
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 20
 
        
interface Ethernet100/1/1
	switchport mode trunk
	switchport trunk allowed vlan 1-5
	channel-group 200
 
        

New Deployment in a vPC Topology and Straight-Through FEX Topology

In a new deployment, configuration synchronization is introduced initially to synchronize the new configuration. Because it is a new deployment, there is no existing running configuration on the FEX ports.


Note In a straight-through FEX topology, you must use configuration terminal mode to pre-provision FEXs or GEMs.


To configure the peer switches in the topologies shown in Figure 4-4 and Figure 4-5, follow these steps:


Step 1 Pre-provision the FEX configuration in configuration terminal mode for both switches as follows:

Provision the N5k-1- slot 100 for FEX 100.

N5k-1# config terminal
N5k-1(config)# slot 100
N5k-1(config-slot)# provision model N2K-C2232P
N5k-1(config)# int ether 1/1-2
N5k-1(config-if-range)# channel-group 100
N5k-1(config-if-range)# int po100
N5k-1(config-if)# fex associate 100
N5k-1(config-if)# switchport mode fex-fabric 
 
   

Provision the N5k-2- slot 102 for FEX 102.

N5k-2# config terminal
N5k-2(config)# slot 102
N5k-2(config-slot)# provision model N2K-C2232P
N5k-2(config)# int ether 1/1-2
N5k-2(config-if-range)# channel-group 102
N5k-2(config-if-range)# int po102
N5k-2(config-if)# fex associate 102
N5k-2(config-if)# switchport mode fex-fabric 
 
   

Provision the N5k-2- slot 2 for a GEM.

N5k-2(config)# slot 2
N5k-2(config-slot)# provision model N55-M16P
 
   

Step 2 Configure the CFSoIP multicast address on each peer switch:

N5k-1(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   
N5k-2(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   

Step 3 Enable CFSoIP on both switches.

N5k-1(config)# cfs ipv4 distribute
 
   
N5k-2(config)# cfs ipv4 distribute
 
   

Step 4 Create a switch profile and configure the peer on both switches.

N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
N5k-1(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-2>
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-1>
 
   

Step 5 Add the referred global configuration to the switch profile. Because the configuration on the interfaces will be synchronized, all policies that are applied on the interface must be synchronized (for example, port profiles, QoS and ACL policies).

N5k-1(config-sync-sp)# port-profile type port-channel pc-profile
N5k-1(config-sync-port-prof)# switchport mode trunk
N5k-1(config-sync-port-prof)# state enabled
 
   

Step 6 Create port-channel interfaces inside the switch profile.


Note Use switch profile mode to create the port-channel interfaces.


N5k-1(config-sync-sp)# interface port-channel 20
N5k-1(config-sync-sp)# interface port-channel 200
 
   

Step 7 Commit the configuration in the switch profile.

N5k-1(config-sync-sp)# commit
 
   

Step 8 Add members to the port channel in configuration terminal mode on both switches. When the configuration is done in configuration terminal mode, both switches must be configured independently.


Note In this topology, port-channel members must not be identical on the peer switches. For Cisco NX-OS Release 5.0(2)N1(1), port-channel members should only be configured in configuration terminal mode, not in the switch profile.


N5k-1(config)# interface Ethernet1/10
N5k-1(config-if)# channel-group 20 force
N5k-1(config)# interface Ethernet100/1/1
N5k-1(config-if)# channel-group 200 force
 
   
N5k-2(config)# interface Ethernet2/1
N5k-2(config-if)# channel-group 20 force
N5k-2(config)# interface Ethernet102/1/5
N5k-2(config-if)# channel-group 200 force
 
   

Note In Cisco NX-OS Release 5.0(2)N2(1), if you do not use the channel-group 200 force command on the Ethernet interfaces, a problem will occur on pre-provisioned interfaces that are offline. In this example, if module 100 is offline, the configuration on P0200 in Step 8 must be configured on the member interfaces. The channel-group 200 force command is not supported in Cisco NX-OS Release 5.0(2)N1(1) and earlier releases.


N5k-1(config)# interface Ethernet100/1/1
N5k-1(config-if)# switchport mode trunk
N5k-1(config-if)# switchport trunk allowed vlan 1-5
 
   
N5k-2(config)# interface Ethernet2/1
N5k-2(config-if)# switchport mode trunk
N5k-2(config-if)# switchport trunk allowed vlan 1-5
 
   
N5k-2(config)# interface Ethernet102/1/5
N5k-2(config-if)# switchport mode trunk
N5k-2(config-if)# switchport trunk allowed vlan 1-5
 
   

Note Ethernet 1/10 is not included in the list because it is not pre-provisioned (it is an offline interface).


Step 9 Modify the port-channel configuration in the switch profile.

N5k-1(config-sync-sp)# interface port-channel 20
N5k-1(config-sync-sp-if)# inherit port-profile pc-profile
N5k-1(config-sync-sp-if)# vpc 20
N5k-1(config-sync-sp-if)# switchport trunk allowed vlan 1-5
N5k-1(config-sync-sp)# interface port-channel 200
N5k-1(config-sync-sp-if)# inherit port-profile pc-profile
N5k-1(config-sync-sp-if)# vpc 200
N5k-1(config-sync-sp-if)# switchport trunk allowed vlan 1-5
 
   

Step 10 Commit the configuration in the switch profile.

N5k-1(config-sync-sp)# commit
 
   

Existing Deployments in a vPC Topology and Straight-Through FEX Topology

In an existing deployment, the configurations are already present and configuration synchronization is used to simplify future configuration modifications.


Note In a straight-through FEX topology, use configuration terminal mode to pre-provision FEXs and GEMs.


To configure the peer switches in the topologies shown in Figure 4-4 and Figure 4-5, follow these steps:


Step 1 Pre-provision the FEXs in configuration terminal mode on both switches.

N5k-1(config)# slot 100
 
   
N5k-2(config)# slot 102
 
   

Step 2 Configure the CFSoIP multicast address on each peer switch:

N5k-1# config terminal
N5k-1(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   
N5k-2# config terminal
N5k-2(config)# cfs ipv4 mcast-address 10.29.170.1
Distribution over this IP type will be affected
Change multicast address for CFS-IP?
Are you sure? (y/n) [n] y
 
   

Step 3 Enable CFSoIP on both switches.

N5k-1(config)# cfs ipv4 distribute
 
   
N5k-2(config)# cfs ipv4 distribute
 
   

Step 4 Create a switch profile on both switches.

N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
 
   

Step 5 Import the running configuration.

N5k-1(config-sync-sp)# import running-config
N5k-1(config-sync-sp-import)# show switch-profile Test buffer
 
   

Import the configuration to the switch profile on both switches. You can import the configuration using one of the following three methods:

Running configuration—All configurations that are allowed inside a switch profile are imported. You must remove unwanted configurations. For example, you must remove port-channel member configurations.

Interface configuration—Only specified interface configurations are imported.

Manual mode—Selected configurations are imported. If the configuration that needs to be imported is small, use the manual mode to paste the desired configuration.

Table 4-2 shows the command sequence to import the running configuration for Step 4:

Table 4-2 Command Sequence to Import the Running Configuration (Step 4) 

Sequence Number
Command
1
vlan 1-10
2
2.1
2.2
2.3
interface port-channel20
switchport mode trunk
vpc 20
switchport trunk allowed vlan 1-5
3
3.1
3.2
interface port-channel100
switchport mode fex-fabric
fex associate 100
3
3.1
3.2
interface port-channel200
switchport mode trunk
vpc 200
switchport trunk allowed vlan 1-5
4
4.1
4.2
4.3
interface port-channel20
switchport mode trunk
vpc 20
switchport trunk allowed vlan 1-5
5
5.1
5.2
5.3
interface Ethernet1/1
fex associate 100
switchport mode fex-fabric
channel-group 100
6
6.1
6.2
6.3
interface Ethernet1/2
fex associate 100
switchport mode fex-fabric
channel-group 100
7
7.1
7.2
7.3
interface Ethernet1/10
switchport mode trunk
switchport trunk allowed vlan 1-5
channel-group 20
8
8.1
8.2
8.3
interface Ethernet100/1/1
switchport mode trunk
switchport trunk allowed vlan 1-5
channel-group 200

N5k-2(config-sync-sp)# import running-config
 
   

Step 6 (Optional) If you do not want to synchronize the fabric configuration, remove the fabric configuration and the member interfaces of PO 20 and PO 200 from the buffer.

N5k-1(config-sync-sp-import)# buffer-delete 3,5,6-8
 
   

The buffer-delete command deletes the unwanted configuration from the buffer.

Step 7 Commit the configuration in the switch profile on both switches.

N5k-1(config-sync-sp-import)# commit
 
   
N5k-2(config-sync-sp-import)# commit
 
   

Step 8 Add the sync peer on both switches.


Note When importing a configuration, use the sync-peers command after you import configurations on both switches independently.


N5k-1# config sync
N5k-1(config-sync)# switch-profile Test
N5k-1(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-2>
 
   
N5k-2# config sync
N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync-sp)# sync-peers destination <out of band mgmt0 IP address of peer switch 
N5k-1>
 
   


Caution When you remove a switch profile using the no switch-profile name [all-config | local-config] command, the configuration in the switch profile is immediately removed from the running configuration. This disrupts the configurations that were present in the switch profile, for example port channel and vPC configurations. For current information about this issue, refer to CSCtl87240 and CSCtl87260 in the Cisco Nexus 5000 Series Switch and Cisco Nexus 2000 Series Fabric Extender Release Notes and in the Cisco Bug Toolkit located at the following URL: http://tools.cisco.com/Support/BugToolKit/action.do?hdnAction=searchBugs.

Reloading a Cisco Nexus 5000 Series Switch

In this deployment, the N5k-2 switch reboots and a new configuration was committed on N5k-1 using a switch profile.

Example 4-4 shows the configuration that was synchronized between the peers prior to the N5k-2 reload.

Example 4-4 Synchronized Configuration for Peer Switches Prior to the N5k-2 Reload

N5k-1 Running Configuration
N5k-2 Running Configuration
interface Ethernet100/1/11
	switchport mode trunk
interface Ethernet100/1/11
	switchport mode trunk

This example shows the configuration change that was made on the N5k-1 during the N5k-2 reload:


Note If the peer is unreachable once the commit is issued (for example, on the N5k-1 switch), the configuration is applied locally.


N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# int ether 100/1/11
N5k-1(config-sync-sp-if)# switchport trunk allowed vlan 5,6
N5k-1(config-sync-sp)# commit
 
   
Verification successful...
Proceeding to apply configuration. This might take a while depending on
amount of configuration in buffer.
Please avoid other configuration changes during this time.
Commit Successful
 
   
N5k-1(config-sync)# show run int ether 100/1/11
 
   
interface Ethernet100/1/11
  switchport mode trunk
  switchport trunk allowed vlan 5-6
 
   

This example shows how to display the vPC consistency parameters:

N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# int ether 100/1/11
N5k-1# show vpc consistency-parameters int ether 100/1/11
 
   
    Legend:
        Type 1 : vPC will be suspended in case of mismatch
 
   
 
   
Name                        Type  Local Value            Peer Value             
-------------               ----  ---------------------- -----------------------
Allowed VLANs               -     5-6                    1-3967,4048-4093      
Local suspended VLANs       -     -                      -
 
   

Synchronizing the Peer Switches After a Switch Reload

To synchronize the configurations on the peer switches after one of the peer switches reloads, follow these steps:


Step 1 Reapply the configurations that were changed on N5k-1.

N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync-sp)# interface ethernet100/1/11
N5k-2(config-sync-sp-if)# switchport trunk all vlan 5-6
 
   

Step 2 Enter the commit command on N5k-2.

N5k-2(config-sync)# switch-profile Test
N5k-2(config-sync-sp)# commit
Verification successful...
Proceeding to apply configuration. This might take a while depending on
amount of configuration in buffer.
Please avoid other configuration changes during this time.
Commit Successful
 
   

Step 3 Verify that the configuration is applied correctly and is synchronized on the peers.

N5k-2# show vpc consistency-parameters int ether 100/1/11
 Legend:
        Type 1 : vPC will be suspended in case of mismatch
 
   
Name                        Type  Local Value            Peer Value             
-------------               ----  ---------------------- -----------------------
Allowed VLANs               -     5-6                    5-6                   
Local suspended VLANs       -     -                      -                     
 
   
 
   
N5k-2# show run int ether 100/1/11
 
   
!Command: show running-config interface Ethernet100/1/11
 
   
interface Ethernet100/1/11
  switchport mode trunk
  switchport trunk allowed vlan 5-6

Note All configurations are applied serially in a best-effort fashion when the FEX comes online.


vPC Peer-Link Failures

When there is a peer-link failure and both switches are operational, the secondary switch shuts down its vPC ports. In a FEX active/active topology, this situation disconnects the active/active FEX on the secondary switch. If the switch profile configuration is changed on the primary switch, the configuration will not be accepted on the secondary switch unless the active/active FEX is pre-provisioned. We recommend that you pre-provision all active/active FEXs when using configuration synchronization.


Note Even if the FEXs have been originally configured in configuration terminal mode and they are operational, you should provision the FEXs in the switch profile to qualify as a provisioned FEX.


In this topology, FEX 100 is provisioned and FEX 101 is not provisioned, and both FEX 100 and 101 are already operational.

Example 4-5 shows the sample running configuration that is present for FEX 100 (which is in an operational state).

Example 4-5 Running Configuration for FEX 100

Running Configuration for FEX 100
fex 100
	pinning max-links 1
	description "FEX0101"
	
interface port-channel200
	switchport mode fex-fabric
	vpc 200
	fex associate 100
 
        
interface Ethernet1/10
	description connects to Fex 100
	fex associate 100
	switchport mode fex-fabric
	channel-group 200

The next step is to provision FEX 100 inside the switch profile.

Example 4-6 shows the running configuration when FEX 100 is provisioned.

Example 4-6 Running Configuration for FEX 100 Provisioned

Running Configuration for FEX 100 with Provisioning
fex 100
	pinning max-links 1
	description "FEX0101"
 
        
slot 100
	provision model N2K-C2148T
 
        
interface port-channel200
	switchport mode fex-fabric
	vpc 200
	fex associate 100
 
        
interface Ethernet1/10
	description connects to Fex 100
	fex associate 100
	switchport mode fex-fabric
channel-group 200

Working Example

This example shows how to provision the FEX:

N5k-1(config-sync-sp)# slot 100
N5k-1(config-sync-sp-slot)# provision model N2K-C2148T
N5k-1(config-sync-sp-slot)# exit
N5k-1(config-sync-sp)# 
N5k-1(config-sync-sp)# commit 
 
   

This example shows that the vPC peer link fails:

N5k-1(config-sync-sp)# sh vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link
 
   
vPC domain id                   : 10  
Peer status                     : peer link is down             
vPC keepalive status           : peer is alive
 
   

This examples shows that in the switch profile, a configuration is added under Ethernet 100/1/1:

N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# int ether 100/1/1
N5k-1(config-sync-sp-if)# switchport mode trunk
N5k-1(config-sync-sp-if)# commit
Verification successful...
Proceeding to apply configuration. This might take a while depending on
amount of configuration in buffer.
Please avoid other configuration changes during this time.
Commit Successful
 
   

This example shows how to verify that both switches are synchronized:

N5k-1(config-if)# sh run int ether 100/1/1
interface Ethernet100/1/1
  switchport mode trunk
 
   
N5k-2(config-if)# sh run int ether 100/1/1
interface Ethernet100/1/1
  switchport mode trunk
 
   

Nonworking Example

Example 4-7 shows the running configuration for FEX 101 that is not provisioned inside the switch profile.

Example 4-7 Running Configuration for FEX 101

Running Configuration for FEX 101
fex 101
	pinning max-links 1
	description "FEX0101"
 
        
interface port-channel201
	switchport mode fex-fabric
	vpc 201
	fex associate 101
 
        
interface Ethernet1/11
	description connects to Fex 101
	fex associate 101
	switchport mode fex-fabric
	channel-group 201

This example shows that the vPC peer link fails:

N5k-1(config-sync-sp)# show vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link
 
   
vPC domain id                   : 10  
Peer status                     : peer link is down             
vPC keepalive status           : peer is alive
 
   

This example shows that configuration changes made on N5k-1 for Ethernet 101/1/1 fail because FEX 101 is not provisioned inside the switch profile:

N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# int ethernet 101/1/1
N5k-1(config-sync-sp-if)# switchport mode trunk
N5k-1(config-sync-sp-if)# commit
Verification successful...
Failed to Commit: Commit Failed
 
   

This example shows how to correct the issue by provisioning FEX 101 inside the switch profile. If FEX 101 is not provisioned inside the switch profile, the configuration changes must be done manually on both switches:

N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# slot 101
N5k-1(config-sync-sp-slot)# provision model N2K-C2148T
N5k-1(config-sync-sp-slot)# commit
 
   

This example shows how to make the same configuration change again:

N5k-1(config-sync)# switch-profile Test
Switch-Profile started, Profile ID is 1
N5k-1(config-sync-sp)# int ether 101/1/1
N5k-1(config-sync-sp-if)# switchport mode trunk 
N5k-1(config-sync-sp-if)# commit
 
   
Verification successful...
Proceeding to apply configuration. This might take a while depending on
amount of configuration in buffer.
Please avoid other configuration changes during this time.
Commit Successful
 
   
!Command: show running-config interface Ethernet101/1/1
 
   
interface Ethernet101/1/1
  switchport mode trunk
 
   
!Command: show running-config interface Ethernet101/1/1
!Time: Tue Oct 19 01:18:14 2010
 
   
version 5.0(2)N1(1)
 
   
interface Ethernet101/1/1
  switchport mode trunk
  speed 1000
 
   

mgmt0 Interface Connectivity is Lost

Configuration synchronization sends switch profile configurations over the mgmt0 interface to the peer switch. When the mgmt0 interface connectivity is lost and the configuration needs to be changed, configure the switch profile on both switches. When the mgmt0 interface is restored, both switches become synchronized.


Note If you make configuration changes when the mgmt0 interface is down, the configurations that are applied on each switch must be identical. If the configurations are not identical, when the mgmt0 interface comes up and you enter a commit command on either switch, the commit fails because of a configuration mismatch.


If you enter the commit command when the mgmt0 interface is up and then the mgmt0 interface goes down, the commit eventually fails when both switches detect that the peer switch is no longer reachable from the mgmt0 interface.

Rollback Failures with Conditional Features

With configuration synchronization, when a conditional feature is present in a checkpoint and not in the running configuration, a rollback to that checkpoint fails. As a workaround, you can reconfigure the conditional feature before a rollback is executed. The workaround applies to the vpc domain and peer-keepalive commands in vpc-domain mode.

This example shows the running configuration of the system when a checkpoint called chkpt is created:

feature vpc
 
   
vpc domain 100
    vpc peer-keepalive destination 10.0.0.1
 
   
interface Ethernet 1/1
    switchport mode trunk
    channel-group 100
 
   
switch-profile Test
    interface port-channel 100
        switchport mode trunk
        vpc peer-link
 
   

If you perform a write-erase at this point and you reload the switch and attempt to perform a rollback to the checkpoint chkpt, the rollback fails. This example shows a rollback failure when this situation occurs:


Note To avoid the rollback failure, preconfigure the feature vpc, vpc domain, and peer-keepalive command before performing the rollback.


N5k-1# rollback running-config checkpoint chkpt verbose
 
   

Note Applying a configuration in parallel might cause a rollback verification to fail.


Collecting Running-Config
Generating Rollback patch for switch profile
Executing Rollback patch for switch profiles. WARNING - This will change the configuration 
of switch profiles and will also affect any peers if configured
========================================================
`config sync `
`switch-profile Test`
Switch-Profile started, Profile ID is 1
`interface port-channel100`
`switchport mode trunk`
Syntax error while parsing 'vpc peer-link'
 
   
========================================================
Generating Running-config for verification
 
   
Verification failed, rolling back to previous configuration
Collecting Running-Config
2010 Oct 14 07:43:12 switch %$ VDC-1 %$ %ASCII-CFG-2-ACFG_OPER_FAIL: Operation failed 
because of Rollback Patch is not Empty
...
 
   

Channel Group Failures

The channel-group command fails for port profiles or pre-provisioned interfaces if the port channel does not exist (auto-creation is not supported). The workaround is to explicitly create the port channel first using the interface port-channel xxx command.


Note Port-channel members must be configured in configuration terminal mode.


Nonworking Example

This example shows the error message that appears when the port-channel interface is not created first:

N5k-1(config-if-range)# int ether 100/1/2-3
N5k-1(config-if-range)# channel-group 200
Pre-provisioned interface: port channel must exist first
 
   

The channel-group command fails when a module comes online if you make a configuration change on a port channel but not on the pre-provisioned interfaces. The failure does not occur in Cisco NX-OS Release 5.0(2)N2(1) which supports the channel-group xxx force command.

At-A-Glance Configuration Modes

Table 4-3 shows which mode is used to configure features and interfaces in an active/active or straight-through topology and for the switches and hosts in a vPC topology.

For example, to configure a port-channel interface in an active/active topology, use the switch profile configuration mode.

Table 4-3 At-A-Glance Topology Configuration Modes 

Configuration
FEX Active/Active Topology
Straight-Through Topology
Switch / Host vPC Topology

Nonport-channel member

Switch profile mode

Switch profile mode

Switch profile mode

Port-channel interface (interface port channel command)

Switch profile mode

Switch profile mode

Switch profile mode

Port-channel member interfaces

Configuration terminal mode

Configuration terminal mode

Configuration terminal mode

FEX/GEM pre-provisioning

Switch profile mode

Configuration terminal mode

Configuration terminal mode

Global configurations (for example, port profiles, QoS, ACLs, and so on)

Switch profile mode

Switch profile mode

Switch profile mode


Terminology

Port-channel interface—Interface that is part of a port channel.

Nonport-channel member—Stand-alone interface that is not part of any port channel.

Port-channel member—Interface that is a member of a port channel.

Switch profile—Predefined configuration profile that is used to synchronize a consistent configuration across peer switches. Switch profiles are used in the configuration synchronization feature.

Port profile—Interface-command profile that can be applied to a range of interfaces (for example, Ethernet, VLAN network interface, or port channel).

Configuration synchronization—Feature that uses a switch profile to synchronize consistent configurations between two peer switches.

config-sync mode—Configuration mode that is used to define and access a switch profile.

Configuration terminal mode (config-t)—Configuration mode used to commit configurations locally on a switch.

Pre-provisioning—Ability to configure offline interfaces before they are connected (or brought online). Pre-provisioning can be done on Cisco Nexus 2000 Fabric Extenders (FEXs) and/or Generic Expansion Module (GEMs).