- Preface
- New and Changed Information
- Overview of Cisco Unified Computing System
- Overview of Cisco UCS Manager
- Overview of Cisco UCS Manager GUI
- Configuring the Fabric Interconnects
- Configuring Ports and Port Channels
- Configuring Communication Services
- Configuring Authentication
- Configuring Organizations
- Configuring Role-Based Access Control
- Configuring DNS Servers
- Configuring System-Related Policies
- Managing Licenses
- Managing Virtual Interfaces
- Registering Cisco UCS Domains with Cisco UCS Central
- LAN Uplinks Manager
- VLANs
- Configuring LAN Pin Groups
- Configuring MAC Pools
- Configuring Quality of Service
- Configuring Network-Related Policies
- Configuring Upstream Disjoint Layer-2 Networks
- Configuring Named VSANs
- Configuring SAN Pin Groups
- Configuring WWN Pools
- Configuring Storage-Related Policies
- Configuring Fibre Channel Zoning
- Configuring Server-Related Pools
- Setting the Management IP Address
- Configuring Server-Related Policies
- Configuring Server Boot
- Deferring Deployment of Service Profile Updates
- Service Profiles
- Configuring Storage Profiles
- Managing Power in Cisco UCS
- Managing Time Zones
- Managing the Chassis
- Managing Blade Servers
- Managing Rack-Mount Servers
- Starting the KVM Console
- CIMC Session Management
- Managing the I/O Modules
- Backing Up and Restoring the Configuration
- Recovering a Lost Password
- Initial System Setup
- Performing an Initial System Setup for a Standalone Configuration
- Initial System Setup for a Cluster Configuration
- Enabling a Standalone Fabric Interconnect for Cluster Configuration
- Configuring the Information Policy on the Fabric Interconnect
- Fabric Evacuation
- Ethernet Switching Mode
- Configuring Ethernet Switching Mode
- Fibre Channel Switching Mode
- Configuring Fibre Channel Switching Mode
- Changing the Properties of the Fabric Interconnects
- Determining the Leadership Role of a Fabric Interconnect
Configuring the Fabric Interconnects
This chapter includes the following sections:
- Initial System Setup
- Performing an Initial System Setup for a Standalone Configuration
- Initial System Setup for a Cluster Configuration
- Enabling a Standalone Fabric Interconnect for Cluster Configuration
- Configuring the Information Policy on the Fabric Interconnect
- Fabric Evacuation
- Ethernet Switching Mode
- Configuring Ethernet Switching Mode
- Fibre Channel Switching Mode
- Configuring Fibre Channel Switching Mode
- Changing the Properties of the Fabric Interconnects
- Determining the Leadership Role of a Fabric Interconnect
Initial System Setup
The first time that you access a fabric interconnect in a Cisco UCS domain, a setup wizard prompts you for the following information required to configure the system:
-
Installation method (GUI or CLI)
-
Setup mode (restore from full system backup or initial setup)
-
System configuration type (standalone or cluster configuration)
-
System name
-
Admin password
-
Management port IPv4 address and subnet mask, or IPv6 address and prefix
-
Default gateway IPv4 or IPv6 address
-
DNS Server IPv4 or IPv6 address
-
Default domain name
Setup Mode
You can choose to either restore the system configuration from an existing backup file, or manually set up the system by going through the Setup wizard. If you choose to restore the system, the backup file must be reachable from the management network.
System Configuration Type
You can configure a Cisco UCS domain to use a single fabric interconnect in a standalone configuration or to use a redundant pair of fabric interconnects in a cluster configuration.
A cluster configuration provides high availability. If one fabric interconnect becomes unavailable, the other takes over. Only one management port (Mgmt0) connection is required to support a cluster configuration; however, both Mgmt0 ports should be connected to provide link-level redundancy.
In addition, a cluster configuration actively enhances failover recovery time for redundant virtual interface (VIF) connections. When an adapter has an active VIF connection to one fabric interconnect and a standby VIF connection to the second, the learned MAC addresses of the active VIF are replicated but not installed on the second fabric interconnect. If the active VIF fails, the second fabric interconnect installs the replicated MAC addresses and broadcasts them to the network through gratuitous ARP messages, shortening the switchover time.
![]() Note | The cluster configuration provides redundancy only for the management plane. Data redundancy is dependent on the user configuration and might require a third-party tool to support data redundancy. |
To use the cluster configuration, you must directly connect the two fabric interconnects together using Ethernet cables between the L1 (L1-to-L1) and L2 (L2-to-L2) high-availability ports, with no other fabric interconnects in between. Also you can connect the fabric interconnects directly through a patch panel to allow the two fabric interconnects to continuously monitor the status of each other and quickly know when one has failed.
Both fabric interconnects in a cluster configuration must go through the initial setup process. You must enable the first fabric interconnect that you set up for a cluster configuration. When you set up the second fabric interconnect, it detects the first fabric interconnect as a peer fabric interconnect in the cluster.
For more information, see to the Cisco UCS 6100 Series Fabric Interconnect Hardware Installation Guide.
Management Port IP Address
In a standalone configuration, you must specify only one IPv4 address, gateway, and subnet mask, or only one IPv6 address, gateway, and network prefix for the single management port on the fabric interconnect. You can configure either an IPv4 or an IPv6 address for the management port IP address.
In a cluster configuration, you must specify the following three IPv4 addresses in the same subnet, or three IPv6 addresses with the same prefix:
-
Management port IP address for fabric interconnect A
-
Management port IP address for fabric interconnect B
-
Cluster IP address
![]() Note | In a cluster configuration, the management port for both fabric interconnects must be configured with the same address type, either IPv4 or IPv6. If you configure the first FI with an IPv4 address then attempt to configure the second FI with an IPv6 address, the configuration will fail. |
Performing an Initial System Setup for a Standalone Configuration
-
Verify the following physical connections on the fabric interconnect:
-
The console port is physically connected to a computer terminal or console server
-
The management Ethernet port (mgmt0) is connected to an external hub, switch, or router
For more information, refer to the Cisco UCS Hardware Installation Guide for your fabric interconnect.
-
-
Verify that the console port parameters on the computer terminal (or console server) attached to the console port are as follows: -
Collect the following information that you will need to supply during the initial setup:
-
System name
-
Password for the admin account. Choose a strong password that meets the guidelines for Cisco UCS Manager passwords. This password cannot be blank.
-
Management port IPv4 and subnet mask, or IPv6 address and prefix.
-
Default gateway IPv4 or IPv6 address.
-
DNS server IPv4 or IPv6 address (optional).
-
Domain name for the system (optional).
-
Initial System Setup for a Cluster Configuration
Performing an Initial System Setup on the First Fabric Interconnect
-
Verify the following physical connections on the fabric interconnect:
-
A console port on the first fabric interconnect is physically connected to a computer terminal or console server
-
The management Ethernet port (mgmt0) is connected to an external hub, switch, or router
-
The L1 ports on both fabric interconnects are directly connected to each other
-
The L2 ports on both fabric interconnects are directly connected to each other
For more information, refer to the Cisco UCS Hardware Installation Guide for your fabric interconnect.
-
-
Verify that the console port parameters on the computer terminal (or console server) attached to the console port are as follows: -
Collect the following information that you will need to supply during the initial setup:
-
System name.
-
Password for the admin account. Choose a strong password that meets the guidelines for Cisco UCS Manager passwords. This password cannot be blank.
-
Three static IPv4 or IPv6 addresses: two for the management port on both fabric interconnects (one per fabric interconnect) and one for the cluster IP address used by Cisco UCS Manager.
-
Subnet mask for the three static IPv4 addresses, or network prefix for the three static IPv6 addresses.
-
Default gateway IPv4 or IPv6 address.
-
DNS server IPv4 or IPv6 address (optional).
-
Domain name for the system (optional).
-
Step 1 | Connect to the console port. | ||||||||||||||||||||||
Step 2 | Power on the
fabric interconnect.
You will see the power on self-test messages as the fabric interconnect boots. | ||||||||||||||||||||||
Step 3 | At the installation method prompt, enter gui. | ||||||||||||||||||||||
Step 4 | If the system
cannot access a DHCP server, you are prompted to enter the following
information:
| ||||||||||||||||||||||
Step 5 | Copy the web link from the prompt into a web browser and go to the Cisco UCS Manager GUI launch page. | ||||||||||||||||||||||
Step 6 | On the Cisco UCS Manager GUI launch page, select Express Setup. | ||||||||||||||||||||||
Step 7 | On the Express Setup page, select Initial Setup and click Submit. | ||||||||||||||||||||||
Step 8 | In the Cluster and Fabric Setup Area: | ||||||||||||||||||||||
Step 9 | In the
System Setup Area, complete the following fields:
| ||||||||||||||||||||||
Step 10 | Click Submit. A page displays the results of your setup operation. |
Performing an Initial System Setup on the Second Fabric Interconnect
You must ensure the following:
Step 1 | Connect to the console port. | ||||||
Step 2 | Power on the
fabric interconnect.
You will see the power on self-test messages as the fabric interconnect boots. | ||||||
Step 3 | At the installation method prompt, enter gui. | ||||||
Step 4 | If the system
cannot access a DHCP server, you are prompted to enter the following
information:
| ||||||
Step 5 | Copy the web link from the prompt into a web browser and go to the Cisco UCS Manager GUI launch page. | ||||||
Step 6 | On the Cisco UCS Manager GUI launch page, select Express Setup. | ||||||
Step 7 | On the
Express
Setup page, select
Initial
Setup and click
Submit.
The fabric interconnect should detect the configuration information for the first fabric interconnect. | ||||||
Step 8 | In the Cluster and Fabric Setup Area: | ||||||
Step 9 | In the System Setup Area, enter the password for the Admin account into the Admin Password of Master field. The Manager Initial Setup Area is displayed. | ||||||
Step 10 | In the Manager Initial Setup Area, the field that is displayed depends on whether you configured the first fabric interconnect with an IPv4 or IPv6 management address. Complete the field that is appropriate for your configuration as follows:
| ||||||
Step 11 | Click
Submit.
A page displays the results of your setup operation. |
Adding Out-of-band IPv4 Addresses to a Fabric Interconnect
All fabric interconnects require an OOB IPv4 address, network mask and gateway. This procedure describes how to configure an OOB IPv4 address for a fabric interconnect that was set up with static IPv6 addresses.
Collect the out-of-band (OOB) IPv4 address you want to assign to the fabric interconnect.
Command or Action | Purpose | |
---|---|---|
Step 1 | UCS-A # scope fabric interconnect a | Enters fabric configuration mode for Fabric A. |
Step 2 | UCS-A/fabric-interconnect # set out-of-band ip ip-addr netmask ip-addrgw ip-addr | Sets the OOB IPv4 address, network mask and gateway address. The system warns that the console session change may be disconnected when the change is committed. |
Step 3 | UCS-A/fabric-interconnect # commit-buffer | Commits the transaction to the system configuration. |
The following example shows configuring an OOB IPv4 address for fabric interconnect A:
UCS-A# scope fabric-interconnect a UCS-A /fabric-interconnect # set out-of-band ip 10.105.214.107 netmask 255.255.255.0 gw 10.105.214.1 Warning: When committed, this change may disconnect the current CLI session UCS-A /fabric-interconnect* # commit-buffer
Enabling a Standalone Fabric Interconnect for Cluster Configuration
You can add a second fabric interconnect to an existing Cisco UCS domain that uses a single standalone fabric interconnect. To do this, you must enable the standalone fabric interconnect for cluster operation by configuring it with the virtual IP or IPv6 address of the cluster, and then add the second fabric interconnect to the cluster.
Command or Action | Purpose | |
---|---|---|
Step 1 | UCS-A# connect local-mgmt |
Enters local management mode. |
Step 2 | UCS-A(local-mgmt) # enable cluster {virtual-ip-addr|virtual-ip6-addr} |
Enables cluster operation on the standalone fabric interconnect with the specified IPv4 or IPv6 address. When you enter this command, you are prompted to confirm that you want to enable cluster operation. Type yes to confirm. The IP address must be the virtual IPv4 or IPv6 address for the cluster configuration, not the IP address assigned to the fabric interconnect that you are adding to the cluster. |
The following example enables a standalone fabric interconnect with a virtual IPv4 address of 192.168.1.101 for cluster operation:
UCS-A# connect local-mgmt UCS-A(local-mgmt)# enable cluster 192.168.1.101 This command will enable cluster mode on this setup. You cannot change it back to stand-alone. Also, any GUI or KVM sessions may be terminated. Are you sure you want to continue? (yes/no): yes UCS-A(local-mgmt)#
The following example enables a standalone fabric interconnect with a virtual IPv6 address of 192.168.1.101 for cluster operation:
UCS-A# connect local-mgmt UCS-A(local-mgmt)# enable cluster ipv6 2001::109 This command will enable IPv6 cluster mode on this setup. You cannot change it back to stand-alone. Also, any GUI or KVM sessions may be terminated. Are you sure you want to continue? (yes/no): yes UCS-A(local-mgmt)#
Add the second fabric interconnect to the cluster.
Configuring the Information Policy on the Fabric Interconnect
You must configure the information policy to display the uplink switches that are connected to Cisco UCS.
You must enable the information policy on the fabric interconnect to view the SAN, LAN, and LLDP neighbors of the fabric interconnect.
- Enabling or Disabling the Information Policy on the Fabric Interconnect
- Viewing the LAN Neighbors of a Fabric Interconnect
- Viewing the SAN Neighbors of a Fabric Interconnect
- Viewing the LLDP Neighbors of a Fabric Interconnect
Enabling or Disabling the Information Policy on the Fabric Interconnect
Step 1 | In the Navigation pane, click the Equipment tab. | ||||||
Step 2 | On the Equipment tab, click the Equipment node. | ||||||
Step 3 | In the Work pane, click the Policies tab. | ||||||
Step 4 | Click the Global Policies subtab. | ||||||
Step 5 | In the
Info
Policy area, select one of the following:
| ||||||
Step 6 | Click Save Changes. |
Viewing the LAN Neighbors of a Fabric Interconnect
Step 1 | In the Navigation pane, click the Equipment tab. |
Step 2 | In the Equipment tab, expand . |
Step 3 | Click the fabric interconnect for which you want to view the LAN neighbors. |
Step 4 | In the Work pane, click the Neighbors tab. |
Step 5 | Click the
LAN subtab.
This subtab lists all the LAN neighbors of the specified Fabric Interconnect. |
Viewing the SAN Neighbors of a Fabric Interconnect
Step 1 | In the Navigation pane, click the Equipment tab. |
Step 2 | In the Equipment tab, expand . |
Step 3 | Click the fabric interconnect for which you want to view the SAN neighbors. |
Step 4 | In the Work pane, click the Neighbors tab. |
Step 5 | Click the
SAN subtab.
This subtab lists all the SAN neighbors of the specified Fabric Interconnect. |
Viewing the LLDP Neighbors of a Fabric Interconnect
Step 1 | In the Navigation pane, click the Equipment tab. |
Step 2 | In the Equipment tab, expand . |
Step 3 | Click the fabric interconnect for which you want to view the LLDP neighbors. |
Step 4 | In the Work pane, click the Neighbors tab. |
Step 5 | Click the
LLDP subtab.
This subtab lists all the LLDP neighbors of the specified Fabric Interconnect. |
Fabric Evacuation
Cisco UCS Manager 2.2(4) introduces fabric evacuation, which is the ability to evacuate all traffic that flows through a Fabric Interconnect from all servers attached to it through an IOM or FEX while upgrading a system.
Upgrading the secondary Fabric Interconnect in a system disrupts the traffic that is active on the Fabric Interconnect. This traffic fails over to the primary Fabric Interconnect. You can use fabric evacuation as follows during the upgrade process:
-
Stop all the traffic that is active through a Fabric Interconnect.
-
For vNICs configured with failover, verify that the traffic has failed over by using Cisco UCS Manager or tools such as vCenter.
-
Upgrade the secondary Fabric Interconnect.
-
Restart all the stopped traffic flows.
-
Change the cluster lead to the secondary Fabric Interconnect.
-
Repeat steps 1 to 4 and upgrade the other Fabric Interconnect.
![]() Note | Fabric evacuation is supported only with the following: |
Configuring Fabric Evacuation
Command or Action | Purpose | |
---|---|---|
Step 1 | In the Navigation pane, click the Equipment tab. | |
Step 2 | On the Equipment tab, expand . | |
Step 3 | In the Work pane, click the General tab. | |
Step 4 | In the Actions area of the General tab, click Configure Evacuation. | The Configure Evacuation dialog box appears. |
Step 5 | To configure fabric evacuation on the specified Fabric Interconnect, click one of the following radio buttons in the Admin Evac Mode field: | |
Step 6 | To evacuate a Fabric Interconnect irrespective of its current evacuation state, check the Force check box. | (Optional) |
Step 7 | Click Apply. | A warning dialog box appears.
Enabling fabric evacuation will stop all traffic through this Fabric Interconnect from servers attached through IOM/FEX. The traffic will fail over to the Primary Fabric Interconnect for fail over vnics. Are you sure you want to continue? |
Step 8 | Click OK to confirm fabric evacuation and continue. |
Ethernet Switching Mode
The Ethernet switching mode determines how the fabric interconnect behaves as a switching device between the servers and the network. The fabric interconnect operates in either of the following Ethernet switching modes:
End-Host Mode
End-host mode allows the fabric interconnect to act as an end host to the network, representing all servers (hosts) connected to it through vNICs. This behavior is achieved by pinning (either dynamically pinned or hard pinned) vNICs to uplink ports, which provides redundancy to the network, and makes the uplink ports appear as server ports to the rest of the fabric. In end-host mode, the fabric interconnect does not run the Spanning Tree Protocol (STP) but it avoids loops by denying uplink ports from forwarding traffic to each other and by denying egress server traffic on more than one uplink port at a time. End-host mode is the default Ethernet switching mode and should be used if either of the following are used upstream:
![]() Note | When you enable end-host mode, if a vNIC is hard pinned to an uplink port and this uplink port goes down, the system cannot repin the vNIC, and the vNIC remains down. |
Switch Mode
Switch mode is the traditional Ethernet switching mode. The fabric interconnect runs STP to avoid loops, and broadcast and multicast packets are handled in the traditional way. Switch mode is not the default Ethernet switching mode, and should be used only if the fabric interconnect is directly connected to a router, or if either of the following are used upstream:
![]() Note | For both Ethernet switching modes, even when vNICs are hard pinned to uplink ports, all server-to-server unicast traffic in the server array is sent only through the fabric interconnect and is never sent through uplink ports. Server-to-server multicast and broadcast traffic is sent through all uplink ports in the same VLAN. |
Configuring Ethernet Switching Mode
When you change the Ethernet switching mode, Cisco UCS Manager logs you out and restarts the fabric interconnect. For a cluster configuration, Cisco UCS Manager restarts both fabric interconnects. The second fabric interconnect can take several minutes to complete the change in Ethernet switching mode and become system ready. The configuration is retained.
While the fabric interconnects are rebooting, all blade servers will lose all LAN and SAN connectivity, causing a complete outage of all services on the blades. This may cause the operating system to crash.
Step 1 | In the Navigation pane, click the Equipment tab. |
Step 2 | On the Equipment tab, expand . |
Step 3 | In the Work pane, click the General tab. |
Step 4 | In the
Actions area of the
General tab, click one of the following
links:
The link for the current mode is dimmed. |
Step 5 | In the dialog box, click
Yes.
Cisco UCS Manager restarts the fabric interconnect, logs you out, and disconnects Cisco UCS Manager GUI. |
Fibre Channel Switching Mode
The Fibre Channel switching mode determines how the fabric interconnect behaves as a switching device between the servers and storage devices. The fabric interconnect operates in either of the following Fibre Channel switching modes:
End-Host Mode
End-host mode allows the fabric interconnect to act as an end host to the connected fibre channel networks, representing all servers (hosts) connected to it through virtual host bus adapters (vHBAs). This behavior is achieved by pinning (either dynamically pinned or hard pinned) vHBAs to Fibre Channel uplink ports, which makes the Fibre Channel ports appear as server ports (N-ports) to the rest of the fabric. When in end-host mode, the fabric interconnect avoids loops by denying uplink ports from receiving traffic from one another.
End-host mode is synonymous with N Port Virtualization (NPV) mode. This mode is the default Fibre Channel Switching mode.
![]() Note | When you enable end-host mode, if a vHBA is hard pinned to an uplink Fibre Channel port and this uplink port goes down, the system cannot repin the vHBA, and the vHBA remains down. |
Switch Mode
Switch mode is the traditional Fibre Channel switching mode. Switch mode allows the fabric interconnect to connect directly to a storage device. Enabling Fibre Channel switch mode is useful in Pod models where there is no SAN (for example, a single Cisco UCS domain that is connected directly to storage), or where a SAN exists (with an upstream MDS).
Switch mode is not the default Fibre Channel switching mode.
![]() Note | In Fibre Channel switch mode, SAN pin groups are irrelevant. Any existing SAN pin groups are ignored. |
Configuring Fibre Channel Switching Mode
When you change the Fibre Channel switching mode, Cisco UCS Manager logs you out and restarts the fabric interconnect. For a cluster configuration, Cisco UCS Manager restarts both fabric interconnects simultaneously. The second fabric interconnect can take several minutes to complete the change in Fibre Channel switching mode and become system ready.
![]() Note | When the Fibre Channel switching mode is changed, both UCS fabric interconnects will reload simultaneously. Reloading of fabric interconnects will cause a system-wide downtime for approximately 10-15 minutes. |
Step 1 | In the Navigation pane, click the Equipment tab. |
Step 2 | On the Equipment tab, expand . |
Step 3 | In the Work pane, click the General tab. |
Step 4 | In the
Actions area of the
General tab, click one of the following links:
The link for the current mode is dimmed. |
Step 5 | In the dialog
box, click
Yes.
Cisco UCS Manager restarts the fabric interconnect, logs you out, and disconnects Cisco UCS Manager GUI. |
Changing the Properties of the Fabric Interconnects
![]() Note | To change the subnet or network prefix for a Cisco UCS domain, you must simultaneously change all subnets or prefixes, the virtual IPv4 or IPv6 address used to access Cisco UCS Manager, and the IPv4 or IPv6 addresses for both fabric interconnects. Both fabric interconnects must maintain the same management address type, either IPv4 or IPv6. You cannot change the management address type for Fabric A without changing the management address type for Fabric B. |
Step 1 | In the Navigation pane, click the Admin tab. |
Step 2 | On the Admin tab, click All. |
Step 3 | In the Work pane, click the General tab. |
Step 4 | In the Actions area, click Management Interfaces to open the Management Interfaces dialog box. |
Step 5 | In the Management Interfaces dialog box, modify the values as necessary. |
Step 6 | To change only the virtual IP address that you use to access Cisco UCS Manager, enter the desired IP address in either the IPv4 Address or the IPv6 Address field in the Virtual IP area. |
Step 7 | To change only the name assigned to the Cisco UCS domain, enter the desired name in the Name field in the Virtual IP area. |
Step 8 | To change the
subnet and IPv4 address, or the network prefix and IPv6 address, and default
gateway assigned to the fabric interconnects, update the following fields:
|
Step 9 | Click OK. |
Step 10 | Log out of Cisco UCS Manager GUI and log back in again to see your changes. |
Determining the Leadership Role of a Fabric Interconnect
Step 1 | In the Navigation pane, click the Equipment tab. |
Step 2 | In the Equipment tab, expand . |
Step 3 | Click the fabric interconnect for which you want to identify the role. |
Step 4 | In the Work pane, click the General tab. |
Step 5 | In the General tab, click the down arrows on the High Availability Details bar to expand that area. |
Step 6 | View the Leadership field to determine whether the fabric interconnect is the primary or subordinate. |