This chapter describes how to diagnose server system problems using LEDs. It also provides information about how to install or replace hardware components, and it includes the following sections:
You can monitor the server inventory, health, and system event logs by using the built-in Cisco Integrated Management Controller (CIMC) GUI or CLI interfaces. See the user documentation for your firmware release at the following URL:
http://www.cisco.com/en/US/products/ps10739/products_installation_and_configuration_guides_list.html
Cisco has also developed the Cisco Server Configuration Utility for C-Series servers, which can aid and simplify the following tasks:
This utility is shipped with new servers on CD. You can also download the ISO from Cisco.com. See the user documentation for this utility at the following URL:
http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/ucsscu/user/guide/20/SCUUG20.html
This section describes the location and meaning of LEDs and buttons, and it includes the following topics:
Figure 3-1 shows the front panel LEDs.
|
|
||
|
|
||
|
|
||
|
|
Operations panel (see Operations Panel LEDs and Buttons). |
Figure 3-2 shows the operations panel LEDs and buttons.
Figure 3-2 Operations Panel LEDs and Buttons
|
|
||
|
|
||
|
|
||
|
|
||
|
|
|
|
---|---|
Figure 3-3 shows the rear panel LEDs and buttons.
Figure 3-3 Rear Panel LEDs and Buttons
|
|
||
|
|
||
|
|
|
|
---|---|
This section describes how to prepare for component installation, and it includes the following topics:
The following equipment is used to perform the procedures in this chapter:
The server can run in two power modes:
You can invoke a graceful shutdown or an emergency shutdown (hard shutdown) by using either of the following methods:
Step 1 Check the color of the Power Status LED (see the “Operations Panel LEDs and Buttons” section).
Step 2 Invoke either a graceful shutdown or a hard shutdown:
Step 3 Disconnect the power cords from the power supplies in your server to completely power off the server.
Use the following procedure to remove or replace the top cover of the server:
Tip You do not have to remove the cover to replace hard drives or power supplies.
a. Simultaneously press the two green release buttons. See Figure 3-4.
b. Push the cover toward the server rear about three inches, until it stops.
c. Lift the cover straight up from the server and set it aside.
a. Place the cover on top of the server about 3 inches behind the front of the chassis. The cover should sit flat when the cover flanges are sitting in the grooves in the chassis.
b. Slide the cover toward the front of the server until it stops at the front panel and the green buttons lock.
Figure 3-4 Removing the Top Cover
|
|
This section shows the locations of the components that are discussed in this chapter. The view in Figure 3-5 is from the top down, with the top cover and internal CPU cage removed.
Figure 3-5 Replaceable Component Locations
|
|
Memory risers, which provide slots for up to 8 DIMMs on each riser. |
|
|
|
RAID battery backup unit (optional when using the LSI 9260 controller) |
|
|
|
||
|
|
||
|
|
||
|
|
PCIe connector 10 (10 of 10) |
|
|
|
Warning Blank faceplates and cover panels serve three important functions: they prevent exposure to hazardous voltages and currents inside the chassis; they contain electromagnetic interference (EMI) that might disrupt other equipment; and they direct the flow of cooling air through the chassis. Do not operate the system unless all cards, faceplates, front covers, and rear covers are in place.
Statement 1029
Warning Class 1 laser product.
Statement 1008
Tip You can press the ID button on the front panel or rear panel to turn on a flashing ID LED on the front and rear panels of the server. This allows you to locate the specific server that you are servicing when you go to the opposite side of the rack. See the “Status LEDs and Buttons” section for locations of the LEDs.
This section describes how to install and replace server components, and it includes the following topics:
The server can have two or four power supplies. Four power supplies are required for four-CPU configurations.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace or install a power supply, follow these steps:
Note If you have ordered a server with power supply redundancy (at least four power supplies), you do not have to power off the server to replace power supplies. Also see the note below about 1+1 redundancy.
Step 1 Remove the power supply that you are replacing or a blank panel from an empty bay. See Figure 3-6:
a. Do one of the following actions:
– If your server has only two power supplies, shut down and power off the server as described in the “Shutting Down and Powering Off the Server” section. See the following note.
Note You can use 2 power supplies with a 2-CPU configuration. In this case, 1+1 redundancy is supported only if your server’s overall power consumption can be supported by a single 850W power supply during the failure and hot-swap. For more information about your server’s power consumption, consult with your Cisco sales representative or use the power calculator accessible at the Unified Computing System Partner Resource Center:
http://www.ciscoprc.com/resourcelib.asp?id=937
– If your server has four power supplies, you do not have to shut down the server.
b. Remove the power cord from the power supply that you are replacing.
a. Grasp the power supply handle while squeezing the release lever towards the handle.
b. Pull the power supply out of the slot.
Step 2 Install a new power supply:
a. Grasp the power supply handle and insert the power supply into the power supply bay.
b. Push the power supply into the bay until the release lever locks.
c. Replace the power cord to the new power supply.
d. Press the Power button to return the server to main power mode.
Figure 3-6 Removing and Replacing Power Supplies
|
|
Depending on the power being drawn by the server, one or more power supplies might actively provide all power to the system while the remaining power supplies are put into a standby state. This is known as cold redundancy.
The server’s CIMC interface reports the power supply redundancy based on the number of supplies that are plugged in with AC power applied. For example:
With cold redundancy, depending on the server’s power usage, one or more supplies might actively provide all power to the system while the remaining supplies are put into a standby state. For example, if you have two supplies connected to AC power, but the power consumption can be satisfied by power supply 1, then power supply 2 is put into a standby state.
The I/O riser is module that connects to the motherboard and that provides the ports for rear panel connectivity. To install or replace the I/O riser, follow these steps:
Step 1 Remove the I/O riser you are replacing. See Figure 3-7:
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Disconnect all cables from the ports on the I/O riser.
Tip Label the cables when you remove them to aid in identifying them for replacement.
c. Slide the server out the front of the rack far enough so that you can remove the top cover.
d. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
e. Pinch and lift up the green retaining clip that secures the I/O riser to the chassis rear panel.
f. Lift the riser straight up from the motherboard connector and out of the chassis.
Step 2 Install a new I/O riser:
a. Align the I/O riser with the empty I/O riser connector on the motherboard.
b. Push down evenly on both ends of the I/O riser until it is fully seated in the motherboard connector.
c. Ensure that the I/O riser rear panel sits flat against the chassis back panel opening.
d. Push the green retaining clip down until it locks over the top of the I/O riser.
f. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Figure 3-7 Removing and Replacing the I/O Riser
|
|
||
|
|
The eUSB drive is a small solid-state flash drive card that connects to the motherboard in either of two eUSB drive connectors.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To install or replace the eUSB drive, follow these steps:
Step 1 Remove the eUSB drive you are replacing. See Figure 3-8:
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
d. Remove the I/O riser to provide clearance. See Replacing the I/O Riser.
e. Locate the eUSB drive and remove the single screw that secures it to the motherboard standoff.
f. Lift straight up on both ends the eUSB drive to disengage it from the motherboard connector.
Step 2 Install a new eUSB drive.
a. Align the connector on the underside of the eUSB drive with the empty drive connector on the motherboard and push on both ends of the drive evenly to seat the connector.
b. Replace the single mounting screw that secures the eUSB drive to the motherboard standoff.
c. Replace the I/O riser. See Replacing the I/O Riser.
e. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Figure 3-8 Removing and Replacing the eUSB Drive
|
|
The factory default is for all USB ports on the server to be enabled. However, the internal USB port can be enabled or disabled in the server BIOS. To enable or disable the internal USB port, follow these steps:
Step 1 Enter the BIOS Setup utility by pressing the F2 key when prompted during bootup.
Step 2 Navigate to the Advanced tab.
Step 3 On the Advanced tab, select USB Configuration.
Step 4 On the USB Configuration page, select USB Ports Configuration.
Step 5 Scroll to USB Port: Internal, press Enter, and then select either Enabled or Disabled from the pop-up menu.
Step 6 Press F10 to save and exit the utility.
The SAS riser is a RAID controller card that has a designated position and motherboard connector inside the server.
Firmware on the RAID controller must be verified for compatibility with the current Cisco IMC and BIOS versions that are installed on the server. If not compatible, upgrade or downgrade the RAID controller firmware accordingly using the Host Upgrade Utility (HUU) for your firmware release to bring it to a compatible level.
See the HUU guide for your Cisco IMC release for instructions on downloading and using the utility to bring server components to compatible levels: HUU Guides
Step 1 Remove the SAS riser you are replacing. See Figure 3-9:
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
d. Pinch and lift up the green retaining clip that secures the SAS riser to the chassis rear panel.
e. Lift the SAS riser straight up from the motherboard connector.
Note Lift up on both ends of the card evenly to avoid damaging its connector.
f. Disconnect the SAS cables and any battery backup unit (BBU) cable from the SAS riser. See Figure 3-10.
Tip Label the SAS cables when you disconnect them to aid correct connection to the new SAS riser.
Step 2 Install a new SAS riser:
a. Connect the SAS cables and BBU cable to the new SAS riser. See Figure 3-10.
b. Align the SAS riser with the empty SAS riser connector on the motherboard.
c. Push down evenly on both ends of the SAS riser until it is fully seated in the motherboard connector.
d. Ensure that the SAS riser rear panel sits flat against the server back panel opening.
e. Push the green retaining clip down until it locks over the top of the SAS riser.
g. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Step 3 Restore the RAID configuration on your drives to the new mass storage controller.
See Restoring RAID Configuration After Replacing a RAID Controller.
Figure 3-9 Removing and Replacing a SAS Riser
|
|
Figure 3-10 SAS Riser Card Connectors
|
|
Note This optional battery backup unit (BBU) is available only when using the optional LSI 9260-8i SAS RAID controller card as the SAS riser. This BBU provides approximately 72 hours of battery backup for the disk write-back cache DRAM in the case of sudden power loss.
Note LSI recommends that you replace the LSI BBU once per year or after 1,000 recharge cycles, whichever comes first. Verify whether BBU replacement is required by looking in the CIMC. Log in to CIMC for the server, then click Server—Inventory—Storage—Battery Backup Unit. If the Battery Replacement Required field says, “True,” then you must purchase a replacement BBU and replace it.
Warning There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.
Statement 1015
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace the SAS riser BBU, follow these steps:
Step 1 Remove the BBU that you are replacing. See Figure 3-11.
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
c. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.
d. Remove the memory riser that sits closest to the BBU. See Replacing Memory Risers.
e. Slide the battery assembly toward the front of the chassis to disengage its retaining clips from the chassis wall slots.
f. Disconnect the cable that is attached to the BBU.
Note When you install the cable from the LSIBBU06, LSIBBU07 or LSIBBU08 BBU (varying capacities) to the LSI 9260-8i RAID controller, make sure to align the red dot on the cable with the red dot on the connector that is on the card to ensure correct polarity.
a. Connect the cable from the SAS riser to the replacement BBU.
b. Insert the two retaining clips on the rear of the BBU into the chassis wall slots and slide the BBU toward the chassis rear until it locks into place.
c. Replace the memory riser that you removed for clearance. See Replacing Memory Risers.
e. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Figure 3-11 Removing and Replacing the RAID Battery Assembly
|
|
Warning There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.
Statement 1015
The CMOS battery retains system settings when the server is disconnected from power.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace or install the motherboard CMOS battery, follow these steps:
Step 1 Remove the CMOS battery. See Figure 3-12:
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
d. Remove the I/O riser to provide clearance. See Replacing the I/O Riser.
f. Bend the battery retaining clip away from the chassis wall and pull the battery from the socket.
Step 2 Install a CMOS battery:
a. Bend the retaining clip away from the chassis wall and insert the battery in the socket.
Note The positive side of the battery marked “+” should face the chassis wall.
b. Push the battery into the socket until it is fully seated.
Note Ensure that the retaining clip clicks over the top of the battery.
c. Replace the I/O riser. See Replacing the I/O Riser.
e. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Figure 3-12 Removing and Replacing the Motherboard CMOS Battery
|
|
This server can operate with 2-, 3-, or 4-CPU configurations. Each CPU supports two memory risers (four memory buffers) connected by serial memory interface (SMI). Each memory buffer has two DDR3 memory bus interfaces.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
Note The minimum CPU configuration is that CPU1 and CPU3 must be installed. See Figure 3-13, which has a view shown facing the front of the server. Only CPU1 and CPU3 are connected to the internal PCIe hub. With CPU1 and CPU3 installed, any other combination operates.
Figure 3-13 CPUs and Memory Risers
|
|
|
|
||||
---|---|---|---|---|---|---|---|
When a return material authorization (RMA) of the motherboard or CPU is done on a Cisco UCS C-series server, there are additional parts that might not be included with the CPU or motherboard spare bill of materials (BOM). The TAC engineer might need to add the additional parts to the RMA to help ensure a successful replacement.
– Heat sink cleaning kit (UCSX-HSCK=)
– Thermal interface pad for C460 (RC-460-TIM=)
– Heat sink cleaning kit (UCSX-HSCK=)
A CPU heatsink cleaning kit is good for up to four CPU and heatsink cleanings. The cleaning kit contains two bottles of solution, one to clean the CPU and heatsink of old thermal interface material and the other to prepare the surface of the heatsink.
New heatsink spares have preinstalled thermal interface material covered by a small sheet of plastic. It is important to clean the old thermal interface material off of the CPU prior to installing the heatsinks. Therefore, when ordering new heatsinks it is still necessary to order the heatsink cleaning kit at a minimum.
To install or replace a CPU heatsink and CPU, follow these steps:
Step 1 Remove the CPU and heatsink that you are replacing:
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
d. Remove all memory risers to uncover the CPU cage screws. See Replacing Memory Risers.
e. Remove all the memory riser dividers by lifting them straight up and sliding them free of the CPU cage. See Figure 3-14.
f. Remove the CPU cage by loosening the six captive screws that secure the cage to the motherboard. See Figure 3-14 for the screw locations.
g. Loosen the two captive screws that secure the heatsink and lift it off of the CPU. See Figure 3-14 and Figure 3-15.
Note Alternate loosening each screw evenly to avoid damaging the heatsink or CPU.
h. Unclip the CPU retaining lever and lift the CPU retaining lid. See Figure 3-15.
i. Lift the CPU out of the socket and set it aside on an antistatic mat or in an antistatic bag.
a. Insert the replacement CPU in the socket with the arrow on the CPU pointing toward the arrow on the socket.
b. Close the CPU retaining lid and clip down the CPU retaining lever.
a. Apply the supplied cleaning solution to the old thermal pad and let it soak for a least 15 seconds.
b. Wipe all of the old thermal pad off the heatsink using a soft cloth that will not scratch the heatsink surface.
c. Apply the supplied preparation solution to the center bottom of the heatsink, where the new thermal pad will be applied.
d. Apply the new thermal pad to the center bottom of the heatsink.
e. Align the heatsink captive screws with the motherboard standoffs, then tighten the captive screws evenly, until the screws stop against the captive springs.
Note Alternate tightening each screw evenly to avoid damaging the heatsink or CPU.
f. Replace the CPU cage. Align the six captive screws with the holes in the motherboard, then tighten each screw evenly.
g. Replace the memory riser dividers. Slide each one into the slots on the CPU cage and the chassis.
h. Replace all memory risers. See Replacing Memory Risers.
j. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Figure 3-14 Locations of CPU Cage Screws
|
CPU cage screw locations (six) on motherboard (cage not shown) |
|
|
|
|
||
|
|
Figure 3-15 Removing a CPU and Heatsink
|
|
||
|
|
||
|
|
The memory risers connect to the motherboard and each riser provides eight DIMM slots. The memory riser is hot-swappable when you use the Attention button, as described in the following procedure.
Figure 3-16 Memory Riser LEDs (Top View)
|
|
||
|
|
Mirror activity LED (indicates whether memory mirroring is enabled) |
|
|
DIMM fault LEDs 1 through 8 |
|
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace or install a hot-swappable memory riser, follow these steps:
Step 1 Remove the memory riser that you are replacing: See Figure 3-17:
a. Slide the server out the front of the rack far enough so that you can remove the top cover.
b. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
c. Press the attention button (ATTN BUTTON) on the top of the memory riser (see Figure 3-16).
d. Wait until the attention LED (ATTN) and the power LED turn off.
e. Simultaneously press both green release buttons on the top of the memory riser to release the riser retaining latches. The latches open up to a 45-degree angle when they are released.
f. Grasp the open retaining latches and lift the memory riser straight up and out of the motherboard connector.
g. If you are installing or replacing DIMMs on the memory riser, use the instructions in Replacing DIMMs.
Step 2 Install (hot-add) a new memory riser:
a. Ensure that the riser release latches are in the open position.
a. Align the riser with the empty motherboard connector.
b. Push the riser into the connector until it is seated and the open release levers engage.
c. Simultaneously press down on each release lever to put them in the locked position. This ensures that the riser is properly seated in the motherboard connector.
d. Press the attention button, then wait until the attention LED turns off.
f. Replace the server in the rack and replace any cables.
Figure 3-17 Removing and Replacing Memory Risers
|
|
This section includes the following sections:
Note To ensure the best server performance, it is important that you are familiar with memory performance guidelines and population rules before you install or replace memory modules.
This section describes the type of memory that the server requires and its effect on performance. The following topics are covered:
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
Each CPU supports two memory risers. Figure 3-18 shows the placement of the CPUs and their corresponding memory risers. The view shown is facing the front of the server. This numbering is also inscribed on the top of the CPU cage.
Figure 3-18 CPUs and Memory Risers
|
|
|
|
||||
---|---|---|---|---|---|---|---|
Each memory riser contains two memory buffers that are connected to the CPU by serial memory interface (SMI) channels. Each memory buffer has two channels, each containing a pair of DDR3 DRAM slots.
In Figure 3-19, the buffers and channels are:
Figure 3-19 DIMM Slots and Memory Buffers
|
|
Following are the DIMM population rules:
For example, the order that you should populate the four channels on a memory riser is as follows (see also Figure 3-19):
DIMM and rank sparing can be enabled in the BIOS configuration utility.
Sparing involves utilizing one of the DIMM pairs or rank pairs within each memory riser as a spare unit. When any of the other DIMM pairs within the same memory riser experiences errors beyond a pre-defined threshold, it fails over to the spare DIMM pair. Spared DIMMs and ranks are hidden from the user and the OS so that the BIOS can migrate to a spare unit when it finds degrading DIMMs.
When sparing is enabled, the available system memory is lesser than the total installed memory.
The memory riser has LEDs on it supper surface that can assist you in isolating a faulty DIMM. The faulty DIMMs are indicated by the DIMM fault LEDs, which light amber to indicate which DIMMs are faulty. See Figure 3-20.
Figure 3-20 Memory Riser LEDs (Top View)
|
|
||
|
|
Mirror activity LED (indicates whether memory mirroring is enabled) |
|
|
DIMM fault LEDs 1 through 8 |
|
To install a DIMM pair, follow these steps:
Note DIMM risers are hot-swappable when you use the attention button, so you do not have power off the server or disconnect power cords. Use the following procedure.
Step 1 Remove the DIMMs that you are replacing: See Figure 3-21:
a. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
b. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
c. Press the attention button (ATTN BUTTON) on the top of the memory riser that contains the faulty DIMM (see Figure 3-20).
d. Wait until the attention LED (ATTN) and the power LED turn off.
e. Simultaneously press both green release buttons on the top of the memory riser to release the riser retaining latches. The latches open up to a 45-degree angle when they are released.
f. Lift the memory riser straight up and out of the motherboard connector.
g. Locate the faulty DIMM and remove it from the connector on the memory riser.
a. Insert the DIMM assembly into the connector on the riser.
b. Push the DIMM into the connector until it is seated properly, and the white clips on either side of the connector lock into place.
c. Push the hot-addable memory riser into the motherboard connector until it is seated and the open release levers engage the chassis and the CPU cage.
d. Simultaneously press down on each release lever to put them in the closed position. This ensures that the riser is properly seated in the motherboard connector.
e. Press the attention button on the top of the memory riser, then wait until the attention LED turns off (see Figure 3-20).
g. Replace the server in the rack and replace any cables.
Figure 3-21 Removing and Replacing DIMMs
|
|
The eight fan modules in the server are numbered as follows when you are facing the front of the server. Each fan module has a fault LED that lights amber when the fan module fails.
Figure 3-22 Fan Module Numbering
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace or install a hot-pluggable fan module, follow these steps:
Step 1 Remove the fan module that you are replacing: See Figure 3-23:
a. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
b. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.
c. Insert your thumb and forefinger in the two green release latches on top of the fan module.
d. Squeeze the release latches together and lift out the fan module.
Step 2 Install a new fan module:
a. Grasp the fan module by the release latches and align it with the empty fan bay and the motherboard connector.
Note As you face the front of the server, the connector on underside of the fan module should be oriented on the right-bottom side of the fan module. See Figure 3-23.
b. Press down on the top corners of the fan module until the connector is fully seated and the release latches lock in place.
d. Replace the server in the rack.
Figure 3-23 Removing and Replacing Fan Modules
|
|
Fan module, front view showing connector on underside of the module |
Tip You do not have to shut down or power off the server to replace hard drives or SSDs because they are hot pluggable.
Note You can mix hard drives and solid state drives (SSDs) in the same server. However, You cannot configure a logical volume (virtual drive) that contains a mix of hard drives and SSDs. That is, when you create a logical volume, it must contain all hard drives or all SSDs.
Note Cisco recommends following the industry-standard practice of using drives of the same capacity when configuring RAID volumes. If you use drives of different capacities, the usable portion of the smallest drive will be used on all drives of the RAID volume.
Note The 500 GB SATA drives (A03-D500GC3) and the 1 TB SATA drives (A03-D1TBSATA) that are sold with the UCS C460 M2 server are supported at full 6G speeds.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace or install a hot-pluggable hard drive, follow these steps:
Step 1 Remove the drive that you are replacing or remove a blank panel from an empty bay (See Figure 3-25):
a. Press the release button on the face of the hard drive.
b. Grasp the ejector lever and pull the hard drive tray out of the slot.
c. If you are replacing an existing drive, remove the four drive tray screws that secure the drive to the tray and then lift the drive out of the tray.
a. Place a new hard drive in the empty drive tray and replace the four drive tray screws.
b. Insert the drive tray into the empty drive bay.
c. Push the tray into the slot until the drive connectors are fully seated in the backplane.
d. Press the ejector lever flat to lock the drive and tray in place.
Figure 3-24 Drive Numbering and Physical Orientation, Facing Server Front
Figure 3-25 Removing and Replacing Hard Drives
|
|
||
|
|
Warning Class 1 laser product.
Statement 1008
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To replace or install a DVD drive, follow these steps:
Step 1 Remove the DVD drive that you are replacing. See Figure 3-26:
a. Power off the server as described in Shutting Down and Powering Off the Server.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
c. Remove the top cover as described in Removing and Replacing the Server Top Cover.
d. Remove all memory risers. See Replacing Memory Risers.
e. Remove all memory riser dividers.
f. Remove the black plastic cover from the floor of the chassis by removing the four screws that secure it. This strip covers the width of the chassis floor behind the fan modules.
g. Push the release button on the rear of the DVD drive, and then push the DVD drive out the front panel.
Step 2 Install a new DVD drive:
a. Insert the rear of the new DVD drive into the empty DVD drive bay on the front panel.
b. Push the drive inward until it seats in its connector and the release button locks in place.
c. Replace the black strip to the floor of the chassis by replacing its four screws.
d. Replace all memory riser dividers.
g. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
Figure 3-26 Removing and Replacing the DVD Drive
|
|
Note If you are installing a Cisco UCS Virtual Interface Card, there are prerequisite considerations. See Special Considerations for Cisco UCS Virtual Interface Cards.
This server has 10 PCIe expansion slots. See Figure 3-27 and Table 3-4 for information about the slots and which slots are hot-swappable. The replacement procedures differ depending on whether the PCIe slot is hot-swappable or not. This section contains the following topics:
|
Lane Width |
|
|
|
|
|
---|---|---|---|---|---|---|
Yes4 |
||||||
1.This is the supported length because of internal clearance. 4.Slot 1 can operate when the server is in standby power mode. |
Note CPU3 must be installed to support PCIe slots 5, 6, 7, 9, and 10.
Note Legacy I/O devices like video cards are only supported on slots 1, 2, 3, 4 and 8.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To install or replace a PCIe card in slots 3, 4, 5, 8, 9, or 10, follow these steps:
a. Shut down and power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
c. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.
d. Remove any cables from the ports of the PCIe card that you are replacing.
Tip Label the cables when you disconnect them to aid correct connection to the new card.
e. Pinch and lift up the green retaining clip that secures the card to the chassis rear panel.
f. Lift the card straight up from the motherboard connector.
Note Lift up on both ends of the card evenly to avoid damaging its connector.
a. Align the PCIe card with the empty PCIe connector on the motherboard.
b. Push down evenly on both ends of the card until it is fully seated in the motherboard connector.
c. Ensure that the card rear panel sits flat against the server back panel opening.
d. Push the green retaining clip down until it locks over the top of the card.
f. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.
The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :
http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html
To install or replace a PCIe card in slots 1, 2, 6, or 7, follow these steps:
Step 1 Remove a hot-swappable PCIe card:
a. Slide the server out the front of the rack far enough so that you can remove the top cover.
Note You might have to detach cables from the rear panel to provide clearance.
b. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.
c. Remove any cables from the ports of the PCIe card that you are replacing.
Tip Label the cables when you disconnect them to aid correct connection to the new card.
d. Press the lightpipe switch that is on the top of the plastic divider for the hot-swappable PCIe slot.
Note Wait for the lightpipe switch LED to turn off before removing the card in the next step.
e. Pinch and lift up the green retaining clip that secures the card to the chassis rear panel.
f. Lift the card straight up from the motherboard connector.
Note Lift up on both ends of the card evenly to avoid damaging its connector.
Step 2 Install a hot-swappable PCIe card:
a. Align the PCIe card with the empty PCIe connector on the motherboard.
b. Push down evenly on both ends of the card until it is fully seated in the motherboard connector.
c. Ensure that the card rear panel sits flat against the server back panel opening.
d. Push the green retaining clip down until it locks over the top of the card.
e. Press the lightpipe switch that is on top of the plastic divider for the hot-swappable PCIe slot. Wait for the LED to turn on to ensure that the PCIe slot receives power.
g. Replace the server in the rack and replace cables.
Table 3-5 describes the requirements for the supported Cisco UCS virtual interface cards (VICs).
|
|
|
|
|
|
|
---|---|---|---|---|---|---|
|
||||||
|
||||||
|
PCIE 16 |
5.See PCIe Slots. 6.The Cisco UCS VIC1225T is not supported for UCS integration at this time. |
Table 3-6 describes the requirements for the supported Cisco UCS Fusion ioDrive2 cards.
|
|
|
|
(rear-panel tab) |
---|---|---|---|---|
|
All8 |
|||
|
Half height9 |
|||
|
||||
|
7.See PCIe Slots. 9.A rear-panel tab adapter is required to fit the half-height cards in the full-height slots. |
Note PCIe slots 8 and 9 are Gen1 x4 electrical lane width slots and provide limited bandwidth if used with these cards.
When a large number of PCIe add-on cards are installed in the server, the system may run out of the following resources required for PCIe devices:
The topics in this section provide guidelines for resolving the issues related to these limited resources.
The system has very limited memory to execute PCIe legacy option ROMs, so when a large number of PCIe add-on cards are installed in the server, the system BIOS might not able to execute all of the option ROMs. The system BIOS loads and executes the option ROMs in the order that the PCIe cards are enumerated (Slot 1, Slot 2, Slot 3, etc.).
If the system BIOS does not have sufficient memory space to load any PCIe option ROM, it skips loading that option ROM, reports a system event log (SEL) event to the CIMC controller and reports the following error in the Error Manager page of the BIOS Setup utility:
To resolve this issue, disable the Option ROMs that are not needed for system booting. The BIOS Setup Utility provides the setup options to enable or disable the Option ROMs at the PCIe slot level for the PCIe expansion slots and at the port level for the onboard NICs. These options can be found in the BIOS Setup Utility Advanced → PCI Configuration page (see Figure 3-28).
Figure 3-28 BIOS Setup Utility Advanced > PCI Configuration Page
If the RAID controller does not appear in the system boot order, even if the option ROMs for the RAID controller slots are enabled, the RAID controller option ROM might not have sufficient memory space to execute. In that case, disable other option ROMs that are not needed for the system configuration to free up some memory space for the RAID controller option ROM.
If the system is configured to primarily perform PXE boot from onboard NICs, make sure that the option ROMs for the onboard NICs to be booted from are enabled in the BIOS Setup Utility. Disable other option ROMs that are not needed to create sufficient memory space for the onboard NICs.
The system has only 64 KB of legacy 16-bit I/O resources available. This 64 KB of I/O space is divided between two I/O Hubs (IOH0 and IOH1) in the system.
The default BIOS setting is to allocate 40 KB to IOH0 and 24 KB to IOH1. Every PCIe device that is connected to IOH0 has to share the 40 KB of I/O. Every PCIe device connected to IOH1 must share that 24 KB of I/O.
When a large number of PCIe cards are installed in the system, the system BIOS might not have sufficient I/O space for some PCIe devices. If the system BIOS is not able to allocate the required I/O resources for any PCIe devices, the following symptoms have been observed:
To work around this problem, rebalance the load on each IOH using the following methods:
1. Adjust the IOH IO Resource Allocation ratio. This option is included in the BIOS Setup Utility Advanced → PCI Configuration page (see Figure 3-28).
2. Physically remove any unused PCIe cards.
3. Physically rearrange the PCIe cards to create a balance between IOH0 and IOH1.
In this server, the PCIe slots are connected to IOH0 and IOH1 as follows:
Note The NVIDIA Tesla C2050 GPU card is supported in Generation M2 servers only. This card is supported only in PCIe slot 5 of the server (see Figure 3-27). When installed in PCIe slot 5, this double-wide card physically covers two slots, slot 5 and slot 6.
Note The DVI connector on the rear panel of the GPU card is disabled. Connect your monitor to one of the VGA connectors on the server instead.
Cisco ships an accessory kit (UCSC-GPU_N01-C460) that includes two adapters that you must use to customize the card for installation in the Cisco C460 M2 server. Each kit includes the following components (see Figure 3-29):
Figure 3-29 Cisco Adapters For NVIDIA Tesla C2050 GPU Card
|
|
||
|
|
To install or replace an NVIDIA Tesla C2050 GPU card, follow these steps:
Step 1 Prepare the server for installation of the card:
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Disconnect all power cords from the power supplies.
c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.
d. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.
e. Open the two green PCIe retainer latches on PCIe slots 5 and 6 (see Figure 3-30).
f. Remove any PCIe cards or blanking panels from PCIe slots 5 and 6.
g. Remove the plastic divider assembly that is over PCIe slots 6 and 7 to make clearance for the card.
Pull up on the pin that attaches this divider assembly to the motherboard, then lift the assembly out of the server. This pin is on the front end of the divider assembly where it touches the motherboard (see Figure 3-30).
Figure 3-30 Cisco C2050 GPU Card Slots
|
|
||
|
|
Step 2 Install adapters on the GPU card to prepare for installation:
a. Remove the default rear-panel faceplate from the rear end of a new card. Remove the two securing screws and set them aside.
b. Install the new rear-panel faceplate to the rear end of the card (see Figure 3-31). Install the two securing screws that you removed in the prior step.
c. Install the front bracket to the front end of the card (see Figure 3-31). Insert the pins on the bracket into the corresponding holes in the card and then tighten the single securing thumbscrew.
d. Connect the power cable adapter to the card. Insert the six-pin connector on the cable into the six-pin connector on the front end of the card.
Figure 3-31 Assembling Cisco C2050 GPU Card Components
|
|
||
|
|
Step 3 Install a new GPU card to the server:
a. Align the card so that its bottom-edge connector is over PCIe slot 5.
b. Tilt the front end of the card downward as you lower it into the chassis so that the attached power cable goes under the CPU cage (behind CPU3). The two thumbscrews on the front bracket that you installed must align with the two threaded holes in the CPU cage (see Figure 3-32).
c. Perform the following two actions simultaneously:
– Insert the two tabs on the rear-panel faceplate of the card into the two chassis rear-panel openings for slots 5 and 6.
– Carefully push down on both ends of the card to seat its connector in the PCIe slot 5 motherboard connector.
d. Close the two green PCIe retainer latches for slots 5 and 6 over the top of the rear-panel faceplate.
e. Tighten the two thumbscrews on the front bracket of the card to secure it to the CPU cage.
Step 4 Install the power cable adapter:
a. Disconnect the PCIe power cable from the motherboard. This is 10-pin connector J166 (see Figure 3-32).
b. Connect the male 10-pin connector of the power cable adapter to the motherboard connector J166.
c. Connect the PCIe power cable into the female 10-pin connector on the power cable adapter, which you attached to the card in Step 2.
Step 6 Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.
Figure 3-32 Installing and Cabling the GPU Card
|
|
||
|
|
The trusted platform module (TPM) is a small circuit board that connects to a motherboard socket and is secured by a one-way screw.
Note For security purposes, the TPM is installed with a one-way screw. It cannot be removed with a standard screwdriver.
Step 1 Verify that the Server has a valid UUID. This can be seen from the Cisco IMC Server Summary page.
a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.
b. Disconnect all power cords from the power supplies.
c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.
d. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.
Step 3 Verify that the jumper that disables onboard TPM is in place before you begin to install the add-on TPM module (see Figure 3-33 for the location).
Note The default from the factory is that this two-pin jumper comes installed and onboard TPM is disabled. The jumper must be in place before you can enable an add-on TPM module.
Step 4 Install a TPM (see Figure 3-33 for the header location):
a. Align the connector that is on the underside of the TPM with the TPM header pins on the motherboard, and then press firmly on both ends of the TPM to seat it.
b. Install the securing screw that holds the TPM to the motherboard.
d. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.
Step 5 Continue with Enabling TPM Hardware Support and the TXT Feature in the BIOS.
|
|
||
|
|
Step 1 When the server boots, press F2 when prompted to enter the BIOS menu.
Step 2 Enter the administrator password or set up one (Security > Set Administrator Password).
Step 3 Enable TPM (Security > TPM Administrative Control: Turn ON)
Step 4 Enable TXT (Advanced > Processor Configuration > Intel Trusted Execution Technology: Enabled).
Step 5 Save and Reboot (press F10).
Step 6 Enter the BIOS menu Clear TPM (Security > TPM Administrative Control: Clear Ownership).
Step 7 Save and Reboot (press F10).
Step 8 Enter the BIOS menu Enable TPM (Security > TPM Administrative Control: Turn ON).
Step 9 Save and Reboot (press F10).
Note It is required that you perform the three reboots. At this point the TPM is ready to be used by ESXi, which takes ownership during the first boot. If you want to boot another OS on the same server, you must repeat the process above to clear the ownership of the TPM before booting that new OS.