Cisco UCS C460 Server Installation and Service Guide
Maintaining the Server
Downloads: This chapterpdf (PDF - 9.56MB) The complete bookPDF (PDF - 12.7MB) | The complete bookePub (ePub - 1.95MB) | Feedback

Table of Contents

Maintaining the Server

Server Monitoring and Management Tools

Cisco Integrated Management Interface (CIMC)

Server Configuration Utility

Status LEDs and Buttons

Front Panel LEDs

Operations Panel LEDs and Buttons

Rear Panel LEDs and Buttons

Preparing for Server Component Installation

Required Equipment

Shutting Down and Powering Off the Server

Removing and Replacing the Server Top Cover

Replaceable Component Locations

Installing or Replacing Server Components

Replacing Power Supplies

Replacement Procedure

Power Supply Cold Redundancy

Replacing the I/O Riser

Replacing an Internal Flash eUSB Drive

Replacement Procedure

Enabling or Disabling the Internal USB Port

Replacing a SAS Riser (RAID Controller)

Replacing the SAS Riser Battery Backup Unit

Replacing the Motherboard CMOS Battery

Replacing CPUs and Heatsinks

Additional CPU-Related Parts To Order With RMA Replacement Motherboards

CPU Replacement Procedure

Replacing Memory Risers

Replacing DIMMs

Memory Performance Guidelines and Population Rules

DIMM Installation Procedure

Replacing Fan Modules

Replacing Hard Drives or Solid State Drives

Replacing a DVD Drive

Replacing a PCIe Card

PCIe Slots

Replacing a PCIe Card in a Non Hot-Swappable Slot

Replacing a PCIe Card in a Hot-Swappable Slot

Special Considerations for Cisco UCS Virtual Interface Cards

Special Considerations for Cisco UCS Fusion ioDrive2 Storage Accelerator Cards

Installing Multiple PCIe Cards and Resolving Limited Resources

Installing an NVIDIA Tesla C2050 GPU Card

Replacing a Trusted Platform Module (TPM)

Enabling the Intel Trusted Execution Technology (TXT) Feature For the TPM

Maintaining the Server

This chapter describes how to diagnose server system problems using LEDs. It also provides information about how to install or replace hardware components, and it includes the following sections:

Server Monitoring and Management Tools

Cisco Integrated Management Interface (CIMC)

You can monitor the server inventory, health, and system event logs by using the built-in Cisco Integrated Management Controller (CIMC) GUI or CLI interfaces. See the user documentation for your firmware release at the following URL:

http://www.cisco.com/en/US/products/ps10739/products_installation_and_configuration_guides_list.html

Server Configuration Utility

Cisco has also developed the Cisco Server Configuration Utility for C-Series servers, which can aid and simplify the following tasks:

  • Monitoring server inventory and health
  • Diagnosing common server problems with diagnostic tools and logs
  • Setting the BIOS booting order
  • Configuring some RAID configurations
  • Installing operating systems

This utility is shipped with new servers on CD. You can also download the ISO from Cisco.com. See the user documentation for this utility at the following URL:

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/ucsscu/user/guide/20/SCUUG20.html

Status LEDs and Buttons

This section describes the location and meaning of LEDs and buttons, and it includes the following topics:

Front Panel LEDs

Figure 3-1 shows the front panel LEDs.

Figure 3-1 Front Panel LEDs

 

1

Hard drive activity LED

2

DVD drive activity LED

3

Hard drive fault LED

4

Power supply fault LED

5

Memory fault LED

6

CPU fault LED

7

Network activity LED

8

Operations panel (see Operations Panel LEDs and Buttons).

 

Table 3-1 Front Panel LEDs

LED Name
State

Hard drive activity

  • Off—There is no hard drive in the hard drive sled (no access, no fault).
  • Green—The hard drive is ready.
  • Green, blinking—The hard drive is reading or writing data.

There are activity LED differences between SATA and SAS hard drives. For SATA drives, the LED stays off during the reading or writing activity.

  • Amber and steady—The hard drive is in fault.

The LED will also turn amber and steady when the locate hard drive function is used.

  • Amber and blinking—The hard drive is in predictive failure.

DVD drive activity

  • Green and steady—The drive is not accessing data.
  • Green and blinking—The drive is accessing data.

Hard drive fault

  • Off—The hard drive is operating properly.
  • Amber—This hard drive has failed.
  • Amber, blinking—The device is rebuilding.

Power supply fault

  • Green—All power supplies are operating properly.
  • Off—One or more power supplies are not operating properly.

Memory fault

  • Green—All DIMMs are operating properly.
  • Off—One or more DIMMs are not operating properly.

CPU fault

  • Green—All CPUs are operating properly.
  • Off—One or more CPUs are not operating properly.

Network activity

  • Off—The Ethernet link is idle.
  • Green and blinking —The Ethernet link is active.
  • Green and steady—The Ethernet link is detected but not active.

The blink rate gets faster as network activity increases.

Operations panel LEDs

See Operations Panel LEDs and Buttons.

Operations Panel LEDs and Buttons

Figure 3-2 shows the operations panel LEDs and buttons.

Figure 3-2 Operations Panel LEDs and Buttons

 

 

1

ID LED

2

ID button

3

Hard drive fault LED

4

System health LED

5

Fan fault LED

6

Power status LED

7

Power button

8

Reset button

9

NMI button

 

 

 

Table 3-2 Operations Panel LEDs

LED Name
State

ID

  • Off—The ID LED is not in use.
  • Blue and blinking—System ID is active via the remote ID button.
  • Blue and steady—System ID is active via the local ID button.

Hard drive fault

  • Green—No hard drives have a fault.
  • Amber—At least one hard drive has failed.

System health

  • Green—The system is not in fault.
  • Amber and steady—The system is in moderate fault.
  • Amber and blinking—The system is in severe fault.

Fan fault

  • Off—All fan modules are operating properly.
  • Amber—At least one fan module has a moderate fault.
  • Amber, blinking—At least one fan module has a severe fault.

Power status

  • Off—The server is in standby power mode or no power is present.
  • Green—The server is in main power mode.

Rear Panel LEDs and Buttons

Figure 3-3 shows the rear panel LEDs and buttons.

Figure 3-3 Rear Panel LEDs and Buttons

 

 

1

System health LED

2

ID LED

3

ID button

4

Power supply status LED

5

Power supply fault LED

6

Power supply AC input LED

 

Table 3-3 Operations Panel LEDs

LED Name
State

System health

  • Green—The system is not in fault.
  • Amber and steady—The system is in moderate fault.
  • Amber and blinking—The system is in severe fault.

ID

  • Off—The ID LED is not in use.
  • Blue and blinking—System ID is active via the remote ID button.
  • Blue and steady—System ID is active via the local ID button.

Power supply status LED

  • Green and steady—The server is in main power mode.
  • Green and blinking—The power supply is off and it is in cold redundancy mode.

Power supply fault LED

  • Off—The power supply is operating properly.
  • Amber and blinking—The power supply is warning of an event, but continues to operate.
  • Amber and steady—The power supply is in critical fault, causing a shut down.

Power supply AC input LED

  • Green and steady—The AC power cord is plugged in and the power is present.
  • Green and blinking—The AC power cord is not plugged in.

Preparing for Server Component Installation

This section describes how to prepare for component installation, and it includes the following topics:

Required Equipment

The following equipment is used to perform the procedures in this chapter:

  • Number 2 Phillips-head screwdriver
  • Number 1 Phillips-head screwdriver
  • Needle-nose pliers
  • Electrostatic discharge (ESD) strap or other grounding equipment such as a grounded mat

Shutting Down and Powering Off the Server

The server can run in two power modes:

  • Main power mode—Power is supplied to all server components and any operating system on your hard drives can run.
  • Standby power mode—Power is supplied only to the service processor and the cooling fans and it is safe to power off the server from this mode.

You can invoke a graceful shutdown or an emergency shutdown (hard shutdown) by using either of the following methods:

  • Use the CIMC management interface.
  • Use the Power button on the server front panel. To use the Power button, follow these steps:

Step 1 Check the color of the Power Status LED (see the “Operations Panel LEDs and Buttons” section).

  • Green indicates that the server is in main power mode and must be shut down before it can be safely powered off. Go to Step 2.
  • Off indicates that the server is already powered off or is in standby mode and can be safely powered off. Go to Step 3.

Step 2 Invoke either a graceful shutdown or a hard shutdown:


Caution To avoid data loss or damage to your operating system, you should always invoke a graceful shutdown of the operating system.

  • Graceful shutdown—Press and release the Power button. The operating system will perform a graceful shutdown and the server goes to standby mode, which is indicated by the Power Status LED being off.
  • Emergency shutdown—Press and hold the Power button for 4 seconds to force the main power off and immediately enter standby mode.

Step 3 Disconnect the power cords from the power supplies in your server to completely power off the server.


 

Removing and Replacing the Server Top Cover

Use the following procedure to remove or replace the top cover of the server:


Tip You do not have to remove the cover to replace hard drives or power supplies.



Step 1 Remove the top cover:

a. Simultaneously press the two green release buttons. See Figure 3-4.

b. Push the cover toward the server rear about three inches, until it stops.

c. Lift the cover straight up from the server and set it aside.

Step 2 Replace the top cover:

a. Place the cover on top of the server about 3 inches behind the front of the chassis. The cover should sit flat when the cover flanges are sitting in the grooves in the chassis.

b. Slide the cover toward the front of the server until it stops at the front panel and the green buttons lock.

Figure 3-4 Removing the Top Cover

 

1

Green release buttons

2

Rear of the server


 

Replaceable Component Locations

This section shows the locations of the components that are discussed in this chapter. The view in Figure 3-5 is from the top down, with the top cover and internal CPU cage removed.

Figure 3-5 Replaceable Component Locations

 

1

Power supplies, up to 4 (accessed through the rear panel).

8

Memory risers, which provide slots for up to 8 DIMMs on each riser.

2

I/O riser module.

9

RAID battery backup unit (optional when using the LSI 9260 controller)

3

eUSB connectors (2 on motherboard).

10

Fan modules (up to 8)

4

SAS riser (a dedicated slot for the RAID controller card)

11

Hard drives (up to 12, accessed through the front panel)

5

Trusted Platform Module (TPM) header

12

DVD drive (accessed through the front panel)

6

CMOS battery

13

PCIe connector 10 (10 of 10)
See also Figure 3-27 for all slot locations and details.

7

CPUs and heat sinks (up to 4, shown without CPU cage)

 

 

Installing or Replacing Server Components


Warning Blank faceplates and cover panels serve three important functions: they prevent exposure to hazardous voltages and currents inside the chassis; they contain electromagnetic interference (EMI) that might disrupt other equipment; and they direct the flow of cooling air through the chassis. Do not operate the system unless all cards, faceplates, front covers, and rear covers are in place.
Statement 1029



Warning Class 1 laser product.
Statement 1008



Caution When handling server components, wear an ESD strap to avoid damage.


Tip You can press the ID button on the front panel or rear panel to turn on a flashing ID LED on the front and rear panels of the server. This allows you to locate the specific server that you are servicing when you go to the opposite side of the rack. See the “Status LEDs and Buttons” section for locations of the LEDs.


This section describes how to install and replace server components, and it includes the following topics:

Replacing Power Supplies

The server can have two or four power supplies. Four power supplies are required for four-CPU configurations.

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

This section contains the following topics:

Replacement Procedure

To replace or install a power supply, follow these steps:


Note If you have ordered a server with power supply redundancy (at least four power supplies), you do not have to power off the server to replace power supplies. Also see the note below about 1+1 redundancy.



Step 1 Remove the power supply that you are replacing or a blank panel from an empty bay. See Figure 3-6:

a. Do one of the following actions:

If your server has only two power supplies, shut down and power off the server as described in the “Shutting Down and Powering Off the Server” section. See the following note.


Note You can use 2 power supplies with a 2-CPU configuration. In this case, 1+1 redundancy is supported only if your server’s overall power consumption can be supported by a single 850W power supply during the failure and hot-swap. For more information about your server’s power consumption, consult with your Cisco sales representative or use the power calculator accessible at the Unified Computing System Partner Resource Center:


http://www.ciscoprc.com/resourcelib.asp?id=937

If your server has four power supplies, you do not have to shut down the server.

b. Remove the power cord from the power supply that you are replacing.

a. Grasp the power supply handle while squeezing the release lever towards the handle.

b. Pull the power supply out of the slot.

Step 2 Install a new power supply:

a. Grasp the power supply handle and insert the power supply into the power supply bay.

b. Push the power supply into the bay until the release lever locks.

c. Replace the power cord to the new power supply.

d. Press the Power button to return the server to main power mode.

Figure 3-6 Removing and Replacing Power Supplies

 

1

Power supply handle

2

Power supply release lever


 

Power Supply Cold Redundancy

Depending on the power being drawn by the server, one or more power supplies might actively provide all power to the system while the remaining power supplies are put into a standby state. This is known as cold redundancy.

The server’s CIMC interface reports the power supply redundancy based on the number of supplies that are plugged in with AC power applied. For example:

  • 2 PSUs: Redundancy lost
  • 3 PSUs: Redundancy degraded
  • 4 PSUs: Full redundancy

With cold redundancy, depending on the server’s power usage, one or more supplies might actively provide all power to the system while the remaining supplies are put into a standby state. For example, if you have two supplies connected to AC power, but the power consumption can be satisfied by power supply 1, then power supply 2 is put into a standby state.

Replacing the I/O Riser

The I/O riser is module that connects to the motherboard and that provides the ports for rear panel connectivity. To install or replace the I/O riser, follow these steps:


Step 1 Remove the I/O riser you are replacing. See Figure 3-7:

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Disconnect all cables from the ports on the I/O riser.


Tip Label the cables when you remove them to aid in identifying them for replacement.


c. Slide the server out the front of the rack far enough so that you can remove the top cover.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

e. Pinch and lift up the green retaining clip that secures the I/O riser to the chassis rear panel.

f. Lift the riser straight up from the motherboard connector and out of the chassis.

Step 2 Install a new I/O riser:

a. Align the I/O riser with the empty I/O riser connector on the motherboard.

b. Push down evenly on both ends of the I/O riser until it is fully seated in the motherboard connector.

c. Ensure that the I/O riser rear panel sits flat against the chassis back panel opening.

d. Push the green retaining clip down until it locks over the top of the I/O riser.

e. Replace the top cover.

f. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Figure 3-7 Removing and Replacing the I/O Riser

 

1

Retaining clip

2

I/O riser back panel

3

I/O riser

 

 


 

Replacing an Internal Flash eUSB Drive

The eUSB drive is a small solid-state flash drive card that connects to the motherboard in either of two eUSB drive connectors.

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

This section contains the following topics:

Replacement Procedure

To install or replace the eUSB drive, follow these steps:


Step 1 Remove the eUSB drive you are replacing. See Figure 3-8:

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

d. Remove the I/O riser to provide clearance. See Replacing the I/O Riser.

e. Locate the eUSB drive and remove the single screw that secures it to the motherboard standoff.

f. Lift straight up on both ends the eUSB drive to disengage it from the motherboard connector.

Step 2 Install a new eUSB drive.

a. Align the connector on the underside of the eUSB drive with the empty drive connector on the motherboard and push on both ends of the drive evenly to seat the connector.

b. Replace the single mounting screw that secures the eUSB drive to the motherboard standoff.

c. Replace the I/O riser. See Replacing the I/O Riser.

d. Replace the top cover.

e. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Figure 3-8 Removing and Replacing the eUSB Drive

 

1

eUSB connectors on motherboard (two)

2

eUSB drive, showing orientation of hole for securing screw


 

Enabling or Disabling the Internal USB Port

The factory default is for all USB ports on the server to be enabled. However, the internal USB port can be enabled or disabled in the server BIOS. To enable or disable the internal USB port, follow these steps:


Step 1 Enter the BIOS Setup utility by pressing the F2 key when prompted during bootup.

Step 2 Navigate to the Advanced tab.

Step 3 On the Advanced tab, select USB Configuration.

Step 4 On the USB Configuration page, select USB Ports Configuration.

Step 5 Scroll to USB Port: Internal, press Enter, and then select either Enabled or Disabled from the pop-up menu.

Step 6 Press F10 to save and exit the utility.


 

Replacing a SAS Riser (RAID Controller)

The SAS riser is a RAID controller card that has a designated position and motherboard connector inside the server. To install or replace a SAS riser, follow these steps:


Step 1 Remove the SAS riser you are replacing. See Figure 3-9:

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

d. Pinch and lift up the green retaining clip that secures the SAS riser to the chassis rear panel.

e. Lift the SAS riser straight up from the motherboard connector.


Note Lift up on both ends of the card evenly to avoid damaging its connector.


f. Disconnect the SAS cables and any battery backup unit (BBU) cable from the SAS riser. See Figure 3-10.


Tip Label the SAS cables when you disconnect them to aid correct connection to the new SAS riser.


Step 2 Install a new SAS riser:

a. Connect the SAS cables and BBU cable to the new SAS riser. See Figure 3-10.

b. Align the SAS riser with the empty SAS riser connector on the motherboard.

c. Push down evenly on both ends of the SAS riser until it is fully seated in the motherboard connector.

d. Ensure that the SAS riser rear panel sits flat against the server back panel opening.

e. Push the green retaining clip down until it locks over the top of the SAS riser.

f. Replace the top cover.

g. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Step 3 Restore the RAID configuration on your drives to the new mass storage controller.

See Restoring RAID Configuration After Replacing a RAID Controller.

Figure 3-9 Removing and Replacing a SAS Riser

 

1

Retaining clip

2

SAS riser

Figure 3-10 SAS Riser Card Connectors

 

1

SAS cable connectors
JT6 = Ports 0 - 3
JT7 = Ports 4 - 7

2

LSI BBU connector JT3
(on LSI 9260-8i only)


 

Replacing the SAS Riser Battery Backup Unit


Note This optional battery backup unit (BBU) is available only when using the optional LSI 9260-8i SAS RAID controller card as the SAS riser. This BBU provides approximately 72 hours of battery backup for the disk write-back cache DRAM in the case of sudden power loss.



Note LSI recommends that you replace the LSI BBU once per year or after 1,000 recharge cycles, whichever comes first. Verify whether BBU replacement is required by looking in the CIMC. Log in to CIMC for the server, then click Server—Inventory—Storage—Battery Backup Unit. If the Battery Replacement Required field says, “True,” then you must purchase a replacement BBU and replace it.



Warning There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.
Statement 1015


The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace the SAS riser BBU, follow these steps:


Step 1 Remove the BBU that you are replacing. See Figure 3-11.

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.

d. Remove the memory riser that sits closest to the BBU. See Replacing Memory Risers.

e. Slide the battery assembly toward the front of the chassis to disengage its retaining clips from the chassis wall slots.

f. Disconnect the cable that is attached to the BBU.

Step 2 Install a new BBU:


Note When you install the cable from the LSIBBU06, LSIBBU07 or LSIBBU08 BBU (varying capacities) to the LSI 9260-8i RAID controller, make sure to align the red dot on the cable with the red dot on the connector that is on the card to ensure correct polarity.


a. Connect the cable from the SAS riser to the replacement BBU.

b. Insert the two retaining clips on the rear of the BBU into the chassis wall slots and slide the BBU toward the chassis rear until it locks into place.

c. Replace the memory riser that you removed for clearance. See Replacing Memory Risers.

d. Replace the top cover.

e. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Figure 3-11 Removing and Replacing the RAID Battery Assembly

 

1

BBU position on inside chassis wall

2

BBU retaining clips

Replacing the Motherboard CMOS Battery


Warning There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.
Statement 1015


The CMOS battery retains system settings when the server is disconnected from power.

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install the motherboard CMOS battery, follow these steps:


Step 1 Remove the CMOS battery. See Figure 3-12:

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

d. Remove the I/O riser to provide clearance. See Replacing the I/O Riser.

e. Locate the CMOS battery.

f. Bend the battery retaining clip away from the chassis wall and pull the battery from the socket.

Step 2 Install a CMOS battery:

a. Bend the retaining clip away from the chassis wall and insert the battery in the socket.


Note The positive side of the battery marked “+” should face the chassis wall.


b. Push the battery into the socket until it is fully seated.


Note Ensure that the retaining clip clicks over the top of the battery.


c. Replace the I/O riser. See Replacing the I/O Riser.

d. Replace the top cover.

e. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Figure 3-12 Removing and Replacing the Motherboard CMOS Battery

 

1

CMOS battery socket

2

Retaining clip


 

Replacing CPUs and Heatsinks

This server can operate with 2-, 3-, or 4-CPU configurations. Each CPU supports two memory risers (four memory buffers) connected by serial memory interface (SMI). Each memory buffer has two DDR3 memory bus interfaces.

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html


Note The minimum CPU configuration is that CPU1 and CPU3 must be installed. See Figure 3-13, which has a view shown facing the front of the server. Only CPU1 and CPU3 are connected to the internal PCIe hub. With CPU1 and CPU3 installed, any other combination operates.


Figure 3-13 CPUs and Memory Risers

 

CPU 4
CPU 3
CPU 2
CPU 1

MEM 8

MEM 7

MEM 6

MEM 5

MEM 4

MEM 3

MEM 2

MEM 1

This section contains the following topics:

Additional CPU-Related Parts To Order With RMA Replacement Motherboards

When a return material authorization (RMA) of the motherboard or CPU is done on a Cisco UCS C-series server, there are additional parts that might not be included with the CPU or motherboard spare bill of materials (BOM). The TAC engineer might need to add the additional parts to the RMA to help ensure a successful replacement.

  • Scenario 1—You are re-using the existing heatsinks:

Heat sink cleaning kit (UCSX-HSCK=)

Thermal interface pad for C460 (RC-460-TIM=)

  • Scenario 2—You are replacing the existing heatsinks:

Heat sink (RC460-BHTS1=)

Heat sink cleaning kit (UCSX-HSCK=)

A CPU heatsink cleaning kit is good for up to four CPU and heatsink cleanings. The cleaning kit contains two bottles of solution, one to clean the CPU and heatsink of old thermal interface material and the other to prepare the surface of the heatsink.

New heatsink spares have preinstalled thermal interface material covered by a small sheet of plastic. It is important to clean the old thermal interface material off of the CPU prior to installing the heatsinks. Therefore, when ordering new heatsinks it is still necessary to order the heatsink cleaning kit at a minimum.

CPU Replacement Procedure

To install or replace a CPU heatsink and CPU, follow these steps:


Step 1 Remove the CPU and heatsink that you are replacing:

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

d. Remove all memory risers to uncover the CPU cage screws. See Replacing Memory Risers.

e. Remove all the memory riser dividers by lifting them straight up and sliding them free of the CPU cage. See Figure 3-14.

f. Remove the CPU cage by loosening the six captive screws that secure the cage to the motherboard. See Figure 3-14 for the screw locations.

g. Loosen the two captive screws that secure the heatsink and lift it off of the CPU. See Figure 3-14 and Figure 3-15.


Note Alternate loosening each screw evenly to avoid damaging the heatsink or CPU.


h. Unclip the CPU retaining lever and lift the CPU retaining lid. See Figure 3-15.

i. Lift the CPU out of the socket and set it aside on an antistatic mat or in an antistatic bag.

Step 2 Install a new CPU:

a. Insert the replacement CPU in the socket with the arrow on the CPU pointing toward the arrow on the socket.

b. Close the CPU retaining lid and clip down the CPU retaining lever.

Step 3 Install a heatsink:


Caution The heatsink must have a new, undamaged thermal pad on the heatsink-to-CPU surface to ensure proper cooling. If you are replacing a heatsink that was previously installed, you must remove the old thermal pad. If you are installing a new heatsink, skip to step d. below.

a. Apply the supplied cleaning solution to the old thermal pad and let it soak for a least 15 seconds.

b. Wipe all of the old thermal pad off the heatsink using a soft cloth that will not scratch the heatsink surface.


Caution Damage to the heatsink surface can damage the heat transferring properties of the heatsink.

c. Apply the supplied preparation solution to the center bottom of the heatsink, where the new thermal pad will be applied.

d. Apply the new thermal pad to the center bottom of the heatsink.

e. Align the heatsink captive screws with the motherboard standoffs, then tighten the captive screws evenly, until the screws stop against the captive springs.


Note Alternate tightening each screw evenly to avoid damaging the heatsink or CPU.


f. Replace the CPU cage. Align the six captive screws with the holes in the motherboard, then tighten each screw evenly.

g. Replace the memory riser dividers. Slide each one into the slots on the CPU cage and the chassis.

h. Replace all memory risers. See Replacing Memory Risers.

i. Replace the top cover.

j. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Figure 3-14 Locations of CPU Cage Screws

 

1

CPU cage screw locations (six) on motherboard (cage not shown)

2

Memory risers (eight)

3

Memory riser dividers (eight)

4

CPU heatsink

5

CPU heatsink captive screws (two on each heatsink)

 

 

Figure 3-15 Removing a CPU and Heatsink

 

1

CPU retaining lever

2

CPU retaining lid

3

CPU

4

Heatsink

5

Heatsink captive screws (two)

 

 


 

Replacing Memory Risers

The memory risers connect to the motherboard and each riser provides eight DIMM slots. The memory riser is hot-swappable when you use the Attention button, as described in the following procedure.

Figure 3-16 Memory Riser LEDs (Top View)

 

1

Attention button (used for hot-swapping and hot-adding)

2

Attention LED (indicates when hot-swapping is safe)

3

Power LED (indicates whether the riser has power)

4

Mirror activity LED (indicates whether memory mirroring is enabled)

5

DIMM fault LEDs 1 through 8
(indicate which DIMM has failed)

 

 

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install a hot-swappable memory riser, follow these steps:


Step 1 Remove the memory riser that you are replacing: See Figure 3-17:

a. Slide the server out the front of the rack far enough so that you can remove the top cover.

b. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

c. Press the attention button (ATTN BUTTON) on the top of the memory riser (see Figure 3-16).

d. Wait until the attention LED (ATTN) and the power LED turn off.

e. Simultaneously press both green release buttons on the top of the memory riser to release the riser retaining latches. The latches open up to a 45-degree angle when they are released.

f. Grasp the open retaining latches and lift the memory riser straight up and out of the motherboard connector.

g. If you are installing or replacing DIMMs on the memory riser, use the instructions in Replacing DIMMs.

Step 2 Install (hot-add) a new memory riser:

a. Ensure that the riser release latches are in the open position.

a. Align the riser with the empty motherboard connector.

b. Push the riser into the connector until it is seated and the open release levers engage.

c. Simultaneously press down on each release lever to put them in the locked position. This ensures that the riser is properly seated in the motherboard connector.

d. Press the attention button, then wait until the attention LED turns off.

e. Replace the top cover.

f. Replace the server in the rack and replace any cables.

Figure 3-17 Removing and Replacing Memory Risers

 

1

Memory riser release buttons

2

Memory riser release latches


 

Replacing DIMMs

This section includes the following sections:


Note To ensure the best server performance, it is important that you are familiar with memory performance guidelines and population rules before you install or replace memory modules.


Memory Performance Guidelines and Population Rules

This section describes the type of memory that the server requires and its effect on performance. The following topics are covered:

Supported DIMMs

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

Memory Channels

Each CPU supports two memory risers. Figure 3-18 shows the placement of the CPUs and their corresponding memory risers. The view shown is facing the front of the server. This numbering is also inscribed on the top of the CPU cage.

Figure 3-18 CPUs and Memory Risers

 

CPU 4
CPU 3
CPU 2
CPU 1

MEM 8

MEM 7

MEM 6

MEM 5

MEM 4

MEM 3

MEM 2

MEM 1

 

Each memory riser contains two memory buffers that are connected to the CPU by serial memory interface (SMI) channels. Each memory buffer has two channels, each containing a pair of DDR3 DRAM slots.

In Figure 3-19, the buffers and channels are:

  • Buffer 1, channel 1: slots 1B and 1D
  • Buffer 1, channel 2: slots 1A and 1C
  • Buffer 2, channel 1: slots 2B and 2D
  • Buffer 2, channel 2: slots 2A and 2C

Figure 3-19 DIMM Slots and Memory Buffers

 

1

Memory Buffer 1

2

Memory Buffer 2

DIMM Population Rules

Following are the DIMM population rules:

  • The minimum configuration for the server is, at least one matched DIMM pair installed in a memory riser on either CPU1 or CPU2 (see Figure 3-18). All four CPUs can run from a single DIMM pair.
  • DIMMs are required to be populated in pairs. DIMMs for this server are sold as two-DIMM kits.
  • The DIMMs in any given pair must be identical.
  • Any DIMM installed in a memory riser corresponding to an empty CPU slot becomes inaccessible.
  • For optimal performance, distribute DIMMs evenly across all installed CPUs and memory buffers.
  • DIMMs within a channel are populated starting with the DIMMs farthest from the memory buffer in a fill-farthest approach.

For example, the order that you should populate the four channels on a memory riser is as follows (see also Figure 3-19):

1. Slots 1B and 1D

2. Slots 1A and 1C

3. Slots 2B and 2D

4. Slots 2A and 2C

DIMM and Rank Sparing

DIMM and rank sparing can be enabled in the BIOS configuration utility.

Sparing involves utilizing one of the DIMM pairs or rank pairs within each memory riser as a spare unit. When any of the other DIMM pairs within the same memory riser experiences errors beyond a pre-defined threshold, it fails over to the spare DIMM pair. Spared DIMMs and ranks are hidden from the user and the OS so that the BIOS can migrate to a spare unit when it finds degrading DIMMs.

When sparing is enabled, the available system memory is lesser than the total installed memory.

  • When using DIMM sparing, the available memory equals total installed memory minus the size of spared DIMMs.
  • When using rank sparing, available memory equals total installed memory minus the size of the spared ranks. Rank size equals DIMM size divided by the number of ranks.

DIMM Installation Procedure

This section contains the following topics:

Identifying a Faulty DIMM

The memory riser has LEDs on it supper surface that can assist you in isolating a faulty DIMM. The faulty DIMMs are indicated by the DIMM fault LEDs, which light amber to indicate which DIMMs are faulty. See Figure 3-20.

Figure 3-20 Memory Riser LEDs (Top View)

 

1

Attention button (used for hot-swapping)

2

Attention LED (indicates when hot-swapping is safe)

3

Power LED (indicates whether the riser has power)

4

Mirror activity LED (indicates whether memory mirroring is enabled)

5

DIMM fault LEDs 1 through 8
(indicate which DIMM has failed)

 

 

DIMM Replacement Procedure

To install a DIMM pair, follow these steps:


Note DIMM risers are hot-swappable when you use the attention button, so you do not have power off the server or disconnect power cords. Use the following procedure.



Step 1 Remove the DIMMs that you are replacing: See Figure 3-21:

a. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

b. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

c. Press the attention button (ATTN BUTTON) on the top of the memory riser that contains the faulty DIMM (see Figure 3-20).

d. Wait until the attention LED (ATTN) and the power LED turn off.

e. Simultaneously press both green release buttons on the top of the memory riser to release the riser retaining latches. The latches open up to a 45-degree angle when they are released.

f. Lift the memory riser straight up and out of the motherboard connector.

g. Locate the faulty DIMM and remove it from the connector on the memory riser.

Step 2 Install a new DIMM:

a. Insert the DIMM assembly into the connector on the riser.

b. Push the DIMM into the connector until it is seated properly, and the white clips on either side of the connector lock into place.

c. Push the hot-addable memory riser into the motherboard connector until it is seated and the open release levers engage the chassis and the CPU cage.

d. Simultaneously press down on each release lever to put them in the closed position. This ensures that the riser is properly seated in the motherboard connector.

e. Press the attention button on the top of the memory riser, then wait until the attention LED turns off (see Figure 3-20).

f. Replace the top cover.

g. Replace the server in the rack and replace any cables.

Figure 3-21 Removing and Replacing DIMMs

 

1

DIMM slots

2

Memory riser


 

Replacing Fan Modules

The eight fan modules in the server are numbered as follows when you are facing the front of the server. Each fan module has a fault LED that lights amber when the fan module fails.

Figure 3-22 Fan Module Numbering

 

FAN 8

FAN 7

FAN 6

FAN 5

FAN 4

FAN 3

FAN 2

FAN 1

 

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install a hot-pluggable fan module, follow these steps:


Caution You do not have to shut down or power off the server to replace fan modules because they are hot- pluggable. However, to maintain proper cooling, do not operate the server for more than one minute with any fan module removed.


Step 1 Remove the fan module that you are replacing: See Figure 3-23:

a. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

b. Remove the top cover as described in “Removing and Replacing the Server Top Cover” section.

c. Insert your thumb and forefinger in the two green release latches on top of the fan module.

d. Squeeze the release latches together and lift out the fan module.

Step 2 Install a new fan module:

a. Grasp the fan module by the release latches and align it with the empty fan bay and the motherboard connector.


Note As you face the front of the server, the connector on underside of the fan module should be oriented on the right-bottom side of the fan module. See Figure 3-23.


b. Press down on the top corners of the fan module until the connector is fully seated and the release latches lock in place.

c. Replace the top cover.

d. Replace the server in the rack.

Figure 3-23 Removing and Replacing Fan Modules

 

1

Fan module release latches

2

Fan module, front view showing connector on underside of the module


 

Replacing Hard Drives or Solid State Drives


Tip You do not have to shut down or power off the server to replace hard drives or SSDs because they are hot pluggable.



Note You can mix hard drives and solid state drives (SSDs) in the same server. However, You cannot configure a logical volume (virtual drive) that contains a mix of hard drives and SSDs. That is, when you create a logical volume, it must contain all hard drives or all SSDs.



Note Cisco recommends following the industry-standard practice of using drives of the same capacity when configuring RAID volumes. If you use drives of different capacities, the usable portion of the smallest drive will be used on all drives of the RAID volume.



Note The 500 GB SATA drives (A03-D500GC3) and the 1 TB SATA drives (A03-D1TBSATA) that are sold with the UCS C460 M2 server are supported at full 6G speeds.


The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install a hot-pluggable hard drive, follow these steps:


Step 1 Remove the drive that you are replacing or remove a blank panel from an empty bay (See Figure 3-25):

a. Press the release button on the face of the hard drive.

b. Grasp the ejector lever and pull the hard drive tray out of the slot.

c. If you are replacing an existing drive, remove the four drive tray screws that secure the drive to the tray and then lift the drive out of the tray.

Step 2 Install a new drive:

a. Place a new hard drive in the empty drive tray and replace the four drive tray screws.

b. Insert the drive tray into the empty drive bay.

c. Push the tray into the slot until the drive connectors are fully seated in the backplane.

d. Press the ejector lever flat to lock the drive and tray in place.

Figure 3-24 Drive Numbering and Physical Orientation, Facing Server Front

 

HDD_01

HDD_02

HDD_03

HDD_04

HDD_05

HDD_06

HDD_07

HDD_08

HDD_09

HDD_10

HDD_11

HDD_12

Figure 3-25 Removing and Replacing Hard Drives

 

1

Ejector lever

2

Release button

3

Drive tray securing screws

 

 


 

Replacing a DVD Drive


Warning Class 1 laser product.
Statement 1008


The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install a DVD drive, follow these steps:


Step 1 Remove the DVD drive that you are replacing. See Figure 3-26:

a. Power off the server as described in Shutting Down and Powering Off the Server.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in Removing and Replacing the Server Top Cover.

d. Remove all memory risers. See Replacing Memory Risers.

e. Remove all memory riser dividers.

f. Remove the black plastic cover from the floor of the chassis by removing the four screws that secure it. This strip covers the width of the chassis floor behind the fan modules.

g. Push the release button on the rear of the DVD drive, and then push the DVD drive out the front panel.

Step 2 Install a new DVD drive:

a. Insert the rear of the new DVD drive into the empty DVD drive bay on the front panel.

b. Push the drive inward until it seats in its connector and the release button locks in place.

c. Replace the black strip to the floor of the chassis by replacing its four screws.

d. Replace all memory riser dividers.

e. Replace all memory risers.

f. Replace the top cover.

g. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.

Figure 3-26 Removing and Replacing the DVD Drive

 

1

DVD drive bay

2

Rear of DVD drive


 

Replacing a PCIe Card


Note If you are installing a Cisco UCS Virtual Interface Card, there are prerequisite considerations. See Special Considerations for Cisco UCS Virtual Interface Cards.


This server has 10 PCIe expansion slots. See Figure 3-27 and Table 3-4 for information about the slots and which slots are hot-swappable. The replacement procedures differ depending on whether the PCIe slot is hot-swappable or not. This section contains the following topics:

PCIe Slots

Figure 3-27 PCIe Slots

 

 

Table 3-4 PCIe Slots

Slot Number
Electrical
Lane Width
Connector Length
Card Length 1
Card Height 2
NCSI 3 Support
Hot Pluggable

1

Gen-2 x8

x24 connector

3/4 length

Full-height

Yes4

Yes

2

Gen-2 x8

x24 connector

3/4 length

Full-height

Yes

Yes

3

Gen-2 x4

x8 connector

1/2 length

Full-height

No

No

4

Gen-2 x4

x8 connector

1/2 length

Full-height

No

No

5

Gen-2 x16

x16 connector

3/4 length

Full-height

No

No

6

Gen-2 x8

x8 connector

3/4 length

Full-height

No

Yes

7

Gen-2 x8

x8 connector

3/4 length

Full-height

No

Yes

8

Gen-1 x4

x8 connector

3/4 length

Full-height

No

No

9

Gen-1 x4

x8 connector

1/2 length

Full-height

No

No

10

Gen-2 x4

x8 connector

1/2 length

Full-height

No

No

1.This is the supported length because of internal clearance.

2.This is the size of the rear panel opening.

3.Network Communications Services Interface protocol.

4.Slot 1 can operate when the server is in standby power mode.


Note CPU3 must be installed to support PCIe slots 5, 6, 7, 9, and 10.



Note Legacy I/O devices like video cards are only supported on slots 1, 2, 3, 4 and 8.


Replacing a PCIe Card in a Non Hot-Swappable Slot

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To install or replace a PCIe card in slots 3, 4, 5, 8, 9, or 10, follow these steps:


Step 1 Remove a PCIe card:

a. Shut down and power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

c. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.

d. Remove any cables from the ports of the PCIe card that you are replacing.


Tip Label the cables when you disconnect them to aid correct connection to the new card.


e. Pinch and lift up the green retaining clip that secures the card to the chassis rear panel.

f. Lift the card straight up from the motherboard connector.


Note Lift up on both ends of the card evenly to avoid damaging its connector.


Step 2 Install a PCIe card:

a. Align the PCIe card with the empty PCIe connector on the motherboard.

b. Push down evenly on both ends of the card until it is fully seated in the motherboard connector.

c. Ensure that the card rear panel sits flat against the server back panel opening.

d. Push the green retaining clip down until it locks over the top of the card.

e. Replace the top cover.

f. Replace the server in the rack, replace cables, and then power on the server by pressing the Power button.


 

Replacing a PCIe Card in a Hot-Swappable Slot

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To install or replace a PCIe card in slots 1, 2, 6, or 7, follow these steps:


Step 1 Remove a hot-swappable PCIe card:

a. Slide the server out the front of the rack far enough so that you can remove the top cover.


Note You might have to detach cables from the rear panel to provide clearance.



Caution If you cannot safely view and access the component, remove the server from the rack.

b. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.

c. Remove any cables from the ports of the PCIe card that you are replacing.


Tip Label the cables when you disconnect them to aid correct connection to the new card.


d. Press the lightpipe switch that is on the top of the plastic divider for the hot-swappable PCIe slot.


Note Wait for the lightpipe switch LED to turn off before removing the card in the next step.


e. Pinch and lift up the green retaining clip that secures the card to the chassis rear panel.

f. Lift the card straight up from the motherboard connector.


Note Lift up on both ends of the card evenly to avoid damaging its connector.


Step 2 Install a hot-swappable PCIe card:

a. Align the PCIe card with the empty PCIe connector on the motherboard.

b. Push down evenly on both ends of the card until it is fully seated in the motherboard connector.

c. Ensure that the card rear panel sits flat against the server back panel opening.

d. Push the green retaining clip down until it locks over the top of the card.

e. Press the lightpipe switch that is on top of the plastic divider for the hot-swappable PCIe slot. Wait for the LED to turn on to ensure that the PCIe slot receives power.

f. Replace the top cover.

g. Replace the server in the rack and replace cables.


 

Special Considerations for Cisco UCS Virtual Interface Cards

Table 3-5 describes the requirements for the supported Cisco UCS virtual interface cards (VICs).

 

Table 3-5 Cisco UCS C460 Requirements for Virtual Interface Cards

Virtual Interface Card (VIC)
Number of VICs Supported in Server
Slots That Support VICs 5
Primary Slot For UCS Integration or Cisco Card NIC Mode
Minimum CIMC Firmware
Minimum VIC Firmware For Use With UCS Integration
Minimum Nexus OS on an Upstream Nexus Fabric Interconnect
Cisco UCS VIC P81E

N2XX-ACPCI01

2

PCIE 1

PCIE 2

PCIE 1

1.4(3)

2.0(2)

5.0

Cisco UCS VIC 1225

UCSC-PCIE-CSC-02

2

PCIE 1

PCIE 2

PCIE 1

1.4(6)

2.1(0)

5.0

Cisco UCS VIC1225T

UCSC-PCIE-C10T-02

2

PCIE 1

PCIE 2

PCIE 16

1.5(1)

2.1(1)

5.0

5.See PCIe Slots.

6.The Cisco UCS VIC1225T is not supported for UCS integration at this time.

Special Considerations for Cisco UCS Fusion ioDrive2 Storage Accelerator Cards

Table 3-6 describes the requirements for the supported Cisco UCS Fusion ioDrive2 cards.

 

Table 3-6 Cisco UCS C460 Requirements for Fusion ioDrive2 Cards

Card
Max. Number of Cards Supported
Slots That Support Cards 7
Minimum CIMC Firmware
Card Height
(rear-panel tab)
Cisco UCS 3.0 TB MLC Fusion ioDrive2

UCSC-F-FIO-3000M

10

All8

1.5(2)

Full height

Cisco UCS 1205 GB MLC Fusion ioDrive2

UCSC-F-FIO-1205M

10

All

1.5(2)

Half height9

Cisco UCS 785 GB MLC Fusion ioDrive2

UCSC-F-FIO-785M

10

All

1.5(2)

Half height

Cisco UCS 365 GB MLC Fusion ioDrive2

UCSC-F-FIO-365M

10

All

1.5(2)

Half height

7.See PCIe Slots.

8.See note below regarding slots 8 and 9.

9.A rear-panel tab adapter is required to fit the half-height cards in the full-height slots.


Note PCIe slots 8 and 9 are Gen1 x4 electrical lane width slots and provide limited bandwidth if used with these cards.


Installing Multiple PCIe Cards and Resolving Limited Resources

When a large number of PCIe add-on cards are installed in the server, the system may run out of the following resources required for PCIe devices:

  • Option ROM memory space
  • 16-bit I/O space

The topics in this section provide guidelines for resolving the issues related to these limited resources.

Resolving Insufficient Memory Space to Execute Option ROMs

The system has very limited memory to execute PCIe legacy option ROMs, so when a large number of PCIe add-on cards are installed in the server, the system BIOS might not able to execute all of the option ROMs. The system BIOS loads and executes the option ROMs in the order that the PCIe cards are enumerated (Slot 1, Slot 2, Slot 3, etc.).

If the system BIOS does not have sufficient memory space to load any PCIe option ROM, it skips loading that option ROM, reports a system event log (SEL) event to the CIMC controller and reports the following error in the Error Manager page of the BIOS Setup utility:

ERROR CODE SEVERITY INSTANCE DESCRIPTION
146 Major N/A PCI out of resources error.
Major severity requires user
intervention but does not
prevent system boot.

 

To resolve this issue, disable the Option ROMs that are not needed for system booting. The BIOS Setup Utility provides the setup options to enable or disable the Option ROMs at the PCIe slot level for the PCIe expansion slots and at the port level for the onboard NICs. These options can be found in the BIOS Setup Utility Advanced → PCI Configuration page (see Figure 3-28).

Figure 3-28 BIOS Setup Utility Advanced > PCI Configuration Page

 

  • Guidelines for RAID controller booting:

If the RAID controller does not appear in the system boot order, even if the option ROMs for the RAID controller slots are enabled, the RAID controller option ROM might not have sufficient memory space to execute. In that case, disable other option ROMs that are not needed for the system configuration to free up some memory space for the RAID controller option ROM.

  • Guidelines for onboard NIC PXE booting:

If the system is configured to primarily perform PXE boot from onboard NICs, make sure that the option ROMs for the onboard NICs to be booted from are enabled in the BIOS Setup Utility. Disable other option ROMs that are not needed to create sufficient memory space for the onboard NICs.

Resolving Insufficient 16-Bit I/O Space

The system has only 64 KB of legacy 16-bit I/O resources available. This 64 KB of I/O space is divided between two I/O Hubs (IOH0 and IOH1) in the system.

The default BIOS setting is to allocate 40 KB to IOH0 and 24 KB to IOH1. Every PCIe device that is connected to IOH0 has to share the 40 KB of I/O. Every PCIe device connected to IOH1 must share that 24 KB of I/O.

When a large number of PCIe cards are installed in the system, the system BIOS might not have sufficient I/O space for some PCIe devices. If the system BIOS is not able to allocate the required I/O resources for any PCIe devices, the following symptoms have been observed:

  • The BIOS might appear to hang while initializing PCIe devices.
  • The PCIe option ROMs might take excessive time to complete, which appears to lock up the system.
  • PCIe boot devices might not be accessible from the BIOS.
  • PCIe option ROMs might report initialization errors. These errors are seen before the BIOS hands control to the operating system.
  • The keyboard might not work.

To work around this problem, rebalance the load on each IOH using the following methods:

1. Adjust the IOH IO Resource Allocation ratio. This option is included in the BIOS Setup Utility Advanced → PCI Configuration page (see Figure 3-28).

2. Physically remove any unused PCIe cards.

3. Physically rearrange the PCIe cards to create a balance between IOH0 and IOH1.

In this server, the PCIe slots are connected to IOH0 and IOH1 as follows:

Slot 1: IOH 0

Slot 2: IOH 0

Slot 3: IOH 0

Slot 4: IOH 0

Slot 5: IOH 1

Slot 6: IOH 1

Slot 7: IOH 1

Slot 8: IOH 0

Slot 9: IOH 1

Slot 10: IOH 1

Installing an NVIDIA Tesla C2050 GPU Card


Note The NVIDIA Tesla C2050 GPU card is supported in Generation M2 servers only. This card is supported only in PCIe slot 5 of the server (see Figure 3-27). When installed in PCIe slot 5, this double-wide card physically covers two slots, slot 5 and slot 6.



Note The DVI connector on the rear panel of the GPU card is disabled. Connect your monitor to one of the VGA connectors on the server instead.


Cisco ships an accessory kit (UCSC-GPU_N01-C460) that includes two adapters that you must use to customize the card for installation in the Cisco C460 M2 server. Each kit includes the following components (see Figure 3-29):

  • One rear-panel faceplate
  • One front bracket
  • One power cable adapter

Figure 3-29 Cisco Adapters For NVIDIA Tesla C2050 GPU Card

1

Rear-panel faceplate

3

Front bracket

2

GPU card (not included in kit)

4

Power cable adapter

To install or replace an NVIDIA Tesla C2050 GPU card, follow these steps:


Step 1 Prepare the server for installation of the card:

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.

e. Open the two green PCIe retainer latches on PCIe slots 5 and 6 (see Figure 3-30).

f. Remove any PCIe cards or blanking panels from PCIe slots 5 and 6.

g. Remove the plastic divider assembly that is over PCIe slots 6 and 7 to make clearance for the card.

Pull up on the pin that attaches this divider assembly to the motherboard, then lift the assembly out of the server. This pin is on the front end of the divider assembly where it touches the motherboard (see Figure 3-30).

Figure 3-30 Cisco C2050 GPU Card Slots

1

PCIe slot 5

3

Release pin on front of divider assembly

2

PCIe slot 6

 

 

Step 2 Install adapters on the GPU card to prepare for installation:

a. Remove the default rear-panel faceplate from the rear end of a new card. Remove the two securing screws and set them aside.

b. Install the new rear-panel faceplate to the rear end of the card (see Figure 3-31). Install the two securing screws that you removed in the prior step.

c. Install the front bracket to the front end of the card (see Figure 3-31). Insert the pins on the bracket into the corresponding holes in the card and then tighten the single securing thumbscrew.

d. Connect the power cable adapter to the card. Insert the six-pin connector on the cable into the six-pin connector on the front end of the card.

Figure 3-31 Assembling Cisco C2050 GPU Card Components

1

Rear-panel faceplate with two securing screws

3

Front bracket

2

GPU card, side view

4

GPU card, top view

Step 3 Install a new GPU card to the server:

a. Align the card so that its bottom-edge connector is over PCIe slot 5.

b. Tilt the front end of the card downward as you lower it into the chassis so that the attached power cable goes under the CPU cage (behind CPU3). The two thumbscrews on the front bracket that you installed must align with the two threaded holes in the CPU cage (see Figure 3-32).

c. Perform the following two actions simultaneously:

Insert the two tabs on the rear-panel faceplate of the card into the two chassis rear-panel openings for slots 5 and 6.

Carefully push down on both ends of the card to seat its connector in the PCIe slot 5 motherboard connector.

d. Close the two green PCIe retainer latches for slots 5 and 6 over the top of the rear-panel faceplate.

e. Tighten the two thumbscrews on the front bracket of the card to secure it to the CPU cage.

Step 4 Install the power cable adapter:

a. Disconnect the PCIe power cable from the motherboard. This is 10-pin connector J166 (see Figure 3-32).

b. Connect the male 10-pin connector of the power cable adapter to the motherboard connector J166.

c. Connect the PCIe power cable into the female 10-pin connector on the power cable adapter, which you attached to the card in Step 2.

Step 5 Replace the top cover.

Step 6 Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Figure 3-32 Installing and Cabling the GPU Card

1

Six-pin connector to GPU card

3

Ten-pin female connector to PCIe power cable

2

Ten-pin male connector to motherboard socket J166

 

 


 

Replacing a Trusted Platform Module (TPM)

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications :

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To install or replace a trusted platform module (TPM), follow these steps:


Step 1 Remove a TPM (see Figure 3-33):

a. Power off the server as described in the “Shutting Down and Powering Off the Server” section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the “Removing and Replacing the Server Top Cover” section.

e. Remove the securing screw that holds the TPM to the motherboard. The TPM header is on the motherboard just behind the CMOS battery (see Figure 3-33).

f. Lift up on both ends of the TPM to free it from the TPM header pins on the motherboard.

Step 2 Install a TPM:

a. Align the connector that is on the underside of the new TPM with the TPM header pins on the motherboard, and then press firmly on both ends of the TPM to seat it.

b. Replace the securing screw that holds the TPM to the motherboard.

c. Replace the top cover.

d. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Step 3 Enable the TPM:

a. Watch during bootup for the F2 prompt, and then press F2 to enter BIOS setup.

b. Log into the BIOS Setup utility with your BIOS Administrator password.

c. On the BIOS Setup utility screen, select the Advanced tab.

d. Select Trusted Computing to open the TPM Security Device Configuration screen.

e. Change TPM SUPPORT to Enabled .

f. Press F10 to save your settings and reboot the server.

Step 4 Verify that the TPM is now enabled.

a. Watch during bootup for the F2 prompt, and then press F2 to enter BIOS setup.

b. Log into the BIOS Setup utility with your BIOS Administrator password.

c. Select the Advanced tab.

d. Select Trusted Computing to open the TPM Security Device Configuration screen.

e. Verify that TPM SUPPORT is Enabled.


Note If you want to use the Intel Trusted Execution Technology (TXT) feature, it must be enabled in the server BIOS as described in Enabling the Intel Trusted Execution Technology (TXT) Feature For the TPM.


Figure 3-33 Removing and Replacing a TPM

1

Securing screw

3

TPM header on motherboard

2

TPM

 

 

Enabling the Intel Trusted Execution Technology (TXT) Feature For the TPM

Intel TXT provides greater protection for information that is used and stored on the business server. A key aspect of that protection is the provision of an isolated execution environment and associated sections of memory where operations can be conducted on sensitive data, invisibly to the rest of the system. Likewise, Intel TXT provides for a sealed portion of storage where sensitive data such as encryption keys can be kept, helping to shield them from being compromised during an attack by malicious code.

To enable the TXT feature, follow these steps:


Step 1 Verify that a TPM is now installed and enabled in the server:

a. Either attach a VGA monitor and USB keyboard to the server, or log in remotely to the CIMC interface of the server and open a virtual KVM console window.

b. Reboot the server.

c. Watch during bootup for the F2 prompt, and then press F2 to enter BIOS setup.

d. Log in to the BIOS Setup utility with your BIOS Administrator password.


Note You must be logged in as the BIOS administrator to perform this procedure. If you have not done so already, set a BIOS administrator password on the Security tab of the BIOS Setup utility.


e. Select the Advanced tab.

f. On the Advanced tab, select Trusted Computing to open the TPM Security Device Configuration screen.

g. Verify that TPM SUPPORT is Enabled . If it is not, set TPM SUPPORT to Enabled.

h. Press Escape to return to the BIOS Setup utility Advanced tab.

Step 2 Enable the Intel Trusted Execution Technology (TXT) feature:

a. On the Advanced tab, select Intel TXT(LT-SX) Configuration to open the Intel TXT(LT-SX) Hardware Support screen.


Note The Intel Trusted Execution Technology feature can be enabled only when the server has a TPM installed on the TPM header.


b. Set TXT Support to Enabled .

Step 3 On the same screen, verify that the Intel Virtualization Technology (VT) and the Intel VT for Directed I/O (VT-d) features are enabled (the factory default).

a. On the Intel TXT(LT-SX) Hardware Support screen, verify that VT-d Support and VT Support are both listed as Enabled .

If they are already enabled, skip to Step 4.

If VT-d Support and VT Support are not enabled, continue with the next steps to enable them.

b. Press Escape to return to the BIOS Setup utility Advanced tab.

c. On the Advanced tab, select Processor Configuration to open the Processor Configuration screen.

d. Set Intel (R) VT and Intel (R) VT-d to Enabled.

Step 4 Press F10 to save your changes and exit the BIOS Setup utility.

Step 5 Verify that the Intel TXT, VT, and VT-d features are enabled:

a. Reboot the server.

b. Watch during bootup for the F2 prompt, and then press F2 to enter BIOS setup.

c. Select the Advanced tab.

d. Select Intel TXT(LT-SX) Configuration and verify that TXT Support, VT-d Support, and VT Support are Enabled.