Cisco UCS C250 Server Installation and Service Guide
Maintaining the Server
Downloads: This chapterpdf (PDF - 3.76MB) The complete bookPDF (PDF - 7.58MB) | Feedback

Maintaining the Server

Table Of Contents

Maintaining the Server

Server Monitoring and Management Tools

Cisco Integrated Management Interface (CIMC)

Server Configuration Utility

Status LEDs

Front Panel LEDs

Rear Panel LEDs

Preparing for Component Installation

Required Equipment

Shutting Down and Powering Off the Server

Removing and Replacing the Server Top Cover

Replaceable Component Locations

Installing or Replacing Components

Installing Hard Drives and Solid State Drives

Installing Fan Modules

Installing Power Supplies

Installing a DVD Drive

Installing DIMMs

Memory Performance Guidelines and Population Rules

DIMM Installation Procedure

Installing CPUs and Heatsinks

Installing a Motherboard CMOS Battery

Replacing a PCIe Riser Card Assembly

Installing a PCIe Card

Replacement Procedure

Special Considerations for the Cisco UCS P81E Virtual Interface Card (N2XX-ACPCI01)

Installing Multiple PCIe Cards and Resolving Limited Resources

Replacing an LSI Battery Backup Unit


Maintaining the Server


This chapter describes how to diagnose hardware problems with status LEDs and how to install or replace hardware components, and includes the following sections:

Server Monitoring and Management Tools

Status LEDs

Preparing for Component Installation

Installing or Replacing Components

Server Monitoring and Management Tools

Cisco Integrated Management Interface (CIMC)

You can monitor the server inventory, health, and system event logs by using the built-in Cisco Integrated Management Controller (CIMC) GUI or CLI interfaces. See the user documentation for your firmware release at the following URL:

http://www.cisco.com/en/US/products/ps10739/products_installation_and_configuration_guides_list.html

Server Configuration Utility

Cisco has also developed the Cisco Server Configuration Utility for C-Series servers, which can aid and simplify the following tasks:

Monitoring server inventory and health

Diagnosing common server problems with diagnostic tools and logs

Setting the BIOS booting order

Configuring some RAID configurations

Installing operating systems

This utility is shipped with new servers on CD. You can also download the ISO from Cisco.com. See the user documentation for this utility at the following URL:

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/ucsscu/user/guide/20/SCUUG20.html

Status LEDs

This section describes the locations and interpretations of LEDs on the server that can provide status and troubleshooting information. This section includes the following topics:

Front Panel LEDs

Rear Panel LEDs

Front Panel LEDs

Figure 3-1 shows the names and locations of the front panel LEDs.

Figure 3-1 Front Panel LEDs

1

Locator LED/Locator button

2

Network activity LED

3

System fault LED

4

Power status LED/Power button

5

CPU fault LED

6

Memory fault LED

7

Power supply fault LED

8

DVD activity LED

9

Hard drive activity LED

10

Hard drive power LED

11

Fan module fault LED

   

Table 3-1 describes the possible states and interpretations for the LEDs shown in Figure 3-1.


Note In the LED states descriptions, a moderate fault is a fault in which the first warning threshold for the component has been passed (for example, overheating). In most cases, the moderate fault self-corrects, at which time the LED returns to normal state. A "severe fault" is a fault that requires immediate service action.


Table 3-1 Front Panel LEDs  

LED Name
State

Locator
(this is also a Locator button)

Off—The Locator LED is not in use.

Blue, flashing—The Locator LED button was pressed and the Locator LED flashes on the front and rear panels to help you find the server in a rack.

Network activity

Off—The server is in standby power mode.

Green—The server is in main power mode.

Green, blinking—The server is communicating with the network. The blink rate is faster as network activity increases.

System status

Green—The server is operating properly within thresholds.

Amber—A moderate fault event has been detected. Investigate other LEDs and sensors to isolate the problem.

Amber, blinking—A severe fault event that requires immediate service action has been detected. Investigate other LEDs and sensors to isolate the problem.

Power status
(this is also the Power button)

Off—No AC power is present.

Amber—The server is in standby power mode.

Green—The server is in main power mode.

See the "Connecting and Powering On the Server (Standalone Mode)" section for definitions of these power modes.

CPU fault

Green—All CPUs are operating properly.

Amber—At least one CPU has a moderate fault.

Amber, blinking—At least one CPU has a severe fault.

Memory fault

Off—All DIMMs are operating properly.

Amber—At least one DIMM has a moderate fault.

Amber, blinking—At least one DIMM has a severe fault.

Power supply fault

Off—All power supplies are operating properly.

Amber—At least one power supply has a moderate fault.

Amber—At least one power supply has a severe fault.

DVD activity

Off—The DVD drive is not in use.

Green, blinking—The DVD drive is reading or writing data.

Hard drive activity

Off—There is no hard drive in the hard drive sled.

Green—The hard drive is ready.

Green, blinking—The hard drive is reading or writing data.

Hard drive fault

Off—The hard drive is operating properly.

Amber—This hard drive has failed.

Amber, blinking—The device is rebuilding.

Fan module fault

Off—The fan module is operating properly.

Amber—This fan module has failed.


Rear Panel LEDs

Figure 3-2 shows the names and locations of the rear panel LEDs.

Figure 3-2 Rear Panel LEDs

1

Rear Locator button/LED

2

Network activity LED

3

Overheating fault LED

4

CPU fault LED

5

Memory fault LED

6

Hard drive fault LED

7

System fault LED

8

Power supply status LED

9

Power supply fault LED

10

10/100/1000 Gigabit Ethernet speed LED

11

10/100/1000 Gigabit Ethernet link status LED

12

10/100 Ethernet speed LED

13

10/100 Ethernet link status LED

   

Table 3-2 describes the possible states and interpretations for the LEDs shown in Figure 3-2.

Table 3-2 Rear Panel LEDs  

LED Name
State

Locator
(This is also a Locator button)

Off—The Locator LED is not in use.

Blue, flashing—The Locator LED/button on the front panel was pressed and the Locator LED flashes on the front and rear panels to help you find the server in a rack.

Network activity

Off—The server is in standby power mode.

Green—The server is in main power mode.

Green, blinking—The server is communicating with the network. The blink rate is faster as network activity increases.

System status

Green—The server is operating properly within thresholds.

Amber—A moderate fault event has been detected. Investigate other LEDs and sensors to isolate the problem.

Amber, blinking—A severe fault event that requires immediate service action has been detected. Investigate other LEDs and sensors to isolate the problem.

CPU fault

Green—All CPUs are operating properly.

Amber—At least one CPU has a moderate fault.

Amber, blinking—At least one CPU has a severe fault.

Memory fault

Off—All DIMMs are operating properly.

Amber—At least one DIMM has a moderate fault.

Amber, blinking—At least one DIMM has a severe fault.

Hard drive fault

Off—The hard drive is operating properly.

Amber—This hard drive has failed.

Fan module fault

Off—All fan modules are operating properly.

Amber—At least one fan module has a moderate fault.

Amber, blinking—At least one fan module has a severe fault.


Note The power supply status LED and the power supply fault LED on the power supply must be read in combination for the following interpretations.


Power supply status

Power supply fault

Status off + Fault off—No AC power is present in any power supplies.

Status green + Fault off—This power supply is operating properly in main power mode.

Status green, blinking + Fault off—This power supply is operating properly in standby power mode.

Status off + Fault amber, blinking—This power supply has had a warning event, but continues to operate. Warning events occur when thresholds are passed for high temperature, high power, or slow power supply fan.

Status off + Fault amber—This power supply has failed. Failure events occur when thresholds are passed for over-voltage, over-current, over-temperature, and power supply fan failure).

See the "Connecting and Powering On the Server (Standalone Mode)" section for definitions of these power modes.


Note The 10/100 Ethernet link status LED and the speed LED must be read in combination for the following interpretations.


10/100 Ethernet speed (left)

10/100 Ethernet link status
(right)

Link status off + speed off—No link is present on this port.

Link status off + speed solid green—A half-duplex, 10-Mbps link is present.

Link status amber + speed solid green—A half-duplex, 100-Mbps link is present.

Link status off + speed blinking green—A full-duplex,
10-Mbps link is present.

Link status amber + speed blinking green—A full-duplex, 100-Mbps link is present.


Note The 10/100/1000 Gigabit Ethernet link status LED and the speed LED must be read in combination for the following interpretations.


10/100/1000 Gigabit Ethernet speed (left)

10/100/1000 Gigabit Ethernet link status (right)

Link status off + speed off—No link is present on this port.

Link status off + speed solid green—A half-duplex, 10-Mbps link is present.

Link status green + speed solid green—A half-duplex, 100-Mbps link is present.

Link status amber + speed solid green—A half-duplex, 1000-Mbps link is present.

Link status off + speed blinking green—A full-duplex,
10-Mbps link is present.

Link status green + speed blinking green—A full-duplex, 100-Mbps link is present.

Link status amber + speed blinking green—A full-duplex, 1000-Mbps link is present.


Preparing for Component Installation

This section describes how to prepare the server for component installation and includes the following topics:

Required Equipment

Shutting Down and Powering Off the Server

Removing and Replacing the Server Top Cover

Replaceable Component Locations


Warning Only trained and qualified personnel must be allowed to install, replace, or service this equipment. Statement 1030

Warning This unit might have more than one power cord. To reduce the risk of electric shock, disconnect the two power supply cords before servicing the unit.
Statement 14

Required Equipment

The following items are used to perform the procedures in this chapter:

Number 2 Phillips-head screwdriver

Number 1 Phillips-head screwdriver

Needle-nose pliers

Electrostatic discharge (ESD) strap or other grounding equipment such as a grounded mat.

Shutting Down and Powering Off the Server


Tip If you have ordered a server with power supply redundancy (at least two power supplies), you do not have to power off the server to replace a power supply because they are redundant and hot-pluggable. The server hard drives and fan modules are also hot-pluggable.


The server can run in two power modes:

Main power mode—Power is supplied to all server components and any operating system on your hard drives can run.

Standby power mode—Power is supplied only to the service processor and the cooling fans and it is safe to power off the server from this mode.

You can invoke a graceful shutdown or an emergency shutdown (hard shutdown) by using either of the following methods:

Use the CIMC management interface. See either the Cisco UCS C-Series Rack-Mount Server Configuration Guide or the Cisco UCS C-Series Rack-Mount Server CLI Configuration Guide.

Use the Power button on the server front panel. To use the Power button, follow these steps:


Step 1 Check the color of the Power Status LED (see the "Front Panel LEDs" section).

Green indicates that the server is in main power mode and must be shut down before it can be safely powered off. Go to Step 2.

Amber indicates that the server is already in standby mode and can be safely powered off. Go to Step 3.

Step 2 Invoke either a graceful shutdown or a hard shutdown:


Caution To avoid data loss or damage to your operating system, you should always invoke a graceful shutdown of the operating system.

Graceful shutdown—Press and release the Power button. The operating system will perform a graceful shutdown and the server goes to standby mode, which is indicated by an amber Power Status LED.

Emergency shutdown—Press and hold the Power button for 4 seconds to force the main power off and immediately enter standby mode.

Step 3 Disconnect the power cords from the power supplies in your server to completely power off the server.


Removing and Replacing the Server Top Cover

To remove or replace the top cover of the server, follow these steps:


Warning This unit might have more than one power cord. To reduce the risk of electric shock, disconnect the two power supply cords before servicing the unit.
Statement 14


Tip You do not have to remove the cover to replace hard drives, fan modules, or power supplies.



Step 1 Remove the top cover:

a. Use a Number 2 Phillips-head screwdriver to turn the locking screw one-quarter turn to the unlocked position (see Figure 3-3).

b. Lift the release latch.

c. Push the cover toward the server rear about one inch, until it stops.

d. Lift the cover from the server and set it aside.

Step 2 Replace the top cover:

a. With the release latch in the raised, open position, set the cover in place about one inch back from the fully closed position.

b. Push the cover toward the server front until it stops.

c. Push the release latch to the flat, closed position.

d. Use a screwdriver to return the locking screw to the locked position.


Figure 3-3 Removing the Top Cover

1

Release latch

2

Locking screw


Replaceable Component Locations

This section shows the locations of the components that are discussed in this chapter. The view in Figure 3-4 is from the top down, with the top cover and internal air baffle removed.

Figure 3-4 Replaceable Component Locations (top view)

1

Fan modules (five), accessed through the front panel

6

DIMM slots (48)

2

Hard drives (up to eight), accessed through the front panel

7

CPUs and heatsinks (up to two)

3

DVD drive

8

Power supplies (up to two), accessed through the rear panel

4

Internal USB port on motherboard, not visible in this view
(active in server Generation M2 only)

9

PCIe riser card assembly (with chassis slots for two standard-profile cards)

5

Motherboard CMOS battery

10

PCIe riser card assembly (with chassis slots for three low-profile cards)


Installing or Replacing Components


Warning This unit might have more than one power cord. To reduce the risk of electric shock, disconnect the two power supply cords before servicing the unit.
Statement 14


Warning Blank faceplates and cover panels serve three important functions: they prevent exposure to hazardous voltages and currents inside the chassis; they contain electromagnetic interference (EMI) that might disrupt other equipment; and they direct the flow of cooling air through the chassis. Do not operate the system unless all cards, faceplates, front covers, and rear covers are in place.
Statement 1029

Caution When handling server components, wear an ESD strap to avoid damage.


Tip You can press the Locator button on the front panel or rear panel to turn on a flashing Locator LED on the server front and rear panels. This button allows you to locate the specific server that you are servicing when you go to the opposite side of the rack. See the "Server Monitoring and Management Tools" section for locations of the LEDs.


This section describes how to remove and replace server components. This section includes the following topics:

Installing Hard Drives and Solid State Drives

Installing Power Supplies

Installing a DVD Drive

Installing Fan Modules

Installing DIMMs

Installing CPUs and Heatsinks

Installing a Motherboard CMOS Battery

Replacing a PCIe Riser Card Assembly

Installing a PCIe Card

Replacing an LSI Battery Backup Unit

Installing Hard Drives and Solid State Drives

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html


Tip You do not have to shut down or power off the server to replace hard drives or SSDs because they are hot pluggable.



Note You can mix hard drives and solid state drives (SSDs) in the same server. However, You cannot configure a logical volume (virtual drive) that contains a mix of hard drives and SSDs. That is, when you create a logical volume, it must contain all hard drives or all SSDs.



Note If you need to make changes to your RAID configuration after installing a drive, use the LSI configuration utility for your mass storage device. See the "RAID Controller Considerations" section for more information.



Note Cisco recommends following the industry-standard practice of using drives of the same capacity when configuring RAID volumes. If you use drives of different capacities, the usable portion of the smallest drive will be used on all drives of the RAID volume.



Note The 500 GB SATA drives (A03-D500GC3) and the 1 TB SATA drives (A03-D1TBSATA) that are sold with the UCS C250 M2 server are supported at 3G speeds. These are 6G drives and might possibly run faster than 3G in this server, but they are supported to run at minimum 3G speeds.


To replace or install a hot-pluggable hard drive or SSD, follow these steps:


Step 1 Remove the drive that you are replacing or remove a blank panel from an empty bay (see Figure 3-6):

a. Press the release button to expose the ejector lever.

b. Pull the ejector lever toward you, and then pull the drive sled from the drive bay.

c. If you are replacing an existing drive, remove the drive from the sled by removing the four screws from the sides of the sled.

Step 2 Install a new drive:

a. Place the drive in the sled with the connectors facing the rear.

b. Install the four securing screws on the sides of the sled.

c. With the ejector lever still open, push the sled into the drive bay until you feel the drive stop against the backplane.

d. Press the ejector lever flat until the lock clicks into place.

Figure 3-5 Hard Drive/SSD CIMC Numbering and Physical Orientation, Facing Server Front

 

HDD_01

HDD_02

HDD_03

HDD_04

HDD_05

HDD_06

HDD_07

HDD_08


Figure 3-6 Removing and Replacing Drives

1

Release button

2

Ejector lever

3

Securing screws (four on sides of sled)

   


Installing Fan Modules

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install a hot-pluggable fan module, follow these steps:


Tip You do not have to shut down or power off the server to replace fan modules because they are hot pluggable. However, to maintain proper cooling, do not operate the server for more than one minute with any fan module removed.



Step 1 Remove the fan module that you are replacing (see Figure 3-8):

a. Press the release button to expose the ejector lever.

b. Pull the ejector lever toward you to pull the fan module from its bay.

Step 2 Install a new fan module:

c. With the ejector lever still open, push the fan module into the bay until you feel it fully engage the backplane and stop.

d. Press the ejector lever flat until the lock clicks into place.

e. Press the Power button to return the server to main power mode.

Figure 3-7 Fan Module CIMC Numbering and Physical Orientation, Facing Server Front

fantray-0

fantray-1

fantray-2

fantray-3

fantray-4


Figure 3-8 Removing and Replacing Fan Modules

1

Release button

2

Ejector lever



Installing Power Supplies

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace or install a power supply, follow these steps:


Note If you have ordered a server with power supply redundancy (at least two power supplies), you do not have to power off the server to replace power supplies.



Step 1 Remove a power supply or a blank panel from an empty bay:

a. Do one of the following actions:

If your server has only one power supply, shut down and power off the server as described in the "Shutting Down and Powering Off the Server" section.

If your server has two power supplies, the power supplies are hot pluggable so you do not have to shut down the server.

b. Remove the power cord from the power supply that you are replacing.

c. Push the release lever toward the center of the power supply and pull on the power supply handle to disengage it from the backplane (see Figure 3-10).

d. Remove the power supply from the bay.

Step 2 Install a new power supply:

a. Insert the new power supply into the bay and push it in until the release lever clicks and locks.

b. Replace the power cord to the new power supply.

c. Press the Power button to return the server to main power mode.

Figure 3-9 Power Supply CIMC Numbering and Physical Orientation, Facing Server Rear

PSU1

PSU0


Figure 3-10 Removing and Replacing a Power Supply

1

Release lever

2

Power supply handle



Installing a DVD Drive

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html


Warning Class 1 laser product.
Statement 1008

To replace a DVD drive, follow these steps:


Step 1 Remove a DVD drive:

a. Power off the server as described in the "Shutting Down and Powering Off the Server" section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the "Removing and Replacing the Server Top Cover" section.

e. Remove the cable from the connector on the rear of the DVD drive (see Figure 3-11).

f. Remove the one screw that secures the DVD drive to the chassis.

g. Push the DVD drive out of the front chassis opening.

Step 2 Replace a DVD drive:

a. Align the new DVD drive with the front panel opening and push it into the chassis until the front plate is even with the chassis front.

b. Replace the one screw that secures the DVD drive to the chassis.

c. Replace the cable to the connector on the rear of the DVD drive.

d. Replace the top cover.

e. Replace the server in the rack, replace power cords and any other cables, then power on the server by pressing the Power button.

Figure 3-11 Removing and Replacing a DVD drive

1

DVD drive (shown with top cover removed)

2

DVD cable connector

3

DVD securing screw (one)

   


Installing DIMMs

This section includes the following sections:

Memory Performance Guidelines and Population Rules

DIMM Installation Procedure


Note To ensure the best server performance, it is important that you are familiar with memory performance guidelines and population rules before you install or replace memory modules.


Memory Performance Guidelines and Population Rules

This section describes the type of memory that the server requires and its effect on performance. The following topics are covered:

Supported DIMMs

Banks and Channels

Memory Population Rules and Supported Configurations

Low-Voltage DIMM Considerations (Generation M2 Only)

Supported DIMMs

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

Banks and Channels

Each channel is identified by a letter—A, B, and C for CPU 1, and D, E, and F for CPU 2. Each channel has eight DIMM slots numbered 0 through 7.

Each bank is identified by numbers, 0 through 7. For example, DIMM slots A1, B1, and C1 belong to Bank 1, while A2, B2, C2 belong to Bank 2.

Figure 3-12 shows how DIMM slot banks and channels are physically arranged on the motherboard.

Figure 3-12 Physical Representation of Banks and Channels

Memory Population Rules and Supported Configurations

This server uses Cisco extended memory technology. To improve performance, write operations are made simultaneously to both DIMMs of a matched pair. When considering the memory configuration of your server, consider the following items:

Refer to Table 3-3 for the supported DIMM configurations and the total memory for each.

Refer to Table 3-4 for the order in which DIMM pairs must be populated for each supported configuration. Find the column for the number of DIMMs in your configuration, then read down the column to see which DIMM slots on each CPU must be populated for that configuration. Also see Figure 3-12 for DIMM slot positions.

Both DIMMs of a pair must be installed side-by-side, under the same CPU, in matching color DIMM sockets.

For example, to add DIMM pairs to go from a a 16-DIMM configuration to a 20-DIMM configuration, refer to Table 3-4. As the differences between the columns for the 16-DIMM and 20-DIMM configurations indicate, you add one DIMM pair to slots B4 and B5 under CPU1 and one DIMM pair to slots E4 and E5 under CPU2.

As shown in Figure 3-12, B4 and B5 are side-by-side blue slots controlled by CPU1; E4 and E5 are side-by-side blue slots controlled by CPU2.

The two DIMMs within a DIMM pair must have the identical manufacturer, type, speed, and size. Cisco provides spare DIMMs for this product in matched pair kits.

We recommend you do not mix different sized DIMMs or DIMMs with different clock rates in the same server. This causes the memory system to operate at the speed of the slowest DIMMs that are installed.

(Generation M2 only) Low-voltage (1.35 V) DIMM pairs and standard-voltage DIMM pairs (1.5 V) can be mixed in the same server. Note that this causes the system BIOS to default to standard-voltage operation (Performance Mode). See the Low-Voltage DIMM Considerations (Generation M2 Only).


Note Low voltage DIMMs are supported in Generation M2 servers only.


1. Refer to Table 3-3, which summarizes the supported DIMM configurations.

2. Refer to Table 3-4 for the slot population rules for each configuration, based on the total number of DIMMs.


Note All Generation M2 server DIMM configurations require two CPUs to be installed in the server.


Table 3-3 Supported DIMM Configurations

Total Memory for CPU1+CPU2
CPU1 DIMMs
 
CPU2 DIMMs
Total Number of DIMMS

64GB

Four 8GB pair-kits
(eight 4GB DIMMs)

 

Four 8GB pair-kits
(eight 4GB DIMMs)

16

80GB

Five 8GB pair-kits
(ten 4GB DIMMs)

 

Five 8GB pair-kits
(ten 4GB DIMMs)

20

96GB

Six 8GB pair-kits
(twelve 4GB DIMMs)

 

Six 8GB pair-kits
(twelve 4GB DIMMs)

24

128GB

Eight 8GB pair-kits
(sixteen 4GB DIMMs)

 

Eight 8GB pair-kits
(sixteen 4GB DIMMs)

32

128GB

Four 16GB pair-kits
(eight 8GB DIMMs)

 

Four 16GB pair-kits
(eight 8GB DIMMs)

16

144GB

Nine 8GB pair-kits
(eighteen 4GB DIMMs)

 

Nine 8GB pair-kits
(eighteen 4GB DIMMs)

36

160GB

Ten 8GB pair-kits
(twenty 4GB DIMMs)

 

Ten 8GB pair-kits
(twenty 4GB DIMMs)

40

192GB

Twelve 8GB pair-kits
(twenty-four 4GB DIMMs)

 

Twelve 8GB pair-kits
(twenty-four 4GB DIMMs)

48

192GB

Six 16GB pair-kits
(twelve 8GB DIMMs)

 

Six 16GB pair-kits
(twelve 8GB DIMMs)

24

256GB

Eight 8GB pair-kits
(sixteen 4GB DIMMs)
plus
Four 16GB pair-kits
(eight 8GB DIMMs)

 

Eight 8GB pair-kits
(sixteen 4GB DIMMs)
plus
Four 16GB pair-kits
(eight 8GB DIMMs)

48

320GB

Four 8GB pair-kits
(eight 4GB DIMMs)
plus
Eight 16GB pair-kits
(sixteen 8GB DIMMs)

 

Four 8GB pair-kits
(eight 4GB DIMMs)
plus
Eight 16GB pair-kits
(sixteen 8GB DIMMs)

48

384GB

Twelve 16GB pair-kits
(twenty-four 8GB DIMMs)

 

Twelve 16GB pair-kits
(twenty-four 8GB DIMMs)

48


Table 3-4 shows the order in which you must populate the DIMM slots for each configuration, based on the total number of DIMMs.


Note Before you install DIMMs, see Table 3-3 to ensure that you are using a supported configuration. Then populate the DIMM slots as shown in Table 3-4.



Note The 4-, 8-, or 12-DIMM configurations are not supported in UCS C250 Generation M2 servers.
The 4-, 8-, or 12-DIMM configurations are supported in UCS C250 Generation M1 only.


Table 3-4 DIMM Slot Population for Supported Configurations

4 DIMM
 
8 DIMM
 
12 DIMM
 
16 DIMM
 
20 DIMM
 
24 DIMM
 
32 DIMM
 
36 DIMM
 
40 DIMM
 
48 DIMM

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

 

CPU1

CPU2

     

B1

   

B1

F0

 

B1

F0

 

B4

F0

 

B4

F0

 

B4

F0

 

B4

F0

 

B4

F0

 

B4

F0

               

B5

 

B5

F1

B5

F1

B5

F1

B5

F1

B5

F1

                           

B2

 

B2

 

B2

F6

                           

B3

 

B3

 

B3

F7

                             

F3

B7

 

B7

F3

                             

F2

B6

 

B6

F2

               

B1

 

B1

F5

B1

F5

B1

F5

B1

F5

B1

F5

   

B0

 

B0

F1

B0

F1

B0

F1

B0

F4

B0

F4

B0

F4

B0

F4

B0

F4

A1

D0

A1

D0

A1

D0

A4

D0

A4

D0

A4

D0

A4

D0

A4

D0

A4

D0

A4

D0

           

A5

D1

A5

D1

A5

D1

A5

D1

A5

D1

A5

D1

A5

D1

                       

A2

D6

A2

 

A2

D6

A2

D6

                       

A3

D7

A3

 

A3

D7

A3

D7

                       

A7

D3

 

D3

A7

D3

A7

D3

                       

A6

D2

 

D2

A6

D2

A6

D2

           

A1

D5

A1

D5

A1

D5

A1

D5

A1

D5

A1

D5

A1

D5

A0

D1

A0

D1

A0

D1

A0

D4

A0

D4

A0

D4

A0

D4

A0

D4

A0

D4

A0

D4

     

E0

C1

E0

C1

E0

 

E0

C4

E0

C4

E0

C4

E0

C4

E0

C4

E0

                 

E1

C5

E1

C5

E1

C5

E1

C5

E1

C5

E1

                           

C2

   

E6

C2

E6

                           

C3

   

E7

C3

E7

                             

E3

 

E3

C7

E3

                             

E2

 

E2

C6

E2

                       

C1

E5

 

C1

E5

 

C1

E5

 

C1

E5

 

C1

E5

 

C1

E5

       

E1

 

C0

E1

 

C0

E1

 

C0

E4

 

C0

E4

 

C0

E4

 

C0

E4

 

C0

E4

 

C0

E4


Low-Voltage DIMM Considerations (Generation M2 Only)

The Generation M2 of this server can be ordered with low-voltage (1.35 V) DIMM pairs or standard-voltage (1.5 V) DIMM pairs. Note the following considerations:

The two low-voltage DIMMs within a DIMM pair must have the identical manufacturer, type, speed, and size. Cisco provides spare DIMMs for this product in matched pair kits.

Low-voltage DIMM pairs and standard-voltage DIMM pairs can be mixed in the same server. Note that this causes the system BIOS to default to standard-voltage operation (Performance Mode). That is, the server cannot operate in Power Saving Mode unless all DIMM pairs in the server are low-voltage DIMMs.


Note CPUs that have a maximum memory frequency less than 1333 MHz support LV DIMMs operating in Power saving Mode only, and do not support Performance mode. However, if your server has one of these CPUs, monitoring output for LV DIMMs will show identical voltages for both Power Saving Mode and Performance Mode. In this case, Performance Mode is not active and its fields only mirror the values from the Power Savings Mode fields.


In Generation M2 only, there is a setting in the BIOS Setup utility that you can use to change the DDR memory mode when the server has all low-voltage DIMMs installed. To access this setting, follow these steps:


Step 1 Enter the BIOS setup utility by pressing the F2 key when prompted during bootup.

Step 2 Select the Advanced tab.

Step 3 Select Low Voltage DDR Mode.

Step 4 In the pop-up window, select either Power Saving Mode or Performance Mode.

Power Saving Mode-Enables low-voltage memory operation. This setting is available only if all DIMMs installed are low-voltage DIMMs.

Performance Mode-Disables low-voltage memory operation. If you mix low-voltage DIMM pairs with standard-voltage DIMM pairs, the system defaults to this setting.

Step 5 Press F10 to save your changes and exit the setup utility.


DIMM Installation Procedure

To install or replace a DIMM, follow these steps:


Note Refer to the DIMM slot information and population rules before removing or installing DIMMs. See the "Memory Performance Guidelines and Population Rules" section.



Step 1 Remove an existing DIMM:

a. Power off the server as described in the "Shutting Down and Powering Off the Server" section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the "Removing and Replacing the Server Top Cover" section.

e. Remove the clear air baffle that covers the motherboard. Loosen the six thumbscrews and then lift the baffle up and out of the server.

f. Push down on the ejector levers at both ends of the DIMM slot (see Figure 3-13).

g. Grasp the DIMM by its top corners and remove it from the server. Place the DIMM on an antistatic mat or in antistatic packaging.

Step 2 Install a DIMM:


Note If you are upgrading and adding memory, be sure to check the supported configurations shown in the "Memory Population Rules and Supported Configurations" section. Upgrading to some configurations might require you to move existing DIMMs.



Note If you are installing low-voltage DIMMs, see the "Low-Voltage DIMM Considerations (Generation M2 Only)" section.


a. With the ejector levers at both ends of the DIMM slot fully open, orient the DIMM's alignment notch with the alignment key in the DIMM slot.

b. Press down on both top corners of the DIMM simultaneously until the ejector levers click and lock in place on the DIMM.

c. Replace the air baffle and the top cover.

d. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Figure 3-13 Removing and Replacing Memory Modules

1

DIMM slot ejector lever (two on each slot)

2

Alignment key in DIMM slot



Installing CPUs and Heatsinks


Note All Generation M2 server DIMM configurations require two CPUs to be installed in the server.


The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To install or replace a CPU and heatsink, follow these steps:


Step 1 Remove an existing CPU and heatsink:

a. Power off the server as described in the "Shutting Down and Powering Off the Server" section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the "Removing and Replacing the Server Top Cover" section.

e. Remove the clear air baffle that covers the motherboard. Loosen the six thumbscrews and then lift the baffle up and out of the server.

f. Loosen the four captive screws that hold the heatsink to the chassis posts and lift the heatsink out of the server (see Figure 3-14).


Note Alternate loosening or tightening each screw evenly to avoid damaging the heatsink or CPU.


g. If you plan to use this heatsink with your new CPU, clean the old thermal compound from the bottom of the heatsink.

h. Unhook the CPU socket latch and open the CPU cover plate.

i. Lift the CPU out of the socket and set it aside on an antistatic mat or in an antistatic bag.

Step 2 Install a CPU and heatsink:

a. Set the new CPU into the socket, being careful to orient the CPU alignment notches with the alignment keys in the socket.

b. Close the CPU cover plate and hook the socket latch back in the locked position.

c. Prepare the thermal compound, which aids heat transference between the CPU and the heatsink:

If you are installing a new heatsink, a thermal pad comes pre-applied on the heatsink. Remove the protective film to expose the thermal pad. Do not use the syringe of thermal compound that is shipped with a replacement CPU in this case.

If you are using your existing heatsink, make sure that all of the old thermal compound is removed from the underside of the heatsink. Then apply the full contents of the syringe that is supplied with the new CPU to apply thermal compound to the top of the CPU. Apply the compound as evenly as possible so that it forms an even layer when the heatsink is tightened down.

d. Align the heatsink captive screws with the chassis posts, then tighten all captive screws evenly.

e. Replace the air baffle and the top cover.

f. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Figure 3-14 Removing and Replacing CPUs and Heatsinks

1

Captive heatsink screws (four per CPU)

2

CPU heatsink

3

CPU socket latch (open)

4

CPU socket cover plate (open)

5

CPU socket alignment key (both sides of CPU)

   


Installing a Motherboard CMOS Battery

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html


Warning There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer's instructions.
Statement 1015

To install or replace a motherboard CMOS battery, follow these steps:


Step 1 Remove a motherboard CMOS battery:

a. Power off the server as described in the "Shutting Down and Powering Off the Server" section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the "Removing and Replacing the Server Top Cover" section.

e. Press the battery socket retaining clip toward the chassis wall (see Figure 3-15).

f. Lift the battery from the socket. Use pliers to grasp the battery if there is not enough clearance for your fingers.

Step 2 Install a motherboard CMOS battery:

a. Press the battery socket retaining clip toward the chassis wall.

b. Insert the new battery into the socket with the battery's flat side and positive marking (+) toward the outer chassis wall. Ensure that the retaining clip clicks over the top of the battery.


Note The flat, positive side of the battery marked with "+" must face the outer chassis wall.


c. Replace the top cover.

d. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Figure 3-15 Removing and Replacing a Motherboard CMOS Battery

1

Motherboard CMOS battery socket
(positive + side toward chassis wall)

2

Battery retaining clip



Note The flat, positive side of the battery marked with "+" must face the outer chassis wall as shown.



Replacing a PCIe Riser Card Assembly

The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

To replace a PCIe riser card assembly, follow these steps:


Step 1 Remove a PCIe riser card assembly:

a. Power off the server as described in the "Shutting Down and Powering Off the Server" section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the "Removing and Replacing the Server Top Cover" section.

e. Remove the riser card retaining plate by loosening the two captive thumbscrews, then lifting the plate out of the chassis (see Figure 3-16).

f. Lift the riser card assembly and any attached PCIe cards straight up and out of the chassis. Lift up on both ends of the assembly evenly to avoid damaging the sockets or the riser cards.

g. Remove any PCIe card from the riser card assembly and set it aside. See the "Installing a PCIe Card" section.

Step 2 Install a PCIe riser card assembly:

a. Replace any PCIe card that was in the riser card assembly.

b. Set the assembly in place, aligning the riser card connector with the socket on the motherboard.

c. Press down evenly on both ends of the assembly to fully engage the riser cards with the sockets on the motherboard.

d. Replace the riser card retaining plate. Align the pegs on the plate with both riser card assemblies, then tighten the captive thumbscrews on the plate.

e. Replace the top cover.

f. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Figure 3-16 Removing and Replacing a PCIe Riser Card Assembly

1

Riser card retaining plate

2

Riser card assembly (standard profile)



Installing a PCIe Card

This section contains the following topics:

Replacement Procedure

Special Considerations for the Cisco UCS P81E Virtual Interface Card (N2XX-ACPCI01)

Installing Multiple PCIe Cards and Resolving Limited Resources


Note For more information about RAID controllers and cables, see also RAID Controller Considerations.



Note If you are installing a Cisco UCS P81E Virtual Interface Card (N2XX-ACPCI01), there are prerequisite considerations. See Special Considerations for the Cisco UCS P81E Virtual Interface Card (N2XX-ACPCI01).


The qualified and supported part numbers for this component are subject to change over time. For the most up-to-date list of replaceable components, see the following URL and then scroll to Technical Specifications:

http://www.cisco.com/en/US/products/ps10493/products_data_sheets_list.html

The PCIe slots in this server are numbered as shown inFigure 3-17. The specifications for each slot are shown in Table 3-5.

Figure 3-17 PCIe Slot Numbering and Physical Orientation, Facing Server Rear

PCIe Slot C

PCIe Slot D

PCIe Slot B

PCIe Slot E

PCIe Slot A

 

Table 3-5 PCIe Expansion Slots 

Slot Number
Electrical
Lane Width
Connector Length
Card Length 1
Card Height 2
NCSI 3 Support

A

Gen-2 x8

x16 extended

1/2 length

Half-height

No

B

Gen-2 x8

x16connector

1/2 length

Half-height

No

C

Gen-2 x8

x16 connector

1/2 length

Half-height

No

D

Gen-2 x16

x16 connector

1/2 length

Full-height

Yes4

E

Gen-2 x16

x16 connector

1/2-length

Full-height

No

1 This is the supported length because of internal clearance.

2 This is the size of the rear panel opening.

3 Network Communications Services Interface protocol.

4 Slot 2 can operate when the server is in standby power mode.


Replacement Procedure

Installing a PCIe card requires that you first remove the riser card assembly from the chassis. To install or replace a PCIe card, follow these steps:


Step 1 Remove a PCIe card:

a. Power off the server as described in the "Shutting Down and Powering Off the Server" section.

b. Disconnect all power cords from the power supplies.

c. Slide the server out the front of the rack far enough so that you can remove the top cover. You might have to detach cables from the rear panel to provide clearance.


Caution If you cannot safely view and access the component, remove the server from the rack.

d. Remove the top cover as described in the "Removing and Replacing the Server Top Cover" section.

e. Remove the riser card retaining plate by loosening the two captive thumbscrews, then lift the plate out of the chassis.

f. Lift the riser card assembly and any attached PCIe cards straight up and out of the chassis. Lift up on both ends of the assembly evenly to avoid damaging the sockets or the riser cards.

g. Remove the screw that secures the PCIe card rear plate to the assembly rear opening and disconnect any cables attached to the card (see Figure 3-18).

h. Pull the PCIe card connector out of the riser card socket and set the card aside.

Step 2 Install a PCIe card:

a. If you are installing a PCIe card to an empty slot on the riser card assembly, remove any blank panel from the assembly rear opening by removing the screw that secures the panel. Set the screw aside.


Note A standard-profile PCIe card must be installed in the right-hand riser card assembly, as viewed from the rear of the server. A low-profile PCIe card can be installed in either the low-profile assembly on the left, or the standard-profile assembly on the right if a standard-profile I/O bracket is used on the card.


b. Align the PCIe card connector with the riser card socket and push on both ends of the card evenly to fully engage the connector with the riser card socket.

c. Install the screw that secures the rear plate of the card to the assembly rear opening.

d. Set the assembly in place, aligning the riser card connector with the socket on the motherboard.

e. Press down evenly on both ends of the assembly to fully engage the riser cards with the sockets on the motherboard.

f. Replace the riser card retaining plate. Align the pegs on the plate with both riser card assemblies, then tighten the captive thumbscrews on the plate.

g. Replace the top cover.

h. Replace the server in the rack, replace power cords and any other cables, and then power on the server by pressing the Power button.

Figure 3-18 Removing and Replacing a PCIe Card


Special Considerations for the Cisco UCS P81E Virtual Interface Card (N2XX-ACPCI01)

The Cisco UCS P81E Virtual Interface Card is a standard-profile, half-length, dual-port 10 Gb PCIe card with SFP+. See the following special considerations and prerequisites.

This card is supported in server Generations M1 and M2.

This server supports installation of up to two of these cards.

These cards are supported only in PCIe slots D and E of this server.


Note This card must be installed in PCIe slot D to use the Cisco Card NIC mode (see Figure 3-17). See also NIC Modes and NIC Redundancy Settings.


This card requires that the server has CIMC firmware version 1.2(1) or later installed. There is a heartbeat LED on the top and bottom of the card that indicates when firmware is active.

To use this card for UCS integration (UCSM mode) with Cisco UCS Manager 2.0(2xx) or later, the minimum card-firmware and uboot image level is 2.0(2g).

To use this card for UCS integration (UCSM mode) with Cisco UCS Manager 1.4 or 2.0(1), the minimum card-firmware and uboot image level is 1.4(1i).

To connect this card to an upstream Cisco Nexus fabric interconnect (switch), the minimum NXOS version on the fabric interconnect must be 5.0 or later.

Installing Multiple PCIe Cards and Resolving Limited Resources

When a large number of PCIe add-on cards are installed in the server, the system may run out of the following resources required for PCIe devices:

Option ROM memory space

16-bit I/O space

The topics in this section provide guidelines for resolving the issues related to these limited resources.

Resolving Insufficient Memory Space to Execute Option ROMs

Resolving Insufficient 16-Bit I/O Space

Resolving Insufficient Memory Space to Execute Option ROMs

The system has very limited memory to execute PCIe legacy option ROMs, so when a large number of PCIe add-on cards are installed in the server, the system BIOS might not able to execute all of the option ROMs. The system BIOS loads and executes the option ROMs in the order that the PCIe cards are enumerated (Slot 1, Slot 2, Slot 3, etc.).

If the system BIOS does not have sufficient memory space to load any PCIe option ROM, it skips loading that option ROM, reports a system event log (SEL) event to the CIMC controller and reports the following error in the Error Manager page of the BIOS Setup utility:

ERROR CODE			SEVERITY			INSTANCE			DESCRIPTION
146			Major			N/A			PCI out of resources error.
									Major severity requires user
									intervention but does not
									prevent system boot.

To resolve this issue, disable the Option ROMs that are not needed for system booting. The BIOS Setup Utility provides the setup options to enable or disable the Option ROMs at the PCIe slot level for the PCIe expansion slots and at the port level for the onboard NICs. These options can be found in the BIOS Setup Utility Advanced Æ PCI Configuration page.

Guidelines for RAID controller booting:

If the server is configured to boot primarily from RAID storage, make sure that the option ROMs for the slots where your RAID controllers installed are enabled in the BIOS, depending on your RAID controller configuration.

If the RAID controller does not appear in the system boot order even with the option ROMs for those slots are enabled, the RAID controller option ROM might not have sufficient memory space to execute. In that case, disable other option ROMs that are not needed for the system configuration to free up some memory space for the RAID controller option ROM.

Guidelines for onboard NIC PXE booting:

If the system is configured to primarily perform PXE boot from onboard NICs, make sure that the option ROMs for the onboard NICs to be booted from are enabled in the BIOS Setup Utility. Disable other option ROMs that are not needed to create sufficient memory space for the onboard NICs.

Resolving Insufficient 16-Bit I/O Space

The system has only 64 KB of legacy 16-bit I/O resources available. This 64 KB of I/O space is divided between the CPUs in the system because the PCIe controller is integrated into the CPUs. This server BIOS has the capability to dynamically detect the 16-bit I/O resource requirement for each CPU and then balance the 16-bit I/O resource allocation between the CPUs accordingly during the PCI bus enumeration phase of the BIOS POST.

When a large number of PCIe cards are installed in the system, the system BIOS might not have sufficient I/O space for some PCIe devices. If the system BIOS is not able to allocate the required I/O resources for any PCIe devices, the following symptoms have been observed:

The system might get stuck in an infinite reset loop.

The BIOS might appear to hang while initializing PCIe devices.

The PCIe option ROMs might take excessive time to complete, which appears to lock up the system.

PCIe boot devices might not be accessible from the BIOS.

PCIe option ROMs might report initialization errors. These errors are seen before the BIOS passes control to the operating system.

The keyboard might not work.

To work around this problem, rebalance the 16-bit I/O load using the following methods:

1. Physically remove any unused PCIe cards.

2. If the system has one or more Cisco virtual interface cards (VICs) installed, disable the PXE boot on the VICs that are not required for the system boot configuration by using the Network Adapters page in the CIMC WebUI to free up some 16-bit I/O resources. Each VIC uses a minimum 16 KB of 16-bit I/O resource, so disabling PXE boot on Cisco VICs would free up some 16-bit I/O resources that can be used for other PCIe cards that are installed in the system.

Replacing an LSI Battery Backup Unit

In the Cisco UCS C250 server, the battery backup unit (BBU) is installed directly onto the LSI controller card.


Note A cable is shipped with the BBU that is used for installing the BBU on a bracket in other server models. Do not use this cable in the Cisco UCS C250 server; install the BBU directly onto the LSI controller card. Connecting the extra cable causes error messages because the multiple connections are detected as multiple BBUs (an unsupported configuration).



Note LSI recommends that you replace the LSI BBU once per year or after 1,000 recharge cycles, whichever comes first. Verify whether BBU replacement is required by looking in the CIMC. Log in to CIMC for the server, then click ServerInventoryStorageBattery Backup Unit. If the Battery Replacement Required field says, "True," then you must purchase a replacement BBU and replace it.



Warning There is danger of explosion if the battery is replaced incorrectly. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer's instructions.
Statement 1015

To install or replace a battery backup unit (BBU), follow these steps:


Step 1 Remove a BBU:

a. Remove the three screws that secure the BBU to the LSI controller card.

b. Lift the BBU straight up to free its connector from the socket on the LSI controller card.

Step 2 Install a BBU:

a. With the battery side up, set the BBU on the front of the LSI controller card.

Align the three screw-holes in the BBU with the three standoffs on the LSI controller card for guidance.

b. Carefully align the J5 connector on the underside of the BBU with the socket on the LSI controller card and then press downward to fully join the J5 connector with the socket.

c. Install the three screws that secure the BBU to the LSI controller card.