Maintaining the System

Maintaining the System

This chapter describes how to use LEDs to diagnose system problems and how to install or replace supported hardware components:

Status LEDs and Buttons

This section describes the location and meaning of LEDs and buttons and includes the following topics:

Front-Panel LEDs

Figure 3-1 shows the front-panel LEDs. Table 3-1 defines the front-panel LED states.

Figure 1. Figure 3-1 Front-Panel LEDs

1

System Power button and power status LED

5

Temperature status LED

2

System unit identification button and LED

6

Power supply status LED

3

System status LED

7

Network link activity LED

4

Fan status LED

8

Internal-drive status LEDs
Table 1. Table 3-1 Front-Panel LEDs States

LED Name

State

1

System Power button/power status LED

  • Off—There is no AC power to the system (all power cords are disconnected).

  • Amber—Both server nodes are powered off. Pressing the button will power on both server nodes.

  • Green—At least one server node is powered on. Pressing the button will power off both server nodes, returning the LED to amber.

2

System unit identification

  • Off—The unit identification LED is not in use.

  • Blue, blinking—The unit identification LED is activated.

3

System status

  • Green—The system is running in normal operating condition.

  • Green, blinking—The system is performing system initialization and memory check.

  • Amber, steady—The system is in a degraded operational state. For example:

    • Power supply redundancy is lost (power supply unplugged or failed).

    • CPUs are mismatched.

    • At least one CPU is faulty.

    • At least one DIMM is faulty.

    • At least one drive in a RAID configuration failed.

  • Amber, blinking—The system is in a critical fault state. For example:

    • Boot failed.

    • Fatal CPU and/or bus error is detected.

    • System is in an over-temperature condition.

4

Fan status

  • Green—All fan modules are operating properly.

  • Amber, steady—One fan module has failed.

  • Amber, blinking—Critical fault; two or more fan modules have failed.

5

Temperature status

  • Green—The system is operating at normal temperature.

  • Amber, steady—One or more temperature sensors have exceeded a warning threshold.

  • Amber, blinking—One or more temperature sensors have exceeded a critical threshold.

6

Power supply status

  • Green—All power supplies are operating normally.

  • Amber, steady—One or more power supplies are in a degraded operational state. (An event warning threshold has been reached, but the power supply continues to operate.)

  • Amber, blinking—One or more power supplies are in a critical fault state. (A critical fault threshold has been reached, causing the power supply to shut down.)

7

Network link activity

  • Off—The Ethernet link is idle.

  • Green—One or more Ethernet LOM ports are link-active.

  • Green, blinking—One or more Ethernet LOM ports are traffic-active.

8

Internal-drive status LEDs

 Use these LEDs to indicate the location of a failing drive. Then open the system cover to find exactly which drive is failing by looking at the LEDs on the drive trays.

  • The two columns of LEDs correspond to the two halves of the internal drive compartment (under either the right- or left-side top cover).

  • The four numbered rows of LEDs correspond to the four horizontal rows of drive bays (14 drive bays in each row).

See Figure 3-8 for an example. In this example, the red LED indicates that the failing drive is in the right half of the internal drive compartment, in row 3.

Rear-Panel LEDs and Buttons

Figure 3-2 shows the rear-panel LEDs and buttons for a S3260 system. This example is shown with a C3X60 M4 server node and an optional four-drive expansion module. Table 3-2 defines the rear-panel LED states.

Figure 2. Figure 3-2 S3260 System Rear-Panel LEDs and Buttons

1

Server node Power button/LED

(on each server node)

8

Not used at this time.

2

Server node unit identification button/LED

(on each server node)

9

10/100/1000 dedicated management port link activity LED (on each SIOC)

3

1 Gb Ethernet port link speed LED

(on each M5 server node only)

10

 10/100/1000 dedicated management port link speed LED (on each SIOC)

4

1 Gb Ethernet port link activity LED

(on each M5 server node only)

11

Solid state drive activity LED (each drive bay)

5

Power supply status LED (each power supply)

12

Solid state drive fault LED (each drive bay)

6

QSFP port link speed LED (on each port)

Actual ports will vary, depending on which SIOC version is used, or which adapter card is installed in a Version 03 SIOC.

13

Not used at this time.

7

QSFP port link activity LED (on each port)

14

SIOC health LED (one each Version 03 SIOC only)
Table 2. Table 3-2 Rear-Panel LED States

LED Name

State

1

Server node power button/LED

  • Off—There is no AC power to the server node. It is safe to remove the server node from the chassis.

  • Amber—The server node is in standby power mode. Power is supplied only to the Cisco IMC. It is safe to remove the server node from the chassis.

  • Green—The server node is in main power mode. Power is supplied to all server node components.

2

Server node unit identification button/LED

  • Off—The Identification LED is not in use.

  • Blue—The Identification LED is activated.

3

1 Gb Ethernet port link speed LED (on each M5 server node only)

  • Off—Link speed is 10/100 Mbps.

  • Green—Link speed is 1 Gbps.

4

1 Gb Ethernet port link activity LED (on each M5 server node only)

  • Off—No link is present.

  • Green—Link is present, no activity.

  • Green, blinking—Link is present, with activity.

5

Power supply status (on each power supply unit)

AC Power Supplies:

  • Off—No AC power input.

  • Green, steady—The power supply is operating normally and supplying DC power to the system.

  • Green, blinking—AC power is OK, DC output not enabled (standby mode).

  • Amber, blinking—An event warning threshold has been reached, but the power supply continues to operate.

  • Amber, steady—A critical fault threshold has been reached, causing the power supply to shut down.

DC Power Supplies:

  • Off—No DC input (12 V main power off, 12 V standby power off).

  • Green, steady—The power supply is operating normally and supplying DC power to the system (12 V main power on).

  • Green, blinking—DC output not enabled (12 V main power off; 12 V standby power on).

  • Amber, blinking—An event warning threshold has been reached, but the power supply continues to operate with 12 V main power.

  • Amber, steady—A critical fault threshold has been reached, causing the power supply to shut down (for example, over-current, over-voltage or over-temperature failure).

6

QSFP port link LED

  • Off—No link.

  • Green—Link is present.

7

QSFP port activity LED

  • Off—No link is present.

  • Green—Link is present, no activity.

  • Green, blinking—Link is present, with activity.

8

Not used at this time.

9

10/100/1000 dedicated

management link activity

  • Off—No link is present.

  • Green—Link is present, no activity.

  • Green, blinking—Link is present, with activity.

10

10/100/1000 dedicated management link speed

  • Off—Link speed is 10/100 Mbps.

  • Green—Link speed is 1 Gbps.

11

Solid state drive activity

  • Off—There is no drive connected.

  • Green—Drive connected, no activity.

  • Green, blinking—The drive is reading or writing data.

12

Solid state drive fault

  • Off—The drive is operating properly.

  • Amber—This drive has failed (or the Locate function has been activated for the drive).

13

Not used at this time.

14

SIOC health

  • Off—System is idle.

  • Green—SIOC is working normally.

  • Green, blinking—There is a problem with the SIOC or a component in the SIOC.

If your system has an optional drive expander module, it has one module status LED:

Table 3. Table 3-3 Drive Expander Module Status LED

LED Status

P3V3

P5V

HDD Present

HDD Status

Off

None

None

None

None

Green

On

On

Plug-in

Good

Amber

On

On

No plug-in

NA

Amber

On

Fail

No plug-in

NA

Amber

On

On

Plug-in

Fail

Internal Diagnostic LEDs

This section contains the following topics:

Diagnostic LEDs in the Main Chassis

The diagnostic LEDs inside the main chassis compartments can be viewed while the system is powered on. See Figure 3-3 for the locations of these internal LEDs. See Table 3-4 for definitions of the LED states.

Figure 3. Figure 3-3 Internal Diagnostic LED Locations in the Main Chassis

1

Drive fault (on each drive carrier)

3

Fan module fault (on each fan module)

2

Drive activity (on each drive carrier)
Table 4. Table 3-4 Internal Diagnostic LED States

LED Name

State

1

SAS drive fault

  • Off—The drive is operating properly.

  • Amber—This drive has failed.

  • Amber, blinking—The device is rebuilding.

2

SAS drive activity

  • Off—There is no drive in the drive tray (no access, no fault).

  • Green—The drive is ready.

  • Green, blinking—The drive is reading or writing data.

3

Fan fault LEDs

  • Off—Component is functioning normally.

  • Amber—Component has failed.

Preparing for System Component Installation or Replacement

This section describes how to prepare for component installation, and it includes the following topics:

Required Equipment For Maintenance Procedures

The following equipment is used to perform the procedures in this chapter:

  • Number 1 Phillips-head screwdriver (for SIOC cover screws)

  • 1/4-inch or equivalent flat-head screwdriver (for separating the optional I/O expander from the server node)

  • Electrostatic discharge (ESD) strap or other grounding equipment such as a grounded mat

  • The tools required for CPU replacement are supplied with a CPU spare.

Shutting Down and Powering Off the S3260 System

You can invoke a graceful shutdown or a hard shutdown of the entire S3260 system by using either the Cisco IMC interface or the system power button on the front panel.


Caution

To completely remove power from the system, you must disconnect power cords from all power supplies.

You can also shut down an individual server node rather than the entire system. See Shutting Down an Individual Server Node.

To use the system power button to shut down the chassis, follow these steps:

SUMMARY STEPS

  1. Check the color of the System Power Status LED (see Front-Panel LEDs.
  2. Invoke either a graceful shutdown or a hard shutdown:

DETAILED STEPS

  Command or Action Purpose

Step 1

Check the color of the System Power Status LED (see Front-Panel LEDs.

  • Green—At least one server node is powered on. Go to Step 2.

  • Amber—Both server nodes are powered off. Go to Step 3.

Step 2

Invoke either a graceful shutdown or a hard shutdown:

Caution

 

To avoid data loss or damage to your operating system, you should always invoke a graceful shutdown of the operating system.

  • Graceful shutdown—Press and release the Power button. The operating system on the server nodes performs a graceful shutdown of both server nodes.

  • Emergency shutdown—Press and hold the Power button for 4 seconds to force the power off both server nodes.

Step 3

Disconnect power cords from all power supplies in your system to completely remove AC power and power off the system chassis.

Shutting Down an Individual Server Node

You can invoke a graceful shutdown or a hard shutdown of a server node by using either the Cisco Integrated Management Controller (Cisco IMC) interface, or the power button that is on the face of the server node.

Shutting Down a Server Node By Using the Cisco IMC GUI

To use the Cisco IMC GUI to shut down the server node, follow these steps:

SUMMARY STEPS

  1. Use a browser and the management IP address of the system to log in to the Cisco IMC GUI.
  2. In the Navigation pane, click the Chassis menu.
  3. In the Chassis menu, click Summary.
  4. In the toolbar above the work pane, click the Host Power link.
  5. In the Server Power Management dialog, select one of the following buttons for the server that you want to shut down:

DETAILED STEPS

  Command or Action Purpose

Step 1

Use a browser and the management IP address of the system to log in to the Cisco IMC GUI.

Step 2

In the Navigation pane, click the Chassis menu.

Step 3

In the Chassis menu, click Summary.

Step 4

In the toolbar above the work pane, click the Host Power link.

The Server Power Management dialog opens. This dialog lists all servers that are present in the system.

Step 5

In the Server Power Management dialog, select one of the following buttons for the server that you want to shut down:

Caution

 

To avoid data loss or damage to your operating system, you should always invoke a graceful shutdown of the operating system. Do not power off a server if any firmware or BIOS updates are in progress.

  • Shut Down—Performs a graceful shutdown of the operating system.

  • Power Off—Powers off the chosen server, even if tasks are running on that server.

It is safe to remove the server node from the chassis when the Chassis Status pane shows the Power State as Off for the server node that you are removing.

The physical power button on the server node face also turns amber when it is safe to remove the server node from the chassis.
Shutting Down a Server Node By Using the Power Button on the Server Node

To use the physical server node power button to shut down the server node only, follow these steps:

SUMMARY STEPS

  1. Check the color of the server node power status LED:
  2. Invoke either a graceful shutdown or a hard shutdown:

DETAILED STEPS

  Command or Action Purpose

Step 1

Check the color of the server node power status LED:

  • Green—The server node is powered on. Go to Step 2

    .

  • Amber—the server node is powered off. It is safe to remove the server node from the chassis

Step 2

Invoke either a graceful shutdown or a hard shutdown:

Caution

 

To avoid data loss or damage to your operating system, you should always invoke a graceful shutdown of the operating system. Do not power off a server if any firmware or BIOS updates are in progress.

  • Graceful shutdown—Press and release the Power button. The software performs a graceful shutdown of the server node.

  • Emergency shutdown—Press and hold the Power button for 4 seconds to force the power off the server node.

When the server node power button turns amber, it is safe to remove the server node from the chassis.

Removing Chassis and Component Covers

This section contains the following topics:

Opening the Main Chassis Top Covers

This system has three hinged top covers on the main chassis. Opening these covers gives access to the internal-drives compartment and the fan module compartment.


Note

The internal drives and cooling fans in the system are hot-swappable and are accessed by opening the top covers. When you rack and cable the system, be sure to allow enough slack in the power and other cables so that the system can be pulled out on the slide rails far enough to allow clearance for opening the top covers.

SUMMARY STEPS

  1. Open the left or right internal-drive compartment cover to access the hot-swappable internal drives:
  2. Open the fan compartment cover to access the hot-swappable fan modules:

DETAILED STEPS

  Command or Action Purpose

Step 1

Open the left or right internal-drive compartment cover to access the hot-swappable internal drives:

  1. a. For either the right or left side cover, pull the latch release buttons on both latches toward the outer edges of the chassis. This causes the spring-loaded latches to pop up.

  2. b. With both latches open, swing open the hinged cover from the center toward the outside.

  3. c. To secure the cover, close it down flat and then push both latches flat until they click and lock.

Step 2

Open the fan compartment cover to access the hot-swappable fan modules:

  1. a. Push both latch-buttons toward the center.

  2. b. While holding both latch-buttons, open the hinged cover from the center toward the rear.

  3. c. To secure the cover, hold both latch-buttons while you close the cover flat. Release the latch-buttons.

Figure 4. Figure 3-4 Opening the Main Chassis Top Covers

1

Latch-release buttons for left internal-drives compartment

3

Latch-release buttons for fan compartment

2

Latch-release buttons for right internal-drives compartment
Removing the System I/O Controller (SIOC) Cover

Note

You do not have to slide the system out of the rack to remove the SIOC from the rear of the system.

SUMMARY STEPS

  1. Power off the connected server nodes based on the scenario:
  2. Remove the SIOC from the system:
  3. Remove the SIOC cover:
  4. Reinstall the SIOC cover:
  5. Reinstall the SIOC in the system:
  6. Power on the server node that you powered off in the first step.

DETAILED STEPS

  Command or Action Purpose

Step 1

Power off the connected server nodes based on the scenario:

  1. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).

  2. If there are two NVMes in the SIOC you are removing, power off both server nodes.

See Shutting Down an Individual Server Node.

Step 2

Remove the SIOC from the system:

  1. Loosen the single captive thumbscrew on the SIOC and then open its two hinged levers to evenly disengage the SIOC from its backplane connector.

  2. Pull the SIOC from the system and set it on an antistatic work surface.

Step 3

Remove the SIOC cover:

  1. a. Use a #1 Phillips-head screwdriver to remove the screws that secure the cover. The version 02 SIOC cover has four securing screws. The Version 03 SIOC cover has six securing screws. See Figure 3-5 or Figure 3-6.

  2. b. Lift the cover straight up off the SIOC.

Step 4

Reinstall the SIOC cover:

  1. a. Set the cover back in place.

  2. b. Replace the four screws that secure the cover.

Step 5

Reinstall the SIOC in the system:

  1. Push the SIOC into its bay until it stops against the internal midplane.

  2. Close the two levers on the SIOC to fully engage the SIOC connector with its midplane.

  3. Tighten the thumbscrew on the SIOC levers.

Step 6

Power on the server node that you powered off in the first step.
Figure 5. Figure 3-5 Removing the SIOC Cover (Version 02 SIOC, UCS-S3260-SIOC)

1

SIOC cover

2

Cover screws, four (two on each side of the SIOC)
Figure 6. Figure 3-6 Removing the SIOC Cover (Version 03 SIOC, UCS-S3260-PCISIOC)

1

SIOC cover, including ejector levers

3

Four securing screws on sides of cover (two each side)

2

Two securing screws on top of cover

Installing or Replacing System Components


Warning

Blank faceplates and cover panels serve three important functions: they prevent exposure to hazardous voltages and currents inside the chassis; they contain electromagnetic interference (EMI) that might disrupt other equipment; and they direct the flow of cooling air through the chassis. Do not operate the system unless all cards, faceplates, front covers, and rear covers are in place. Statement 1029

Caution

This system weighs approximately 190 pounds (86 kilograms) when fully loaded with components. We recommend that you use a mechanical lift when lifting the system. Attempting this procedure alone could result in personal injury or equipment damage. Consider temporarily removing components such as hard drives while you move the system.

Replacing Hard Drives or Solid State Drives

This section includes the following topics:

4K Sector Format Drives Considerations

  • • You must boot 4K sector format drives in UEFI mode, not legacy mode. See Setting Up Booting in UEFI Mode in the BIOS Setup Utility, or Setting Up Booting in UEFI Mode in the Cisco IMC GUI.

  • • Do not configure 4K sector format and 512-byte sector format drives as part of the same RAID volume.

  • • Operating system support on 4K sector drives is as follows: Windows: Win2012 and Win2012R2; Linux: RHEL 6.5, 6.6, 6.7, 7.0, 7.2; SLES 11 SP3, and SLES 12. ESXi/Vmware is not supported.

  • • The 6 TB drives have 4096-byte sectors. VMware ESXi does not support this capability and therefore will not work with 6 TB drives.

Setting Up Booting in UEFI Mode in the BIOS Setup Utility

SUMMARY STEPS

  1. Use a web browser and the management IP address of the system to log into the Cisco IMC GUI management interface.
  2. Select Compute and then Server 1.
  3. Click Launch KVM to open a virtual KVM window for the server node.
  4. In the Launch KVM dialog, select Server 1 and click Launch.
  5. Reboot server node 1. Watch the KVM window for the prompt to press F2.
  6. Enter the BIOS setup utility by pressing the F2 key when prompted during bootup.
  7. Go to the Boot Options tab.
  8. Set UEFI Boot Options to Enabled.
  9. Under Boot Option Priorities, set your OS installation media (such as a virtual DVD) as your Boot Option #1.
  10. Go to the Advanced tab.
  11. Select LOM and PCIe Slot Configuration.
  12. Set the PCIe Slot ID: HBA Option ROM to UEFI Only.
  13. Press F10 to save changes and exit the BIOS setup utility. Allow the server to reboot.
  14. After the server reboots and the OS installs, verify the installation:

DETAILED STEPS

  Command or Action Purpose

Step 1

Use a web browser and the management IP address of the system to log into the Cisco IMC GUI management interface.

Step 2

Select Compute and then Server 1.

Step 3

Click Launch KVM to open a virtual KVM window for the server node.

Step 4

In the Launch KVM dialog, select Server 1 and click Launch.

Step 5

Reboot server node 1. Watch the KVM window for the prompt to press F2.

Step 6

Enter the BIOS setup utility by pressing the F2 key when prompted during bootup.

Step 7

Go to the Boot Options tab.

Step 8

Set UEFI Boot Options to Enabled.

Step 9

Under Boot Option Priorities, set your OS installation media (such as a virtual DVD) as your Boot Option #1.

Step 10

Go to the Advanced tab.

Step 11

Select LOM and PCIe Slot Configuration.

Step 12

Set the PCIe Slot ID: HBA Option ROM to UEFI Only.

Step 13

Press F10 to save changes and exit the BIOS setup utility. Allow the server to reboot.

Step 14

After the server reboots and the OS installs, verify the installation:

Setting Up Booting in UEFI Mode in the Cisco IMC GUI

SUMMARY STEPS

  1. Use a web browser and the management IP address of the system to log into the Cisco IMC GUI management interface.
  2. Select Compute and then Server 1.
  3. Select the BIOS tab.
  4. Under BIOS Properties, set Configured Boot Order to UEFI.
  5. Click Save Changes.
  6. Click Configure Boot Order.
  7. Select the Advanced tab.
  8. Click Add Local HDD.
  9. In the Add Local Disk dialog, enter the information for the 4K sector format drive. Enter a name and specify Slot M.
  10. Click Save Changes.
  11. Click Add Virtual Media.
  12. In the Add Virtual Media dialog, enter a name for your OS installation virtual media.
  13. Click Save Changes.
  14. Click Close.
  15. Click Launch KVM to open a virtual KVM window for the server node.
  16. In the Launch KVM dialog, select Server 1 and click Launch.
  17. Activate virtual media. Pull down the Virtual Media menu on the KVM window and select Activate Virtual Devices.
  18. Reboot the server node.
  19. Press F6 during the boot to enter the boot device menu.
  20. Select UEFI: Cisco vKVM-Mapped vDVD and press Enter.
  21. Proceed with the installation of your OS.

DETAILED STEPS

  Command or Action Purpose

Step 1

Use a web browser and the management IP address of the system to log into the Cisco IMC GUI management interface.

Step 2

Select Compute and then Server 1.

Step 3

Select the BIOS tab.

Step 4

Under BIOS Properties, set Configured Boot Order to UEFI.

Step 5

Click Save Changes.

Step 6

Click Configure Boot Order.

Step 7

Select the Advanced tab.

Step 8

Click Add Local HDD.

Step 9

In the Add Local Disk dialog, enter the information for the 4K sector format drive. Enter a name and specify Slot M.

Step 10

Click Save Changes.

Step 11

Click Add Virtual Media.

Step 12

In the Add Virtual Media dialog, enter a name for your OS installation virtual media.

Step 13

Click Save Changes.

Step 14

Click Close.

Step 15

Click Launch KVM to open a virtual KVM window for the server node.

Step 16

In the Launch KVM dialog, select Server 1 and click Launch.

Step 17

Activate virtual media. Pull down the Virtual Media menu on the KVM window and select Activate Virtual Devices.

Step 18

Reboot the server node.

Step 19

Press F6 during the boot to enter the boot device menu.

Step 20

Select UEFI: Cisco vKVM-Mapped vDVD and press Enter.

Step 21

Proceed with the installation of your OS.

After the OS installs and the system reboots, your OS is listed as a boot option.
Replacing HDDs or SSDs in the Internal Drive Compartment

This section contains the following topics:

Internal Drive Population Guidelines

The system has 56 internal drive bays in the main chassis. Figure 3-7 shows the internal drive bay numbering. When populating internal drives and for optimal results please adhere to the following guidelines:

  • Populate HDDs starting from the lowest-numbered bays to the highest. Populate row 1 – 14, then row 15 – 28, and so on.

  • Populate internal SSDs starting from the highest-numbered bays to the lowest. Populate bay 56, then bay 55, and so on.

    For example, a system with 36 HDDs and 2 SSDs should have the HDDs in bays 1 – 36 and the SSDs in bays 56 and 55.

  • The four colored boxes shown in Figure 3-7 represent the four power groups in which power is distributed to the drive bays.

Figure 7. Figure 3-7 Internal Drive Bay Numbering
Identifying a Faulty Internal Drive

The system has internal-drive fault LEDs on the right-front handle (see Figure 3-1). Use these LEDs to get an indication of the location of a failing drive.

SUMMARY STEPS

  1. Observe the internal-drive fault LEDs on the right-front handle.
  2. Open the right- or left-side cover and look at the fault LEDs on the drive trays.

DETAILED STEPS

  Command or Action Purpose

Step 1

Observe the internal-drive fault LEDs on the right-front handle.

  • The two columns of LEDs correspond to the two halves of the internal drive compartment (under either the right- or left-side top cover).

  • The four numbered rows of LEDs correspond to the four horizontal rows of drive bays (14 drive bays in each row).

See Figure 3-8 for an example. In this example, the amber LED indicates that the failing drive is in the right half of the internal drive compartment, in row 3.

Step 2

Open the right- or left-side cover and look at the fault LEDs on the drive trays.

A solid amber fault LED indicates a failed drive.

Figure 8. Figure 3-8 Internal-Drive Status LED Example

1

Internal-drive fault LEDs on right-front handle of system, indicating faulty drive in right side of row 3

3

Front of system

2

Fault LED on drive carrier
Replacing Internal Drives

Note

Cisco PID UCSC-C3X60-HD4TB (Toshiba model MG03SCA400) has been EOLed and should be replaced if any problems occur. This Cisco PID now uses a newer HGST model.


Note

SAS HDDs and SSDs are hot-swappable and can be replaced without removing power from the system.

SUMMARY STEPS

  1. Slide the system out the front of the rack far enough so that you can open the top cover.
  2. Identify a failing drive as described in Identifying a Faulty Internal Drive.
  3. Open the internal-drive compartment cover.
  4. Remove the faulty drive:
  5. Install a new drive:
  6. Close the chassis cover and push the system back into the rack.

DETAILED STEPS

  Command or Action Purpose

Step 1

Slide the system out the front of the rack far enough so that you can open the top cover.

Caution

 

If you cannot safely view and access the component, remove the system from the rack.

Step 2

Identify a failing drive as described in Identifying a Faulty Internal Drive.

Step 3

Open the internal-drive compartment cover.

Step 4

Remove the faulty drive:

  1. Press the release button on the drive carrier. The drive lever pops up.

  2. Lift the drive lever to the fully open, 90-degree position, then lift the drive straight up out of its bay. Spare drives are already installed in a carrier, so it is not necessary to remove the old drive from its carrier.

Step 5

Install a new drive:

Note

 

Observe the drive population guidelines in Internal Drive Population Guidelines.

  1. Align the new drive with the empty bay. Orient the drive so that its connector aligns with the connector on the board.

  2. Lower the drive until it touches the board connector and the drive lever begins to close.

  3. Press the drive lever down flat until it clicks and locks.

Step 6

Close the chassis cover and push the system back into the rack.

Figure 9. Figure 3-9 Internal Drive Carrier Features

1

Drive lever

3

Drive fault LED

2

Release button

4

Drive activity LED
Replacing Hard Drives in the Optional Disk Expansion Tray

This section contains the following topics:

Disk Expansion Tray Drives Population Guidelines

The optional disk expansion tray can hold up to four 3.5-inch hard drives,. Drive numbering is shown in Figure 3-10. When populating these drives, follow these guidelines.

  • Populate drives starting from the lowest-numbered bay to the highest.

    Figure 10. Figure 3-10 Disk Expansion Tray and Solid State Drive Numbering
Identifying a Faulty Disk Expansion Tray Drive

Each drive has a fault LED that lights solid amber to indicate a failing drive.

Replacing Drive Expander Module Drives

Note

SAS/SATA drives are hot-swappable and can be replaced without removing power from the system.

SUMMARY STEPS

  1. Identify a failing drive as described in Identifying a Faulty Disk Expansion Tray Drive.
  2. Remove the faulty drive:
  3. Install a new drive:

DETAILED STEPS

  Command or Action Purpose

Step 1

Identify a failing drive as described in Identifying a Faulty Disk Expansion Tray Drive.

Step 2

Remove the faulty drive:

  1. Press the release button on the drive carrier. The drive lever pops up.

  2. Lift the drive lever to the fully open position, then pull the drive straight up out of its bay.

    Spare drives are already installed in a carrier, so it is not necessary to remove the old drive from its carrier.

Step 3

Install a new drive:

  1. Align the new drive with the empty bay and then push the drive in until it touches the board connector and the drive lever begins to close.

  2. Press the drive lever down flat until it clicks and locks.

Figure 11. Figure 3-11 Drive Expander Module Drive Carrier Features

1

Drive lever

3

Drive fault LED

2

Release button

4

Drive activity LED
Replacing SAS/SATA Solid State Drives in the Rear Panel Bays

This section contains the following topics:

Rear-Panel SAS/SATA Solid State Drive Population Guidelines

There are four bays for SAS/SATA SSDs in the rear panel. Drive numbering is shown in Figure 3-12. When populating these drives, follow these guidelines.

  • Server 1 SSD1 and SSD2 can be managed by server node 1. See also Management Architecture.

  • Server 2 SSD1 and SSD1 can be managed by server node 2.

  • Populate drives starting with the lowest-numbered bay to the highest within each server-controlled pair.

  • If a pair of SSDs is not present, keep the blanking panel in the empty two SSD bays to ensure proper air flow (see Figure 3-12).

Identifying a Faulty Solid State Drive

Each solid state drive bay has a fault LED that lights solid amber to indicate a failing drive (see Rear-Panel LEDs and Buttons.

Replacing Solid State Drives

Note

SAS/SATA solid state drives are hot-swappable and can be replaced without removing power from the system.

SUMMARY STEPS

  1. Remove a faulty solid state drive:
  2. Install a new solid state drive:

DETAILED STEPS

  Command or Action Purpose

Step 1

Remove a faulty solid state drive:

  1. Grasp and pinch the release latch toward the center.

  2. Pull the solid state drive straight out of the bay.

Step 2

Install a new solid state drive:

Note

 

Observe the drive population guidelines in Rear-Panel SAS/SATA Solid State Drive Population Guidelines.

  1. Align the new drive with the empty bay (with the label facing up) and then push the drive in until it touches the board connector.

  2. Grasp and pinch the release latch toward the center while you push the drive fully into the bay, and then release the release latch.

Figure 12. Figure 3-12 Solid State Drive Bay Features

1

SSD release latch

3

Solid state drive activity LED (each bay)

2

Solid state drive fault LED (each bay)

4

Blanking panel over lower bays (when not in use)

Replacing Fan Modules

Each fan module contains two fans. See Figure 3-13 for the fan numbering.

You do not have to shut down or power off the system to replace fan modules because they are hot-swappable. Do not operate the system with a fan module removed for more than one minute.


Tip
Each fan module has a fault LED that lights amber if the fan module fails.

SUMMARY STEPS

  1. Slide the system out the front of the rack far enough so that you can open the fan compartment cover.
  2. Open the fan compartment cover as described in Opening the Main Chassis Top Covers.
  3. Remove the fan module:
  4. Install a new fan module:
  5. Close the fan compartment cover and then push the system back into the rack.

DETAILED STEPS

  Command or Action Purpose

Step 1

Slide the system out the front of the rack far enough so that you can open the fan compartment cover.

Caution

 

If you cannot safely view and access the component, remove the system from the rack.

Step 2

Open the fan compartment cover as described in Opening the Main Chassis Top Covers.

Step 3

Remove the fan module:

  1. Grasp the two latches on the top of the fan and pinch them toward the center.

  2. Lift the fan module straight out of the bay.

Step 4

Install a new fan module:

Note

 

The arrow on the fan module that indicates the air flow direction should point to the rear of the system.

  1. Align the fan module with the bay so that the connector on the bottom of the fan module is aligned with the socket on the floor of the chassis.

  2. Lower the fan module until it touches the socket, then push down firmly until the latch locks.

Step 5

Close the fan compartment cover and then push the system back into the rack.
Figure 13. Figure 3-13 Fan Modules (Top View)

1

Fan module fault LED

3

Air flow direction arrow pointing toward rear of system

2

Fan module release latches

Replacing a Server Node

The system can support one or two server nodes.

  • • Cisco IMC releases earlier than 2.0(13): If your S3260 system has only one server node, it must be installed in bay 1.

  • • Cisco IMC releases 2.0(13) and later: If your S3260 system has only one server node, it can be installed in either server bay.


Note

Whichever bay a server node is installed to, it must have a corresponding SIOC. That is, a server node in bay 1 must be paired with a SIOC in SIOC slot 1; a server node in bay 2 must be paired with a SIOC in SIOC bay 2. See Overview of Cisco UCS S3260 ArchitectureOverview of Cisco UCS S3260 Architecture, for more information.

To replace a server node, including exporting and importing its configuration, see the service note for your server node:

Replacing a Disk Expansion Tray

The system can support one optional disk expansion tray, in server bay 2 only.


Tip
The tray contains one status LED (see Figure 3-14 for the location and Table 3-3 for definitions of the LED states).
Figure 14. Figure 3-14 Disk Expansion Tray External Features (Four-Drive Version Shown)

1

Disk expansion tray status LED

2

Disk expansion tray ejector levers (two)

Note

The disk expansion tray is hot-swappable, which means that you can remove it without shutting down system power.

The disk expansion tray is accessed from the rear of the system, so you do not have to pull the system out from the rack.

SUMMARY STEPS

  1. Remove a disk expansion tray from the system:
  2. Remove any drives from the old tray and move them to your new disk expansion tray. Install each drive to the same position that it occupied in the old tray.

DETAILED STEPS

  Command or Action Purpose

Step 1

Remove a disk expansion tray from the system:

  1. Grasp the two ejector levers and pinch their latches to release the levers (see Figure 3-14).

  2. Rotate both levers to the outside at the same time to evenly disengage the tray from the midplane connectors.

  3. Pull the tray straight out from the system.

Step 2

Remove any drives from the old tray and move them to your new disk expansion tray. Install each drive to the same position that it occupied in the old tray.

Step 3

Install a new disk expansion tray (server bay 2 only):

  1. a. With the two ejector levers open, align the new tray with the empty bay.

  2. b. Push the tray into the bay until it engages with the midplane connectors.

  3. c. Rotate both ejector levers toward the center until they lay flat and their latches lock into the rear of the tray.

Replacing a System I/O Controller (SIOC)

The system can support up to two system I/O controllers (SIOCs).

For information about the management architecture and the chassis management controller (CMC) that is in each SIOC, see Management Architecture.

This section contains the following topics:


Note

If you are replacing a Version 02 SIOC (UCSC-S3260-SIOC) with a Version 03 SIOC (UCS-S3260-PCISIOC), use the procedure in Migrating From Migrating From Version 02 SIOC to Version 03 SIOC. Version 03 SIOCs require Cisco IMC 4.0(1n) or later and for UCS Manager-controlled systems, Cisco UCS Manager 4.0(1n) or later.


Note

The Version 03 SIOC is supported only with S3260 M5 or later server nodes.

Note

The Version 03 SIOC is supported only with S3260 M5 or later server nodes.

Note

The Version 03 SIOC is supported only with S3260 M5 or later server nodes.

Note

If you move a SIOC from one chassis to another, the SIOC’s CMC configuration is treated as incompatible and is automatically deleted. The CMC will sync with the active CMC configuration.

Figure 15. Figure 3-15 SIOC External Mechanical Features (Version 03 SIOC shown)

1

Lever thumbscrew

3

Right ejector lever.

2

Left ejector lever.
Replacing a SIOC in a Single-SIOC System

You do not have to slide the system out of the rack to remove the SIOC from the rear of the system. The SIOC can be replaced without powering off the chassis.

SUMMARY STEPS

  1. Export the CMC configuration from the SIOC.
  2. Export the adapter configuration from the SIOC.
  3. Power off the connected server nodes based on the scenario:
  4. Remove the SIOC from the system:
  5. Version 03 SIOC Only: If desired, transfer the adapter card and any NVMe SSDs from the existing Version 03 SIOC to the new Version 03 SIOC:
  6. Install the new SIOC:
  7. Wait for the CMC in the new SIOC to finish booting.
  8. Import the CMC configuration that you exported earlier to the new SIOC’s CMC.
  9. Import the adapter configuration that you exported earlier to the new SIOC’s CMC.
  10. Power on the server node that you powered off earlier.
  11. Update the SIOC firmware using the Host Upgrade Utility. This will ensure that the firmware version on the SIOC is compatible with the latest system firmware.

DETAILED STEPS

  Command or Action Purpose

Step 1

Export the CMC configuration from the SIOC.

Step 2

Export the adapter configuration from the SIOC.

Step 3

Power off the connected server nodes based on the scenario:

  1. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).

  2. If there are two NVMes in the SIOC you are removing, power off both server nodes.

See Shutting Down an Individual Server Node.

Step 4

Remove the SIOC from the system:

  1. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.

  2. Pull the SIOC from the system.

Step 5

Version 03 SIOC Only: If desired, transfer the adapter card and any NVMe SSDs from the existing Version 03 SIOC to the new Version 03 SIOC:

  1. Remove the cover from the existing Version 03 SIOC as described in Removing the System I/O Controller (SIOC) Cover.

  2. Remove the adapter card as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).

  3. Remove the NVMe SSDs as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).

  4. Remove the cover from the replacement Version 03 SIOC.

  5. Install the adapter card to the replacement SIOC as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).

  6. Install the NVME SSDs to the replacement SIOC as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).

  7. Install the cover to the replacement SIOC.

Step 6

Install the new SIOC:

  1. Push the SIOC into its bay until it stops against the internal backplane.

  2. Close the two ejector levers on the SIOC to fully engage its connector with the midplane connector.

  3. Tighten the thumbscrew on the SIOC ejector levers.

Step 7

Wait for the CMC in the new SIOC to finish booting.

Step 8

Import the CMC configuration that you exported earlier to the new SIOC’s CMC.

Step 9

Import the adapter configuration that you exported earlier to the new SIOC’s CMC.

Step 10

Power on the server node that you powered off earlier.

Step 11

Update the SIOC firmware using the Host Upgrade Utility. This will ensure that the firmware version on the SIOC is compatible with the latest system firmware.

See the Cisco Host Upgrade Utility User Guide For S3260 Storage Servers for instructions on updating the firmware.

Note

 

If you are using Cisco Card NIC mode, which uses the SIOC’s uplink interfaces to manage the system, you might need to configure the uplink’s port speed to either 4x10 Gbps or 40 Gbps to match your network installation. See Setting SIOC QSFP Port Speed.

Replacing a SIOC in a Dual-SIOC System

You do not have to slide the system out of the rack to remove the SIOC from the rear of the system. The SIOC can be replaced without powering off the chassis.

SUMMARY STEPS

  1. Set the CMC in the SIOC as the standby CMC, if it is not already.
  2. Export the adapter configuration from the SIOC.
  3. Power off the connected server nodes based on the scenario:
  4. Remove the SIOC from the system:
  5. Version 03 SIOC Only: If desired, transfer the adapter card and any NVMe SSDs from the existing Version 03 SIOC to the new Version 03 SIOC:
  6. Install the new SIOC:
  7. Wait for the CMC in the new SIOC to finish booting.
  8. Power on the server node that you powered off earlier.
  9. Update the SIOC firmware using the Host Upgrade Utility. This will ensure that the firmware version on the SIOC is compatible with the latest system firmware.
  10. Import the adapter configuration that you exported earlier to the new SIOC’s CMC.

DETAILED STEPS

  Command or Action Purpose

Step 1

Set the CMC in the SIOC as the standby CMC, if it is not already.

Step 2

Export the adapter configuration from the SIOC.

Step 3

Power off the connected server nodes based on the scenario:

  1. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).

  2. If there are two NVMes in the SIOC you are removing, power off both server nodes.

See Shutting Down an Individual Server Node.

Step 4

Remove the SIOC from the system:

  1. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.

  2. Pull the SIOC from the system.

Step 5

Version 03 SIOC Only: If desired, transfer the adapter card and any NVMe SSDs from the existing Version 03 SIOC to the new Version 03 SIOC:

  1. Remove the cover from the existing Version 03 SIOC as described in Removing the System I/O Controller (SIOC) Cover.

  2. Remove the adapter card as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).

  3. Remove the NVMe SSDs as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).

  4. Remove the cover from the replacement Version 03 SIOC.

  5. Install the adapter card to the replacement SIOC as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).

  6. Install the NVME SSDs to the replacement SIOC as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).

  7. Install the cover to the replacement SIOC.

Step 6

Install the new SIOC:

  1. Push the SIOC into its bay until it stops against the internal backplane.

  2. Close the two ejector levers on the SIOC to fully engage its connector with the midplane connector.

  3. Tighten the thumbscrew on the SIOC ejector levers.

Step 7

Wait for the CMC in the new SIOC to finish booting.

Step 8

Power on the server node that you powered off earlier.

The configuration in the active CMC is automatically synched with the standby CMC in the SIOC that you just installed.

Step 9

Update the SIOC firmware using the Host Upgrade Utility. This will ensure that the firmware version on the SIOC is compatible with the latest system firmware.

See the Cisco Host Upgrade Utility User Guide For S3260 Storage Servers for instructions on updating the firmware.

Note

 

If you are using Cisco Card NIC mode, which uses the SIOC’s uplink interfaces to manage the system, you might need to configure the uplink’s port speed to either 4x10 Gbps or 40 Gbps to match your network installation. See Setting Setting SIOC QSFP Port Speed.

Step 10

Import the adapter configuration that you exported earlier to the new SIOC’s CMC.
Setting SIOC QSFP Port Speed

Optional: Set the port speeds for the SIOC QSFP ports:

SUMMARY STEPS

  1. Connect a keyboard and console to the system or log in remotely and view a virtual KVM window.
  2. Reboot the system and press F8 when prompted to launch the Cisco IMC Configuration Utility.
  3. After the first utility screen is displayed, press F1 twice to go to the third utility screen, which has the settings for Adapter Port Speeds.
  4. Set the desired port speeds. “Adapter-1” is SIOC 1; “Adapter-2” is SIOC 2, if present.
  5. Press F10 to save your changes and exit the utility.

DETAILED STEPS

  Command or Action Purpose

Step 1

Connect a keyboard and console to the system or log in remotely and view a virtual KVM window.

Step 2

Reboot the system and press F8 when prompted to launch the Cisco IMC Configuration Utility.

Step 3

After the first utility screen is displayed, press F1 twice to go to the third utility screen, which has the settings for Adapter Port Speeds.

Step 4

Set the desired port speeds. “Adapter-1” is SIOC 1; “Adapter-2” is SIOC 2, if present.

Step 5

Press F10 to save your changes and exit the utility.

Updating SIOC Adapter Firmware in a System With Dual SIOCs But Only One Server Node

The recommended method for updating SIOC adapter firmware is to run the Cisco Host Upgrade Utility (HUU) on the server node that is associated with the SIOC. However, if a system has only one server node and two SIOCs, this method does not work to upgrade firmware on the SIOC that is not associated with a server node (SIOC 1 is associated with a server node in bay 1; SIOC 2 is associated with a server node in bay 2). In this case, you must manually upgrade the adapter firmware on the un-associated SIOC:

Using the Cisco IMC GUI Interface to Update SIOC Adapter Firmware

SUMMARY STEPS

  2. Activate the firmware:
  3. Reset the adapter in the SIOC:

DETAILED STEPS

  Command or Action Purpose

Step 1

  1. In the Admin menu, click Firmware Management.

  2. In the Component Column, select Adapter-SIOC1 or Adapter-SIOC2.

  3. Click Update. The Update Firmware dialog opens.

  4. Browse to select the firmware for the SIOC adapter.

  5. Click Install Firmware to begin download and installation.

Step 2

Activate the firmware:

  1. In the Component Column, select Adapter-SIOC1 or Adapter-SIOC2.

  2. Click Activate.

    Note

     
    You must reset the adapter to make the activated firmware version the running version. Resetting the adapter also causes the host to reset.

Step 3

Reset the adapter in the SIOC:

  1. In the Navigation pane, click the Networking menu.

  2. Click the Adapter Card tab. The General tab appears.

  3. Select Adapter-SIOC1 or Adapter-SIOC2.

  4. In the Actions area of the General tab, click Reset and then click Yes to confirm.

  5. The adapter and host reset. The activated firmware becomes the running version.
Using the Cisco IMC PMCLI Interface to Update SIOC Adapter Firmware

SUMMARY STEPS

  1. Update and activate the adapter firmware by using the following commands:
  2. Reset the adapter:

DETAILED STEPS

  Command or Action Purpose

Step 1

Update and activate the adapter firmware by using the following commands:

Example:
server1/chassis# update-adapter-fw tftp nnn.nnn.nnn.nnn /fw-image.bin activate 1
Adapter firmware update has started. Please check the status using "show adapter detail".
You have chosen to automatically activate the new firmware image. Please reset your
adapter after the update finishes.
 
Server# scope chassis
Server/chassis# update-adapter-fw <protocol> <remote server IP address> <image file path>
<activate|no-activate> [SIOC slot number]

Step 2

Reset the adapter:

Example:
server/chassis# adapter-reset SIOC1

Note

 

You must reset the adapter to make the activated firmware version the running version. Resetting the adapter also causes the host to reset. 

server/chassis# adapter-reset <SIOC1 or SIOC2>

The adapter and host reset. The activated firmware becomes the running version.

Replacing a Power Supply

The system requires four power supplies, which are redundant as 3+1.

Note Do not mix power supply types in the server. Both power supplies must be identical.

This section contains these procedures:

Replacing an AC Power Supply

To replace or install an AC power supply, follow these steps:


Note

You do not have to power off the system to replace up to one power supply because they are redundant as 3+1.

SUMMARY STEPS

  1. Remove the power supply that you are replacing or a blank panel from an empty bay (see Figure 3-16:
  2. Install a new power supply:

DETAILED STEPS

  Command or Action Purpose

Step 1

Remove the power supply that you are replacing or a blank panel from an empty bay (see Figure 3-16:

  1. Remove the power cord from the power supply that you are replacing.

  2. Grasp the power supply handle while pinching the release lever toward the handle.

  3. Pull the power supply out of the bay.

Step 2

Install a new power supply:

  1. Grasp the power supply handle and insert the new power supply into the empty bay.

  2. Push the power supply into the bay until the release lever locks.

  3. Connect the power cord to the new power supply.

  4. If you powered off the system, press and hold the system Power button for four seconds to return the system to main power mode.

    Figure 16. Figure 3-16 Removing and Replacing Power Supplies

    1

    Power supply release lever

    2

    Power supply handle
Installing a DC Power Supply (First-Time Installation)

Warning

A readily accessible two-poled disconnect device must be incorporated in the fixed wiring. Statement 1022

Warning

This product requires short-circuit (overcurrent) protection, to be provided as part of the building installation. Install only in accordance with national and local wiring regulations. Statement 1045


Warning

Installation of the equipment must comply with local and national electrical codes. Statement 1074

Caution

Before beginning this wiring procedure, turn off the DC power source from your facility’s circuit breaker to avoid electric shock hazard.

SUMMARY STEPS

  1. Turn off the DC power source from your facility’s circuit breaker to avoid electric shock hazard.
  2. Wire the non-terminated end of the cable to your facility’s DC power input source.
  3. Connect the terminated end of the cable to the socket on the power supply. The connector is keyed so that the wires align for correct polarity and ground, as shown in Figure 3-17.
  4. Return DC power from your facility’s circuit breaker.

DETAILED STEPS

  Command or Action Purpose

Step 1

Turn off the DC power source from your facility’s circuit breaker to avoid electric shock hazard.

Note

 

The required DC input cable is Cisco part CAB-48DC-40A-8AWG. This 3-meter cable has a 3-pin connector on one end that is keyed to the DC input socket on the power supply, as shown in Figure 3-17. The other end of the cable has no connector so that you can wire it to your facility’s DC power.

Step 2

Wire the non-terminated end of the cable to your facility’s DC power input source.

Step 3

Connect the terminated end of the cable to the socket on the power supply. The connector is keyed so that the wires align for correct polarity and ground, as shown in Figure 3-17.

Step 4

Return DC power from your facility’s circuit breaker.
Figure 17. Figure 3-17 1050 W, –48 VDC Power Supply and Cable

1

Keyed cable connector (CAB-48DC-40A-8AWG)

3

PSU status LED

See Rear-Panel LEDs and Buttons, page 3-4 for details.

2

Keyed DC input socket
Replacing a DC Power Supply

Note

You do not have to power off the system to replace up to one power supply because they are redundant as 3+1.

SUMMARY STEPS

  1. Remove the power supply that you are replacing or a blank panel from an empty bay:
  2. Install a new power supply:

DETAILED STEPS

  Command or Action Purpose

Step 1

Remove the power supply that you are replacing or a blank panel from an empty bay:

  1. Remove the CAB-48DC-40A-8AWG keyed cable connector from the power supply that you are replacing.

  2. b. Grasp the power supply handle while pinching the release lever toward the handle.

  3. c. Pull the power supply out of the bay.

Step 2

Install a new power supply:

  1. a. Grasp the power supply handle and insert the new power supply into the empty bay.

  2. b. Push the power supply into the bay until the release lever locks.

  3. c. Connect the CAB-48DC-40A-8AWG keyed cable connector to the new power supply.

  4. d. If you powered off the system, press and hold the system Power button for four seconds to return the system to main power mode.

Replacing an RTC Battery Inside the S3260 SIOC

The real-time clock (RTC) battery retains settings when the SIOC is disconnected from power. The battery type in the SIOC is CR1632. Cisco supports the industry-standard CR1632 battery, which can be purchased from most electronic stores.


Note

When the RTC battery is removed or it completely loses power, settings that were stored in the CMC of the SIOC are lost. You must reconfigure the CMC settings after installing a new battery.

SUMMARY STEPS

  1. Power off the connected server nodes based on the scenario:
  2. Remove the SIOC from the system:
  3. Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.
  4. Gently pry under the battery to lift it from its socket on the SIOC board.
  5. Insert the new battery into the socket and then press down until it sits flat.
  6. Replace the cover to the SIOC.
  7. Replace the SIOC to the system:
  8. Power on the server node that you powered off earlier.
  9. Reconfigure the settings of the CMC in this SIOC.

DETAILED STEPS

  Command or Action Purpose

Step 1

Power off the connected server nodes based on the scenario:

  1. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).

  2. If there are two NVMes in the SIOC you are removing, power off both server nodes.

See Shutting Down an Individual Server Node.

Step 2

Remove the SIOC from the system:

  1. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.

  2. Pull the SIOC from the system.

Step 3

Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.

Step 4

Gently pry under the battery to lift it from its socket on the SIOC board.

Note

 

The battery socket is in different locations in version 02 and Version 03 SIOCs. See the following figures.

Step 5

Insert the new battery into the socket and then press down until it sits flat.

Step 6

Replace the cover to the SIOC.

Step 7

Replace the SIOC to the system:

  1. Push the SIOC into its bay until it stops against the internal midplane.

  2. Close the two levers on the SIOC to fully engage th SIOC connector with its backplane.

  3. Tighten the thumbscrew on the SIOC levers.

Step 8

Power on the server node that you powered off earlier.

Step 9

Reconfigure the settings of the CMC in this SIOC.
Figure 18. Figure 3-18 RTC Battery Location Inside the Version 02 S3260 SIOC (UCS-S3260-SIOC)

1

Battery socket on version 02 SIOC board, with CR1632 battery
Figure 19. Figure 3-19 RTC Battery Location Inside the Version 03 S3260 SIOC (UCS-S3260-PCISIOC)

1

Battery socket on Version 03 SIOC board, with CR1632 battery

Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only)

The Version 03 SIOC uses a removable PCIe adapter card that plugs into a horizontal socket. (The version 02 SIOC uses an embedded chip and so does not have a removable adapter card.)

Cisco VIC Cards Supported in Version 03 SIOC (UCSC-S3260-PCISIOC)

At this time, the following Cisco Virtual Interface Cards (VICs) are supported in the Version 03 SIOC:

  • • Cisco UCS VIC 1455 (UCSC-PCIE-C25Q-04)

  • • Cisco UCS VIC 1495 (UCSC-PCIE-C100-04)


Note

If your S3260 system has two Version 03 SIOCs, the adapter card in each must be identical. Do not mix adapter cards between Version 03 SIOCs.

SUMMARY STEPS

  1. Power off the connected server nodes based on the scenario:
  2. Remove the SIOC from the system:
  3. Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover, page 3-13.
  4. Remove an existing adapter card:
  5. Install a new adapter card:
  6. Turn the SIOC over and reinstall the SIOC cover. Install the six screws that secure the cover.
  7. Replace the SIOC to the system:
  8. Power on the server node that you powered off earlier.

DETAILED STEPS

  Command or Action Purpose

Step 1

Power off the connected server nodes based on the scenario:

  1. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2)

  2. If there are two NVMes in the SIOC you are removing, power off both server nodes.

See Shutting Down an Individual Server Node, page 3-10.

Step 2

Remove the SIOC from the system:

  1. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.

  2. Pull the SIOC from the system.

Step 3

Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover, page 3-13.

Step 4

Remove an existing adapter card:

  1. Remove the SIOC side panel that is closest to the adapter card. Turn the SIOC upside down and remove the three screws that secure the side panel from the underside of the SIOC.

  2. Turn the SIOC over.

  3. Slide the adapter card horizontally to disconnect its edge connector from the socket.

Step 5

Install a new adapter card:

  1. Set the card in place on the SIOC floor. Align the card’s edge connector with the socket.

  2. Gently slide the card into the socket, pushing evenly on both ends of the card. Stop when the card is fully in the socket and its rear-panel tab sits flat.

  3. Return the side panel to the SIOC. Set the side panel in place so that its edge sits on top of the SIOC floor and its three screw holes align with those on the SIOC.

  4. Turn the SIOC upside down and install the three screws that secure the side panel.

Step 6

Turn the SIOC over and reinstall the SIOC cover. Install the six screws that secure the cover.

Step 7

Replace the SIOC to the system:

  1. Push the SIOC into its bay until it stops against the internal midplane.

  2. Close the two levers on the SIOC to fully engage the SIOC connector with its backplane.

  3. Tighten the thumbscrew on the SIOC levers.

Step 8

Power on the server node that you powered off earlier.
Figure 20. Figure 3-20 Adapter Card in Version 03 SIOC (UCS-S3260-PCISIOC)

1

Horizontal socket for adapter card

3

Location of three screws that secure the side

panel (access from the underside of the SIOC)

2

Removable SIOC side panel (three screws on underside)

Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only)

SIOC NVMe SSD Population Rules

The Version 03 SIOC can contain up to two 2.5-inch NVMe SSDs in a sled assembly. You must remove the sled when installing or replacing the SSDs.

Each NVMe SSD slot in a SIOC is numbered for the server that supports it (SERVER 1 NVME or SERVER 2 NVME).

  • Populate an NVMe SSD in SERVER 1 NVME only if you have a server node in upper server bay 1.

  • Populate an NVME SSD in SERVER 2 NVME only if you have a server node in lower server bay 2.


Note

If your system configuration has an M5 server node in lower server bay 2 and an I/O expander in upper server bay 1, you cannot populate an NVMe SSD in the SERVER 1 NVME slot in either SIOC. See Figure 3-21and Figure 3-22 for illustrations of the system connectivity and labeling.

Figure 21. Figure 3-21 NVMe SSD Connectivity in Version 03 SIOC (UCS-S3260-PCISIOC)
Procedure

SUMMARY STEPS

  1. Power off the connected server nodes based on the scenario:
  2. Remove the SIOC from the system:
  3. Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.
  4. Remove the NVMe SSD sled assembly from the SIOC.

DETAILED STEPS

  Command or Action Purpose

Step 1

Power off the connected server nodes based on the scenario:

  1. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).

  2. If there are two NVMes in the SIOC you are removing, power off both server nodes.

See Shutting Down an Individual Server Node.

Step 2

Remove the SIOC from the system:

  1. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.

  2. Pull the SIOC from the system.

Step 3

Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.

Note

 

The NVMe SSD sled (UCS-S3260-NVMSLD2) in the Version 03 SIOC is not interchangeable with the NVMe SSD sled (UCS-S3260-NVMSLD1) in the S3260 M5 server node.

Step 4

Remove the NVMe SSD sled assembly from the SIOC.

Loosen the two captive thumbscrews that secure the sled to the SIOC floor and then pull the sled so that the connectors on any installed SSDs disengage from the two sockets on the SIOC board.

Figure 22. Figure 3-22 NVMe SSDs in Version 03 SIOC (UCS-S3260-PCISIOC)

1

NVMe sled-assembly thumbscrews

3

SERVER 2 NVME

Populate this slot only when you have a server node in lower server bay 2.

2

SERVER 1 NVME

Populate this slot only when you have a server node in upper server bay 1.

Replacing an S3260 Chassis

This procedure is for replacing an S3260 chassis and transferring all components to the new chassis.

SUMMARY STEPS

  1. Export the component configurations (CMC, BMC, VIC adapters) and save them to a local computer using the import-export command mode.
  2. Perform a graceful shutdown of all server nodes.
  3. Disconnect all power cables from all power supplies to fully remove power from the chassis.
  4. Remove each server node and install each to the same bay position in the new chassis.
  5. If the system has a disk expansion tray in place of a server node, remove it with its disks in place and install the tray with disks to the same bay position in the new chassis.
  6. Remove each SIOC and install each to the same bay position in the new chassis.
  7. Remove each rear-panel boot SSD and install each to the same bay position in the new chassis.
  8. Remove each power supply and install each to the same bay position in the new chassis.
  9. Open the main chassis top cover on the old and new chassis.
  10. Remove each top-loading hard drive and install each to the same bay position in the new chassis.
  11. Open the fan compartment cover on the old and new chassis.
  12. Remove each cooling fan and install each to the same bay position in the new chassis.
  13. If you have not done so already, remove the empty chassis from your rack.
  14. Close all covers on the new chassis.
  15. Install the new chassis to your rack.
  16. Power on, connect cables to the system, and log in to the Cisco IMC Configuration Utility as described in Connecting the System and Opening the Setup Utility.
  17. Configure the network settings for the system as described in Setting Up the System Using the Cisco IMC Configuration Utility.
  18. After the system has connection to the network, use the Host Upgrade Utility to update the system to your desired level and ensure that all components in the system are running compatible software levels.
  19. Import the component configurations that you exported in step 1.
  20. Verify that all configurations imported successfully.
  21. Verify that chassis health is optimal.

DETAILED STEPS

  Command or Action Purpose

Step 1

Export the component configurations (CMC, BMC, VIC adapters) and save them to a local computer using the import-export command mode.

Refer to the Cisco UCS Integrated Management Controller CLI Configuration Guide for S3260 Storage Servers. See the chapter, Server Utilities and the section, Exporting and Importing the Cisco IMC and BMC Configuration.

Step 2

Perform a graceful shutdown of all server nodes.

See Shutting Down an Individual Server Node for more information.

Step 3

Disconnect all power cables from all power supplies to fully remove power from the chassis.

Step 4

Remove each server node and install each to the same bay position in the new chassis.

If a server node has an I/O expander attached, move the node and expander attached together.

See the service note for your server node version for more information: Server Node Service Notes.

Step 5

If the system has a disk expansion tray in place of a server node, remove it with its disks in place and install the tray with disks to the same bay position in the new chassis.

See Replacing a Disk Expansion Tray for more information.

Step 6

Remove each SIOC and install each to the same bay position in the new chassis.

See Replacing a System I/O Controller (SIOC)for more information.

Step 7

Remove each rear-panel boot SSD and install each to the same bay position in the new chassis.

See Replacing SAS/SATA Solid State Drives in the Rear Panel Bays for more information.

Step 8

Remove each power supply and install each to the same bay position in the new chassis.

See Replacing a Power Supply for more information.

Step 9

Open the main chassis top cover on the old and new chassis.

See Opening the Main Chassis Top Covers for more information.

Step 10

Remove each top-loading hard drive and install each to the same bay position in the new chassis.

See Replacing HDDs or SSDs in the Internal Drive Compartment for more information.

Step 11

Open the fan compartment cover on the old and new chassis.

Step 12

Remove each cooling fan and install each to the same bay position in the new chassis.

See Replacing Fan Modules for more information.

Step 13

If you have not done so already, remove the empty chassis from your rack.

Step 14

Close all covers on the new chassis.

Step 15

Install the new chassis to your rack.

Caution

 

This system weighs approximately 190 pounds (86 kilograms) when fully loaded with components. We recommend that you use a mechanical lift when lifting the system. Attempting this procedure without a lift could result in personal injury or equipment damage. Although not required, consider temporarily removing components such as hard drives while you move the system.

See Installing the System in a Rack for instructions.

Note

 

When a SIOC is moved to a new chassis, the CMC settings are reset to the factory defaults and so you must perform initial setup as you would for a new system.

Step 16

Power on, connect cables to the system, and log in to the Cisco IMC Configuration Utility as described in Connecting the System and Opening the Setup Utility.

Step 17

Configure the network settings for the system as described in Setting Up the System Using the Cisco IMC Configuration Utility.

Step 18

After the system has connection to the network, use the Host Upgrade Utility to update the system to your desired level and ensure that all components in the system are running compatible software levels.

See the Cisco Host Upgrade Utility User Guide For S3260 Storage Servers for instructions.

Step 19

Import the component configurations that you exported in step 1.

Refer to the Cisco UCS Integrated Management Controller CLI Configuration Guide for S3260 Storage Servers. See the chapter Server Utilities and the section Exporting and Importing the Cisco IMC and BMC Configuration.

Step 20

Verify that all configurations imported successfully.

Step 21

Verify that chassis health is optimal.

Basic Troubleshooting: Reseating a SAS/SATA Drive

Sometimes it is possible for a false positive UBAD error to occur on SAS/SATA HDDs installed in the server.

  • Only drives that are managed by the UCS MegaRAID controller are affected.

  • Drives can be affected regardless where they are installed in the server (front-loaded, rear-loaded, and so on).

  • Both SFF and LFF form factor drives can be affected.

  • Drives installed in all Cisco UCS C-Series servers with M3 processors and later can be affected.

  • Drives can be affected regardless of whether they are configured for hotplug or not.

  • The UBAD error is not always terminal, so the drive is not always defective or in need of repair or replacement. However, it is also possible that the error is terminal, and the drive will need replacement.

Before submitting the drive to the RMA process, it is a best practice to reseat the drive. If the false UBAD error exists, reseating the drive can clear it. If successful, reseating the drive reduces inconvenience, cost, and service interruption, and optimizes your server uptime.


Note

Reseat the drive only if a UBAD error occurs. Other errors are transient, and you should not attempt diagnostics and troubleshooting without the assistance of Cisco personnel. Contact Cisco TAC for assistance with other drive errors.

To reseat the drive, see Reseating a SAS/SATA Drive.

Reseating a SAS/SATA Drive

Sometimes, SAS/SATA drives can throw a false UBAD error, and reseating the drive can clear the error.

Use the following procedure to reseat the drive.


Caution

This procedure might require powering down the server. Powering down the server will cause a service interruption.

Before you begin

Before attempting this procedure, be aware of the following:

  • Before reseating the drive, it is a best practice to back up any data on it.

  • When reseating the drive, make sure to reuse the same drive bay.

    • Do not move the drive to a different slot.

    • Do not move the drive to a different server.

    • If you do not reuse the same slot, the Cisco management software (for example, Cisco IMM) might require a rescan/rediscovery of the server.

  • When reseating the drive, allow 20 seconds between removal and reinsertion.

Procedure

Step 1

Attempt a hot reseat of the affected drive(s). Choose the appropriate option:

  1. For a rear-loading drive, see Replacing SAS/SATA Solid State Drives in the Rear Panel Bays

    Note

     

    While the drive is removed, it is a best practice to perform a visual inspection. Check the drive bay to ensure that no dust or debris is present. Also, check the connector on the back of the drive and the connector on the inside of the server for any obstructions or damage.

    Also, when reseating the drive, allow 20 seconds between removal and reinsertion.

Step 2

During boot up, watch the drive's LEDs to verify correct operation.

See Front-Panel LEDs.

Step 3

If the error persists, cold reseat the drive, which requires a server power down. Choose the appropriate option:

  1. Use your server management software to gracefully power down the server.

    See the appropriate Cisco management software documentation.

  2. If server power down through software is not available, you can power down the server by pressing the power button.

    See Shutting Down a Server Node By Using the Power Button on the Server Node.

  3. Reseat the drive as documented in Step 1.

  4. When the drive is correctly reseated, restart the server, and check the drive LEDs for correct operation as documented in Step 2.

Step 4

If hot and cold reseating the drive (if necessary) does not clear the UBAD error, choose the appropriate option:

  1. Contact Cisco Systems for assistance with troubleshooting.

  2. Begin an RMA of the errored drive.