Storage Controller Information

This appendix contains information about the supported RAID and HBA storage controllers, including the embedded software RAID controller.

Supported Storage Controllers

Each S3260 M5 server node supports a single RAID or HBA controller card.

Each server node also includes an embedded MegaRAID controller that can control two rear-panel SATA solid state drives (SSDs) in a RAID 0 or 1 configuration. This embedded software RAID is available only when the server node has the UCS S3260 12G Dual Pass-Through Controller (UCS-S3260-DHBA) installed.

Table 1. S3260 M5 Storage Controller Options

Controller

Maximum Drives Controlled

RAID Levels

Optional Supercap Backup?

Embedded software RAID (PCH SATA)

(Requires that UCS-S3260-DHBA is installed in the server node.)

2 rear-panel SATA SSDs

0, 1, 10

No

Cisco UCS S3260 Dual RAID Controller

UCS-S3260-DRAID

66

0, 1, 5, 6, 10, 50, 60

Yes

Cisco UCS S3260 Dual Pass-Through Controller

UCS-S3260-DHBA

66

Not applicable

No

Cisco UCS S3260 Dual RAID Controller Information

The Cisco UCS S3260 Dual RAID Controller (UCS-S3260-DRAID) is based on the Broadcom 3316 SAS/SATA, 16-port RAID-on-chip (RoC). This controller is divided into two boards: a base board and a power board. The power board supplies power to the base board.

This RAID controller has the following features:

  • Provides a sixteen-port 12Gb/s SAS and 6Gb/s SATA interface.

  • Provides a 40-bit or a 72-bit 1866-MHz DDR3/3L SDRAM/MRAM interface with a hardware RAID assist-engine for parity calculations.

  • Provides a full-featured hardware-based RAID solution that supports RAID levels 0, 1, 5, 6, 10, 50, and 60.

  • Increases system performance and provides fault-tolerant data storage.

  • Supports data striping across multiple disks, which reduces disk access time because multiple disks simultaneously read or write data.

  • Backs up data with either data mirroring or a parity block. You can use either backup method to recover lost data in the event of a disk failure.

  • Maximum 50 W power consumption.

Cisco UCS S3260 Dual Pass-Through Controller Information

The Cisco UCS S3260 Dual Pass-Through Controller (UCS-S3260-DHBA) is based on the Broadcom 3316 SAS/SATA, 16-port RAID-on-chip (RoC). This controller is divided into two boards: a base board and a power board. The power board supplies power to the base board.

This pass-through controller has the following features:

  • Dual Broadcom SAS3316-based subsystems.

  • Dual 8x SAS-3 lanes (12G, 6G, 3G, and SATA), one for each subsystem.

  • Dual 8x PCIe Gen-3 lanes via the server-board mezzanine connector, one for each subsystem.

  • Maximum 50 W power consumption.

Best Practices For Configuring RAID Controllers

This section contains recommendations for best practices when configuring RAID controllers.

4K Sector Format Drives

Do not configure 4K sector format and 512-byte sector format drives as part of the same RAID volume.

Storage Controller Card Firmware Compatibility

Firmware on the storage controller (RAID or HBA) must be verified for compatibility with the current Cisco IMC and BIOS versions that are installed on the server. If not compatible, upgrade or downgrade the storage controller firmware using the Host Upgrade Utility (HUU) for your firmware release to bring it to a compatible level.

See the HUU guide for your Cisco IMC release for instructions on downloading and using the utility to bring server components to compatible levels: HUU Guides.

Choosing Between RAID 0 and JBOD

The RAID controller supports JBOD mode (non-RAID) on physical drives that are in pass-through mode and directly exposed to the OS. We recommended that you use JBOD mode instead of individual RAID 0 volumes when possible.

RAID 5/RAID 6 Volume Creation

The RAID controller allows you to create a large RAID 5 or 6 volume by including all the drives in the system in a spanned array configuration (RAID 50/RAID 60). Where possible, we recommend you create multiple, smaller RAID 5 or 6 volumes with fewer drives per RAID array. This provides redundancy and reduces the operations time for initialization, RAID rebuilds, and other operations.

Choosing I/O Policy

The I/O policy applies to reads on a specific virtual drive. It does not affect the read-ahead cache. RAID volumes can be configured in two types of I/O policies. These are:

  • Cached I/O—In this mode, all reads are buffered in cache memory. Cached I/O provides faster processing.

  • Direct I/O—In this mode, reads are not buffered in cache memory. Data is transferred to the cache and the host concurrently. If the same data block is read again, it comes from cache memory. Direct I/O makes sure that the cache and the host contain the same data.

Although Cached I/O provides faster processing, it is useful only when the RAID volume has a small number of slower drives. With the S3260 4-TB SAS drives, Cached I/O has not shown any significant advantage over Direct I/O. Instead, Direct I/O has shown better results over Cached I/O in a majority of I/O patterns. We recommended you use Direct I/O (the default) in all cases.

Background Operations (BGOPs)

The RAID controller conducts different background operations like Consistency Check (CC), Background Initialization (BGI), Rebuild (RBLD), Volume Expansion & Reconstruction (RLM), and Patrol Real (PR)

While these BGOPs are expected to limit their impact to I/O operations, there have been cases of higher impact during some of the I/O operations such as Format. In these cases, both the I/O operation and the BGOPs may take more time to complete. In such cases, we recommend you limit concurrent BGOPs and other intensive I/O operations where possible.

Embedded Software RAID

Each server node includes an embedded MegaRAID controller that can control two rear-panel SATA solid state drives (SSDs) in a RAID 0 or 1 configuration. This embedded software RAID is available only when the server node has the UCS S3260 12G Dual Pass-Through Controller (UCS-S3260-DHBA) installed. When this HBA is installed, the two rear-panel SSDs can be controlled through software RAID mode or AHCI mode, when selected in the server BIOS.


Note

Embedded software RAID is not available when the HW RAID Cisco UCS S3260 Dual RAID Controller (UCS-S3260-DRAID) is installed. In that case, the rear-panel SSDs are controlled by hardware RAID.


Note
The VMware ESX/ESXi operating system is not supported with the embedded SATA MegaRAID controller in SW RAID mode. You can use VMWare in AHCI mode.

Note

The Microsoft Windows Server 2016 Hyper-V hypervisor is supported for use with the embedded MegaRAID controller in SW RAID mode, but all other hyperviors are not supported. All Hypervisors are supported in AHCI mode.


Note

The embedded RAID controller in server node 1 can control the upper two rear-panel SSDs; the embedded RAID controller in server node 2 can control the lower two rear-panel SSDs.

Accessing the Software RAID Configuration Utility

To configure RAID settings for the embedded SATA RAID controllers, use the utility that is built into the BIOS.

Procedure

Step 1

Boot the server and press F2 when prompted to enter the BIOS Setup utility.

Step 2

Choose the Advanced tab.

Step 3

Select the utility: LSI Software RAID Configuration Utility (SATA)

Enabling SATA Mode For the Embedded Controller

This procedure uses the server's BIOS Setup Utility

Procedure

Step 1

Set the SATA mode:

  1. Boot the server and press F2 when prompted to enter the BIOS Setup utility.

  2. Choose the Advanced tab, and then choose LOM and PCIe Slots Configuration.

  3. Choose one of the options from the dialog:

    • LSI SW RAID—Enable the embedded SATA RAID controller for control of rear-panel SATA SSDs.

      Note 

      This menu option does not appear when the server is set to boot in Legacy mode (UEFI mode is required). To change the boot mode, use the BIOS setting for Boot Options > Boot Mode.

    • AHCI—Enable control of the rear-panel SSDs by AHCI through your OS rather than the embedded RAID controller.

    • Disabled—Disable the embedded RAID controller.
Step 2

Press F10 to save your changes and exit the utility.

Installing LSI MegaSR Drivers For Windows and Linux


Note
The required drivers for this controller are already installed and ready to use. However, if you will use this controller with Windows or Linux, you must download and install additional drivers for those operating systems.

This section explains how to install the LSI MegaSR drivers for the following supported operating systems:

  • Microsoft Windows Server

  • Red Hat Enterprise Linux (RHEL)

  • SUSE Linux Enterprise Server (SLES)

For the specific supported OS versions, see the Hardware and Software Compatibility Matrix for your server release.

Downloading the MegaSR Drivers

The MegaSR drivers are included in the driver ISO for your server and OS.

Procedure
Step 1

Find the drivers ISO file download for your server online and download it to a temporary location on your workstation:

  1. See the following URL: http://www.cisco.com/cisco/software/navigator.html.

  2. Type the name of your server in the Select a Product search field and then press Enter.

  3. Click Unified Computing System (UCS) Drivers.

  4. Click the release number that you are downloading.

  5. Click the Download icon to download the drivers ISO file.
Step 2

Continue through the subsequent screens to accept the license agreement and then browse to a location where you want to save the driver ISO file.

Microsoft Windows Server Drivers

Installing Microsoft Windows Server Drivers

The Windows Server operating system automatically adds the driver to the registry and copies the driver to the appropriate directory.

Before you begin

Before you install this driver on an embedded controller, you must configure a RAID drive group on the embedded controller for the drives where you will install the OS.

To access the configuration utility, open the BIOS Setup Utility, go to the Advanced tab, and then choose the utility for the embedded controller: LSI Software RAID Configuration Utility (SATA)

Procedure
Step 1

Download the Cisco UCS C-Series drivers’ ISO, as described in Downloading the MegaSR Drivers.

Step 2

Prepare the drivers on a USB thumb drive:

  1. Burn the ISO image to a disk.

  2. Browse the contents of the drivers folders to the location of the embedded MegaRAID drivers: /<OS>/Storage/Intel/C600/

  3. Expand the Zip file, which contains the folder with the MegaSR driver files.

  4. Copy the expanded folder to a USB thumb drive.
Step 3

Start the Windows driver installation using one of the following methods:

  • To install from local media, connect an external USB DVD drive to the server (if the server does not have a DVD drive installed) and then insert the first Windows installation disk into the DVD drive. Skip to Step 6.

  • To install from remote ISO, log in to the server’s Cisco IMC interface and continue with the next step.
Step 4

Launch a Virtual KVM console window and click the Virtual Media tab.

  1. Click Add Image and browse to select your remote Windows installation ISO file.

  2. Check the check box in the Mapped column for the media that you just added, and then wait for mapping to complete.

Step 5

Power cycle the server.
Step 6

Press F6 when you see the F6 prompt during bootup. The Boot Menu window opens.

Step 7

On the Boot Manager window, choose the physical disk or virtual DVD and press Enter. The Windows installation begins when the image is booted.

Step 8

Press Enter when you see the prompt, “Press any key to boot from CD.”

Step 9

Observe the Windows installation process and respond to prompts in the wizard as required for your preferences and company standards.

Step 10

When Windows prompts you with “Where do you want to install Windows,” install the drivers for embedded MegaRAID:

  1. Click Load Driver. You are prompted by a Load Driver dialog box to select the driver to be installed.

  2. Connect the USB thumb drive that you prepared in Step 3 to the target server.

  3. On the Windows Load Driver dialog, click Browse.

  4. Use the dialog box to browse to the location of the drivers folder on the USB thumb drive, and then click OK.

    Windows loads the drivers from the folder and when finished, the driver is listed under the prompt, “Select the driver to be installed.”

  5. Click Next to install the drivers.

Updating Microsoft Windows Server Drivers
Procedure
Step 1

Click Start, point to Settings, and then click Control Panel.

Step 2

Double-click System, click the Hardware tab, and then click Device Manager. Device Manager starts.

Step 3

In Device Manager, double-click SCSI and RAID Controllers, right-click the device for which you are installing the driver, and then click Properties.

Step 4

On the Driver tab, click Update Driver to open the Update Device Driver wizard, and then follow the wizard instructions to update the driver.

Linux Drivers

Downloading the Driver Image File

See Downloading the MegaSR Drivers for instructions on downloading the drivers. The Linux driver is included in the form of dud-[driver version].img, which is the boot image for the embedded MegaRAID stack.


Note
The LSI MegaSR drivers that Cisco provides for RHEL and SLES are for the original GA versions of those distributions. The drivers do not support updates to those OS kernels.
Preparing Physical Thumb Drive for Linux

This topic describes how to prepare physical Linux thumb drive from the driver image files.

This procedure requires a CD or DVD drive that you can use to burn the ISO image to disk; and a USB thumb drive.

Alternatively, you can mount the dud.img file as a virtual floppy disk, as described in the installation procedures.

For RHEL and SLES, you can use a driver disk utility to create disk images from image files.

Procedure
Step 1

Download the drivers ISO, as described in Downloading the MegaSR Drivers and save it to your Linux system.

Step 2

Extract the dud.img or dd.iso driver file:

Note 

For RHEL 7.1 and later, there is no dud.img file--the driver is contained in a dd.iso file.

  1. Burn the Cisco UCS C-Series Drivers ISO image to a disc.

  2. Browse the contents of the drivers folders to the location of the embedded MegaRAID drivers: /<OS>/Storage/Intel/C600-M5/

  3. Expand the Zip file, which contains the folder with the driver files.
Step 3

Copy the driver update disk image dud-[driver version].img (or dd.iso) to your Linux system.

Step 4

Insert a blank USB thumb drive into a port on your Linux system.
Step 5

Create a directory and mount the dud.img or dd.iso image to that directory:

Example:
mkdir <destination_folder>
mount -oloop <driver_image> <destination_folder>
Step 6

Copy the contents in the directory to your USB thumb drive.
Installing the Red Hat Enterprise Linux Driver

For the specific supported OS versions, see the Hardware and Software Compatibility Matrix for your server release.

This topic describes the fresh installation of the RHEL device driver on systems that have the embedded MegaRAID stack.


Note

If you use an embedded RAID controller with Linux, both the pSATA and the sSATA controller must be set to LSI SW RAID mode.

Before you begin

Before you install this driver on an embedded controller, you must configure a RAID drive group on the embedded controller.

To access the configuration utility, open the BIOS Setup Utility, go to the Advanced tab, and then choose the utility for the embedded controller: LSI Software RAID Configuration Utility (SATA)

Procedure
Step 1

Prepare the dud.img (or .iso) file using one of the following methods:

Note 

For RHEL 7.1 and later, there is no dud.img file--the driver is contained in a dd.iso file.
Step 2

Extract the dud.img (or dd.iso) file:

  1. Burn the Cisco UCS C-Series Drivers ISO image to a disk.

  2. Browse the contents of the drivers folders to the location of the embedded MegaRAID drivers: /<OS>/Storage/Intel/C600-M5/

  3. Copy the dud-<driver version>.img (or dd.iso) file to a temporary location on your workstation.

  4. If you are using RHEL 7.x, rename the saved dd.iso to dd.img.

    Note 

    If you are using RHEL 7.x, renaming the dd.iso file to dd.img simplifies this procedure and saves time. The Cisco UCS virtual drive mapper can map only one .iso at a time, and only as a virtual CD/DVD. Renaming the file to dd.img allows you to mount the RHEL installation ISO as a virtual CD/DVD and the renamed dd.img as a virtual floppy disk or removable disk at the same time. This avoids the steps of unmounting and remounting the RHEL ISO when the dd.iso driver file is prompted for.

Step 3

Start the Linux driver installation using one of the following methods:

  • To install from local media, connect an external USB CD/DVD drive to the server and then insert the first RHEL installation disk into the drive. Then continue with Step 5.

  • To install from virtual disk, log in to the server’s Cisco IMC interface. Then continue with the next step.
Step 4

Launch a Virtual KVM console window and click the Virtual Media tab.

  1. Click Add Image and browse to select your remote RHEL installation ISO image.

    Note 

    An .iso file can be mapped only as a virtual CD/DVD.

  2. Click Add Image again and browse to select your RHEL 6.x dud.img or the RHEL 7.x dd.img file that you renamed in step 2.

    Note 

    Map the .img file as a virtual floppy disk or virtual removable disk.

  3. Check the check boxes in the Mapped column for the media that you just added, then wait for mapping to complete.

Step 5

Power-cycle the target server.
Step 6

Press F6 when you see the F6 prompt during bootup. The Boot Menu window opens.

Note 

Do not press Enter in the next step to start the installation. Instead, press e to edit installation parameters.

Step 7

On the Boot Menu window, use the arrow keys to select Install Red Hat Enterprise Linux and then press e to edit installation parameters.

Step 8

Append one of the following blacklist commands to the end of the line that begins with linuxefi:

  • For RHEL 6.x (32- and 64-bit), type:

    linux dd blacklist=isci blacklist=ahci nodmraid noprobe=<atadrive number>

    Note 

    The noprobe values depend on the number of drives. For example, to install RHEL 6.x on a RAID 5 configuration with three drives, type:

    Linux dd blacklist=isci blacklist=ahci nodmraid noprobe=ata1 noprobe=ata2

  • For RHEL 7.x (32- and 64-bit), type:

    linux dd modprobe.blacklist=ahci nodmraid
Step 9

Optional: To see full, verbose installation status steps during installation, delete the Quiet parameter from the line.

Step 10

On the Boot Menu window, press Ctrl+x to start the interactive installation.

Step 11

Below Driver disk device selection, select the option to install your driver .img file. (Type r to refresh the list if it is not populated.)

Note 

The installer recognizes the driver file as an .iso file, even though you renamed it to dd.img for mapping.

Type the number of the driver device ISO in the list. Do not select the RHEL ISO image. In the following example, type 6 to select device sdb: 

5) sr0 iso9660 RHEL-7.6\x20Server.x
6) sdb iso9660 CDROM
# to select, ‘r’ - refresh, or ‘c’ -continue: 6

The installer reads the driver file and lists the drivers.
Step 12

Under Select drivers to install, type the number of the line that lists the megasr driver. In the following example, type 1: 

1) [ ] /media/DD-1/rpms/x86_61/kmod-megasr-18.01.2010.1107_e17.6-1.x86_61.rpm
# to toggle selection, or ‘c’ -continue: 1

Your selection is displayed with an X in brackets.

1) [X] /media/DD-1/rpms/x86_61/kmod-megasr-18.01.2010.1107_e17.6-1.x86_61.rpm
Step 13

Type c to continue.

Step 14

Follow the RHEL installation wizard to complete the installation.
Step 15

When the wizard’s Installation Destination screen is displayed, ensure that LSI MegaSR is listed as the selection. If it is not listed, the driver did not load successfully. In that case, select Rescan Disc.

Step 16

After the installation completes, reboot the target server.
Installing the SUSE Linux Enterprise Server Driver

For the specific supported OS versions, see the Hardware and Software Compatibility Matrix for your server release.

This topic describes the fresh installation of the SLES driver on systems that have the embedded MegaRAID stack.


Note

If you use an embedded RAID controller with Linux, both the pSATA and the sSATA controller must be set to LSI SW RAID mode.

Before you begin

Before you install this driver on an embedded controller, you must configure a RAID drive group on the embedded controller.

To access the configuration utility, open the BIOS Setup Utility, go to the Advanced tab, and then choose the utility for the embedded controller: LSI Software RAID Configuration Utility (SATA)

Procedure
Step 1

Prepare the dud.img file using one of the following methods:

Step 2

Extract the dud.img file that contains the driver:

  1. Burn the ISO image to a disk.

  2. Browse the contents of the drivers folders to the location of the embedded MegaRAID drivers: /<OS>/Storage/Intel/C600-M5/...

  3. Within the SLES folder for your version, the dud-<driver version>.img file is packaged in a compressed .gz file. Extract the .img file from the .gz file.

  4. Copy the dud-<driver version>.img file to a temporary location on your workstation.

Step 3

Start the Linux driver installation using one of the following methods:

  • To install from local media, connect an external USB DVD drive to the server and then insert the first SLES installation disk into the drive. Then continue with Step 5.

  • To install from remote ISO, log in to the server’s Cisco IMC interface. Then continue with the next step.
Step 4

Launch a Virtual KVM console window and click the Virtual Media tab.

  1. Click Add Image and browse to select your remote SLES installation ISO file.

  2. Click Add Image again and browse to select your dud-<driver version>.img file.

  3. Check the check boxes in the Mapped column for the media that you just added, then wait for mapping to complete.

Step 5

Power-cycle the target server.
Step 6

Press F6 when you see the F6 prompt during bootup. The Boot Menu window opens.

Step 7

On the Boot Manager window, select the physical or virtual SLES installation ISO and press Enter.

The SLES installation begins when the image is booted.
Step 8

When the first SLES screen appears, select Installation.

Step 9

Press e to edit installation parameters.

Step 10

Append the following parameter to the end of the line that begins with linuxefi:

brokenmodules=ahci
Step 11

Optional: To see detailed status information during the installation, add the following parameter to the line that begins with linuxefi:

splash=verbose
Step 12

Press Ctrl+x to start the installation.

The installation proceeds. The installer finds the LSI driver automatically in the dud-<driver version>.img file that you provided. With verbose status messages, you see the driver being installed when LSI MegaRAID SW RAID Module is listed.

Step 13

Follow the SLES installation wizard to complete the installation. Verify installation of the driver when you reach the Suggested Partitioning screen:

  1. On the Suggested Partitioning screen, select Expert Partitioner.

  2. Navigate to Linux > Hard disks and verify that there is a device listed for the LSI - LSI MegaSR driver. The device might be listed as a type other than sda. For example:

    dev/sdd: LSI - LSI MegaSR

    If no device is listed, the driver did not install properly. In that case, repeat the steps above.
Step 14

When installation is complete, reboot the target server.

For More RAID Utility Information

The Broadcom utilities have help documentation for more information about using the utilities.