Installing Linux Drivers

Prerequisites for Installing Linux Drivers

Ensure that you adhere to the following best practice for installing the drivers. Upgrade the infrastructure in the following order before installing or upgrading the drivers.

  • Upgrade the infrastructure firmware which includes the UCS Manager, the Fabric Interconnects, and the chassis I/O Modules.

  • Upgrade the server and adapter firmware.


Caution

Failure to adhere to the proper upgrade sequence can cause the server to crash.

Installing Linux Drivers During OS Install


Note

The examples shown in this procedure are on Linux operating system distribution RHEL.

Procedure

Installing Linux Drivers to the Local Drive Using a Driver Disk
Step 1

Enable CDN from the BIOS settings.

Step 2

Boot the server using RHEL installation .ISO image through vMedia.
Step 3

In Red Hat Installation menu, select Install Red Hat Enterprise Linux, but do not press Enter yet.

Step 4

Press Tab to edit the configuration options.

Step 5

At the end of vmlinuz line, add dd and press Enter.

To configure Consistent Device Name (CDN), add biosdevname=1 option next to dd.

Step 6

When Driver disk device selection menu is displayed, un-map installation .ISO image and map the driver .ISO image.

Step 7

Press r to refresh the menu.

A driver list is displayed.
Step 8

Enter the corresponding driver number to install the driver.
Step 9

Un-map the driver .ISO image and map the installation .ISO image back.
Step 10

Press r to ensure that the correct .ISO image is loaded.

Step 11

Press c to continue the installation. For further details, refer to the RHEL 7.X Deployment Guide.

Step 12

Complete the RHEL installation.

Step 13

Verify that the default RHEL driver was replaced by the driver in the driver .ISO image.

For the eNIC driver, cat /sys/module/enic/version.

For the fNIC driver, cat /sys/module/fnic/version.

To read the CDN label assigned to a given VNIC, use the command biosdevname -d in the host terminal.

Installing Linux to SAN Storage and fNIC Driver

The examples shown in this procedure are on Linux operating system distribution RHEL.

Procedure

Step 1

Create a vHBA on the Cisco VIC card.

Step 2

In Cisco UCS Manager, mark the vHBA bootable and add the WWPN of the SAN storage.

For more information on how to do this step, see the Cisco UCS Manager Configuration Guide.

Step 3

Boot the server using the RHEL installation DVD through vMedia.

Step 4

For the installation target, choose the SAN storage device.

Step 5

Finish the OS installation.
Step 6

After the OS installation is complete, install the fNIC driver from Cisco driver .ISO image.
Step 7

After the driver is successfully installed, reboot the server.

Installing Linux Drivers using RPM

If the management connection is over the eNIC, Cisco recommends using the serial or KVM console to complete the driver installation. Completing an rmmod of the current driver results in a loss of eNIC network connectivity.


Note

On SLES 12 SP1 and later, the Cisco eNIC and usNIC drivers are bundled together in to a single RPM (vs. being packaged in separate RPMs, as they are for other Linux distributions). Bundling both drivers into a single RPM is necessary because of how kernel module dependencies are managed in SLES 12 SP1 and later. If you are not using Cisco usNIC functionality, the usNIC driver will effectively be ignored.

The eNIC and usNIC drivers have their own distinct version numbers. If you install the cisco-enic-usnic RPM on SLES 12 SP1 or later, once those drivers are loaded into the running kernel (e.g., via rebooting), use cat /sys/module/enic/version and cat /sys/module/usnic_verbs/version to view their respective version numbers. The cisco-enic-usnic RPM has its own distinct version number as well. Because it represents the packaging of the eNIC and usNIC drivers, the RPM version number may look similar, but does not reflect the specific version of either driver. You can also query the Description of the RPM to see exactly what versions of the drivers will be installed.

The following shows a sample query of the Description of the RPM to view the driver versions to be installed:
$ rpm -qip cisco-enic-usnic-kmp-default-<RPM_VERSION>.x86_64.rpm
Name        : cisco-enic-usnic-kmp-default  Relocations: (not relocatable)
...
Summary     : Cisco VIC Ethernet NIC drivers
Description :
This RPM contains both the Cisco VIC Linux Ethernet driver (enic.ko,
version <ENIC_VERSION>) and the Cisco Userspace NIC (usNIC) Linux
Ethernet driver (usnic_verbs.ko, version <USNIC_VERSION>).  Some
Linux distros require both kernel modules to be in the same RPM in
order to properly test for symbol compatibility (because
usnic_verbs.ko depends on enic.ko) when installing into post-GA
upgrade kernels.

SUMMARY STEPS

  1. Install the binary RPM by entering the corresponding command for your driver:
  2. Load the driver in one of the following ways:
  3. Verify the driver installation by entering the sbin/lsmod | grep {enic | fnic} command.

DETAILED STEPS

Step 1

Install the binary RPM by entering the corresponding command for your driver:

Note 

If installing usNIC functionality, refer to the usNIC Deployment Guide for version packaging considerations.

Driver Command

eNIC driver on RHEL

rpm -ivh kmod-enic-version.x86_64.rpm

eNIC and usNIC drivers on SLES 12 SP4 and later

rpm -ivh cisco-enic-usnic-kmp-default-version.x86_64.rpm

fNIC driver on RHEL

rpm -ivh kmod-fnic-version.x86_64.rpm

fNIC driver on SLES

rpm -ivh cisco-fnic-kmp-default-version.x86_64.rpm

The driver is installed but not loaded.

Step 2

Load the driver in one of the following ways:

  • Reboot the host.

  • Load the driver manually while the host is running by entering the modprobe {enic | fnic} command.

Note 

If an error message displays that the module is in use, remove any modules that are using the driver, then remove the driver. Issuing the lsmod | grep {enic | fnic} command can help determine which modules are in use. Use rmmod {enic | fnic} to remove the driver.

If there are many references to the eNIC or fNIC module and it is not possible to remove all of the dependencies, reboot the system.

Step 3

Verify the driver installation by entering the sbin/lsmod | grep {enic | fnic} command.

Installing Linux Drivers Using the Source Tarball

Procedure

Step 1

Extract the tarball and build/install the driver:

Example:

$ tar xf enic-<version>.tar.bz2
   $ cd enic-<version>
   $ ./autogen.sh
   $ ./configure
   $ make
   # make install (requires administrative privileges)
Step 2

After the driver is installed, re-create the boot loader with the new enic driver:

Example:

SLES12: 
# mkinitrd --force -m enic (make initrd ramfs image with enic module)

Note that SLES does not support third-party kernel modules by default. In order to enable the unsupported module, you need to add "allow_unsupported_modules 1" to /etc/modprobe.d/10-unsupported-modules.conf

Note that you will also need to compile and install an updated Cisco usNIC driver as well. The Cisco usNIC driver source code is part of the same UCS Drivers download as the enic sources.

SLES11:

 # mkinitrd -m enic
RHEL/CentOS 6.x, 7.x, XS:
 # mkinitrd <img file> <kernel version> --force --preload enic

Example:

 # mkinitrd /boot/initramfs-3.10.0-123.el7.x86_64.img \
 3.10.0-123.el7.x86_64 --force --preload enic

Ubuntu:

# mkinitramfs -o /boot/initrd.img-`uname -r` `uname -r`

At this point the driver is installed but not loaded in the running kernel.
Step 3

There are two ways to load the updated enic driver:

  • Automatically - If the enic driver was successfully installed, rebooting the server should load it into the running kernel. Even if your Linux distribution has an older inbox version of the enic driver, the newly-installed enic driver should take precedence.

  • Manually - Manually unloading a previous enic driver and loading a newer enic driver is possible in some scenarios, but is not recommended. For example, SLES does not support rmmod'ing kernel modules from a running kernel; SuSE recommends configuring modules in /etc/modprobe.d/ and rebooting. This procedure requires administrative privileges.

    1. Unload the currently-running driver:

      # rmmod enic
      Note 

      Ensure that you are not connected to the server where you are unloading the enic driver via an enic interface when you execute this command, or your connection will be severed. You may wish to execute this command either on the console or in a KVM session, for example.

    2. Load the updated driver from the local build directory: 
      # /sbin/insmod ./enic.ko
      Or you can load from lib/modules if the driver was installed: 
      # /sbin/modprobe enic
Step 4

After updating the enic driver and loading it, you should verify that the kernel is running the updated driver.

  1. Check that a driver named "enic" is present in the running kernel: # /sbin/lsmod | grep enic

  2. Check the kernel log to see the version the enic driver the last time it was loaded: # /bin/dmesg | grep enic

  3. Check to ensure that the version is what you expect it to be.

  4. Be sure to check the enic version from the last time it was loaded in the kernel. For example, dmesg may still show log messages from when the prior enic driver was loaded.