Table Of Contents
Release Notes for Cisco SN iSCSI Driver for Linux Version 2.1.1
February 27, 2002
Note You can find the most current documentation on Cisco.com. This set of electronic documents may contain updates and modifications made after the hard-copy documents were printed.
These release notes support Cisco Storage Networking iSCSI Driver for Linux version 2.1.1.
For a list of software caveats that apply to version 2.1.1, see the "Caveats" section. The caveats are updated for every maintenance version and are located on Cisco.com and the Documentation CD-ROM.
These Release Notes describe the following topics:
The iSCSI Driver for Linux provides an IP host with the ability to access storage through an IP network. The iSCSI driver uses iSCSI protocol to transport SCSI requests and responses over an IP network between the host and a Cisco SN 5400 Series system.
Architecturally, the iSCSI driver combines with the host TCP/IP stack, network drivers, and NICs to provide the same functions as a SCSI adapter driver with a host bus adapter (HBA).
The iSCSI driver provides a transport for SCSI requests and responses for storage devices; however, instead of providing a transport for directly attached devices, the driver transports the SCSI requests and responses between the IP host and a Cisco SN 5400 Series system via an IP network. The SN 5400 Series system, in turn, transports SCSI requests and responses between it and the storage devices attached to it.
Once the iSCSI driver is installed, the IP host will proceed with a discovery process for iSCSI storage devices as follows:
Step 1 The iSCSI driver requests available iSCSI targets from the SN 5400 Series system.
Step 2 The SN 5400 Series system sends available iSCSI target names to the IP host.
Step 3 The IP host logs into the iSCSI targets.
Step 4 The SN 5400 Series system accepts the IP host login.
Step 5 The IP host queries targets for device information.
Step 6 Targets respond with device information.
Step 7 The IP host creates a table of internal devices.
The iSCSI Driver for Linux provides IP access to a maximum of 16 remote SCSI targets. Each target will be probed for up to 32 LUNs, until the Linux kernel's limit of SCSI devices has been reached. Remote SCSI targets can be accessed through one or more SN 5400 Series systems up to a maximum of 16 SN 5400 Series systems.
Note The iSCSI protocol is an IETF-defined protocol for IP storage (ips). For more information about the iSCSI protocol, refer to the IETF standards for IP storage at http://www.ietf.org.
This section describes the system requirements for version 2.1.1 and includes the following information:
Operating System Requirements
•This driver requires either a Linux kernel version 2.2.16 or later, or a Linux kernel version 2.4.16 or later, running on an Intel IA32 (80386, 80486, Pentium) or equivalent, processor. Compilation requires that the kernel header files match the kernel version you want to use the driver with. Once compiled, the objects and executables can be moved to another host running the same level of the operating system.
•The iSCSI Driver for Linux supports multiple processors, and may be run on up to four multiprocessors.
•To ensure the best performance for iSCSI drivers, the extended windowing feature of TCP should be enabled on all IP hosts connecting to the SN 5400 Series system. In general, a larger window size enhances SN 5400 Series system performance.
•The receive and transmit flow control feature of the Gigabit Ethernet driver should be enabled on all IP hosts connecting to the SN 5400 Series system.
•If you are using a 3Com Gigabit Ethernet Server network interface card, the minimum supported revision level is "B" (3C985B-SX). Using a card with a lower revision level will decrease performance.
Note Additional information about Linux kernel issues can be found in the Linux Kernel HOWTO document at http://www.linux.org/docs/.
Do not add mount entries for iSCSI devices to /etc/fstab because the Linux boot process normally mounts filesystems listed in /etc/fstab before the network is configured. The script "iscsi-mountall" will manage the checking and mounting of devices listed in the file /etc/fstab.iscsi, which has the same format as /etc/fstab. This script is automatically invoked by the iSCSI startup script.
To avoid the configuration problems associated with device name changes resulting from configuration changes, use filesystem UUIDs or labels (see man pages for mke2fs, mount, and fstab) or logical volume management (see Linux LVM).
It is very important to unmount all filesystems on iSCSI devices before the iSCSI driver stops. If the iSCSI driver stops while iSCSI devices are mounted, buffered writes may not be committed to disk and filesystem corruption can occur.
Since Linux will not unmount filesystems that are being used by a running process, before iSCSI devices can be unmounted, any processes using those devices must be stopped.
To avoid filesystem corruption, the iSCSI shutdown script will automatically kill all processes using devices in /etc/fstab.iscsi, first by sending them SIGTERM, and then by sending any remaining processes SIGKILL. It will then unmount all iSCSI filesystems and kill the iSCSI daemon, terminating all connections to iSCSI devices.
Caution Filesystems not listed in /etc/fstab.iscsi may not be automatically unmounted.
SN 5400 Series System Software Requirements
The iSCSI Driver for Linux version 2.1.1 can connect to a Cisco SN 5400 Series system running software release 2.1.1 or later; this driver cannot connect to a Cisco SN 5420 Storage Router running software release 1.1.x.
Obtaining the iSCSI Driver and Updated SN 5400 Series System Software
From time to time, Cisco releases updated versions of SN 5400 Series system software and iSCSI drivers. Updated versions of SN 5400 Series system software and the Cisco Storage Networking iSCSI drivers, accompanying readme files, release notes and example configuration file are available for download.
If you are a registered Cisco.com user, you can download SN 5400 Series system software and iSCSI drivers. If you are a non-registered Cisco.com user, you can download only iSCSI drivers.
You can access software by following these instructions:
Step 2 At the Software Center web page, under Software Products & Downloads, click Storage Networking Software.
Step 3 At the Storage Networking Software web page, click the appropriate link for your software. For example, click Cisco SN 5420 Storage Router Software for SN 5420 software.
Step 4 At the software download web page, click the file that you want to download. Another software download web page will be displayed with detailed information about the download file and Cisco's Software License Agreement. Follow instructions on that page to download the software.
Step 5 To install and configure SN 5400 Series system software, see the appropriate SN 5400 Series system software configuration guide and release notes. To install and configure an iSCSI driver, see the readme file that accompanies the iSCSI driver (in the downloaded driver archive file) and the appropriate release notes.
Upgrading to a New Version
To upgrade to a new version of iSCSI driver software, follow these instructions:
Step 1 Unmount all iSCSI file systems and stop the old iSCSI driver. For example, to manually stop the iSCSI version 1.8.7 driver that was installed in the default directory, enter:/etc/rc.d/init.d/NuScsiTcp stop
If you are upgrading from another 2.1.x release, issue the following command to stop the iSCSI driver:/etc/rc.d/init.d/iscsi stop
Step 2 Select a directory in which to unpack the linux-iscsi-<version>.tgz file, such as /usr/src. Make the directory if it does not exist. The <version> is the three digit version of the driver. For example:mkdir -p /usr/src/cp /tmp/linux-iscsi-2.1.1.tgz /usr/src
Step 3 Change to the directory created in Step 2, and uncompress the file using the tar command. For example:cd /usr/src/tar xzvf linux-iscsi-2.1.1.tgz
This creates a driver source subdirectory named linux-iscsi-2.1.1.
Step 4 Change to the linux_iscsi_2.1.1 subdirectory, created in Step 3, and compile the iSCSI driver. If your kernel sources are not in the usual place, add "TOPDIR=/path/to/kernel" or edit the definition of TOPDIR in the Makefile. For example:cd /usr/src/linux-iscsi-2.1.1make
Step 5 Install the iSCSI driver. You must have super-user (root) authority to install the driver. For example:make install
Step 6 The install process does not overwrite any existing /etc/iscsi.conf files. Compare the new version of the iscsi.conf file to the existing /etc/iscsi.conf file. Update the existing file with any additional information from the new file and save it as /etc/iscsi.conf. The iscsi.conf man page has a more detailed description of the configuration file format. To read the man page:man iscsi.conf
Step 7 (Optional) If you are using a non-Red Hat Linux distribution you may need to edit your boot scripts to properly run the iSCSI setup script. You may also need to change the order of the boot script to ensure that iSCSI services are started after the network has been initialized.
Step 8 Verify that your iSCSI partitions are listed in /etc/fstab.iscsi. It has the same format as /etc/fstab. The init scripts will automatically mount and unmount these partitions automatically.
Step 9 Reboot the system to start and reload the iSCSI driver.
If you do not wish to reboot your machine, you can start the iSCSI devices immediately with the start command. For example:/etc/rc.d/init.d/iscsi start
See the readme file for additional information about installing iSCSI driver software.
Uninstalling iSCSI Driver Software
To uninstall the iSCSI driver software, follow these instructions:
Step 1 Change to the linux-iscsi-<version> directory. The <version> is the three digit version. For example:cd /usr/src/linux-iscsi-2.1.1
Step 2 Remove the iSCSI driver. You must have super-user (root) authority to remove the driver. For example:make remove
This deletes the appropriate files from /lib/modules and /usr/local/sbin directories. The configuration files in /etc are not deleted, since they will be needed if another driver is installed at a later time.
Step 3 Back up one directory and delete the source code. For example:cd ..rm -fr linux-iscsi-2.1.1
As of 22 February, 2002 there are several issues with the Linux kernel code that can cause problems when using SCSI devices (including iSCSI devices). Linux kernels released after this date may or may not have fixed these problems.
•Linux kernels 2.2.16 through 2.2.20 and 2.4.0 through 2.4.17 are known to have a problem in the SCSI error recovery process. In some cases, a successful LUN reset may be ignored and the SCSI layer will continue on to the later stages of the error recovery process.
The problem occurs when multiple SCSI commands for a particular LUN are queued in the low-level SCSI driver when a LUN reset occurs. Even if the low-level driver correctly reports that all the commands for the LUN have been completed by the LUN reset, Linux will assume only one command has been completed and continue the error recovery process. (If only one command has timed out or failed, Linux will correctly terminate the error recovery process following the LUN reset.)
This action is undesirable because the later states of error recovery may send target resets, which can affect other SCSI initiators using the target. It is also undesirable because there are more serious bugs in the later stages of the Linux SCSI error recovery process.
•Linux kernels 2.2.16 through 2.2.20 and 2.4.0 through 2.4.2 may take SCSI devices offline after Linux issues a target reset as part of the error recovery process. Taking a device offline causes all I/O to the device to fail until the HBA driver is reloaded.
After the error recovery process does a target reset, it sends a SCSI Test Unit Ready command to check if the SCSI target is operational again. If this command returns SCSI sense data, instead of correctly retrying the command, Linux will treat it as a fatal error and immediate take the SCSI device offline.
The Test Unit Ready will almost always be returned with sense data because most targets return a deferred error in the sense data on the first command received after a target reset. This is a way of telling the initiator that a target reset has occurred. Therefore, the affected Linux kernel versions almost always take the SCSI device offline after a target reset occurs.
This bug is fixed in Linux kernels 2.4.3 and later.
•Linux kernels 2.2.16 through 2.2.20 and 2.4.0 through 2.4.17 appear to have problems when SCSI commands are completed with a check condition containing sense data. This can result in applications receiving I/O errors, short reads or short writes.
After receiving a SCSI command with sense data, the Linux SCSI midlayer checks if some sectors of an I/O request have been transferred and informs the application layer if an I/O request has partially completed. The SCSI midlayer then retries the request for any sectors that have not yet been transferred. Because of the partial I/O request completion, applications may receive short reads or writes.
All UNIX applications should handle these conditions, but there may be some applications that do not.
There are also some cases where the application receives an I/O error rather than a short read or write. The exact cause of the I/O errors is still being investigated, but it appears to be a bug in the Linux kernel's SCSI layer or block device layer.
•Linux kernel 2.2.16 through 2.2.20 and 2.4.0 through 2.4.17 may crash on a NULL pointer if a SCSI device is taken offline while one of the Linux kernel's I/O daemons (e.g. kpiod, kflushd, etc.) is trying to do I/O to the SCSI device. The exact cause of this problem is still being investigated.
Note Some of the other bugs in the Linux kernel's error recovery handling may result in a SCSI device being taken offline, thus triggering this bug and resulting in a Linux kernel crash.
This document lists caveats for iSCSI Driver for Linux version 2.1.1. Caveats describe unexpected behavior or defects in iSCSI software versions. Severity 1 caveats are the most serious caveats; severity 2 caveats are less serious.
This document describes open and resolved severity 1 and 2 caveats and selected caveats of other severities:
•The "Open Caveats" section lists open caveats that apply to the current version and may apply to previous versions.
•The "Resolved Caveats" section list caveats resolved in this version, but open in previous versions.
Within the sections, the caveats are sorted alphanumerically by caveat number.
Note If you have an account with Cisco.com, you can use Bug Navigator II to find caveats of any severity for any version. You can reach Bug Navigator II on Cisco.com at Service & Support:
There are no severity 1 or 2 caveats open against the iSCSI driver version 2.1.1. For a more complete list of caveats against this release, access Cisco.com as described in the section "Cisco.com" at the end of this document.
All of the caveats listed in this section are resolved in iSCSI driver version 2.1.1.
Both make and make_install may fail when installing the Linux iSCSI driver on a SuSE Linux distribution.
Workaround: This is a SuSE distribution problem. Follow the SuSE instructions and copy the following files to directory /lib/modules/'uname -r'/build/include/linux:
Both make and make_install may fail when installing the Linux iSCSI driver on a SuSE Linux distribution. The chkconfig command is present in the SuSE distribution, but it is just an empty script. This causes the make to believe the distribution is Red Hat, and the installation fails to complete correctly.
Workaround: The problem has been resolved by modifying the Makefile.
Targets mapped to LUN 10 or greater will not be found by the Linux driver when using "Target,Lun" in the NuScsiTcp.conf file.
Workaround: The problem has been resolved by correcting the processing of the configuration file.
The following sections describe the related documentation available for the iSCSI Driver for Linux version 2.1.1 and the Cisco SN 5400 Series system. These documents consist of the iSCSI driver release notes, readme and example configuration files, and the SN 5400 Series system hardware installation and software configuration guides.
The SN 5400 Series system hardware installation and software configuration documentation sets are available as printed manuals or electronic documents. The iSCSI driver readme file and example configuration file are available in electronic format, as part of the software download package. See the "Obtaining the iSCSI Driver and Updated SN 5400 Series System Software" section for details.
This Release Notes document is the only document specific to iSCSI Driver for Linux version 2.1.1. It is located on Cisco.com and the Documentation CD-ROM.
Each release of SN 5400 Series system software includes an associated Release Notes document, which is also available as an electronic document on Cisco.com and the Documentation CD-ROM.
Refer to the Cisco SN 5420 Storage Router Hardware Installation Guide for SN 5400 Series system hardware installation procedures. This document is available as a printed manual. It is also available as an electronic document on Cisco.com and the Documentation CD-ROM.
Refer to the Cisco SN 5420 Storage Router Software Configuration Guide Release 2.1 for SN 5400 Series system software configuration information and procedures. This document is available as a printed manual. It is also available as an electronic document onCisco.com and the Documentation CD-ROM.
For documentation on the SN 5400 Series system web-based GUI, refer to the SN 5400 Series system web-based GUI online Help system.
Service and Support
For service and support for a product purchased from a reseller, contact the reseller, who offers a wide variety of Cisco service and support programs described in "Service and Support" of Cisco Information Packet shipped with your product.
Note If you purchased your product from a reseller, you can access Cisco.com as a guest. Cisco.com is Cisco Systems' primary real-time support channel. Your reseller offers programs that include direct access to Cisco.com services.
For service and support for a product purchased directly from Cisco, use Cisco.com.
Software Configuration Tips on the Cisco TAC Home Page
A variety of Cisco SN 5400 Series system software and iSCSI driver installation, configuration and usage tips are available on the Cisco Technical Assistance Center (TAC) Web Site.
You can access "tech tips" by following these instructions:
Step 2 At the Cisco Routers web page, under Cisco SN 5420 Storage Router Software, click Product Support.
Step 3 At the Cisco SN 5420 Storage Router Product Support web page, click the appropriate links for additional information about installing and configuring SN 5400 Series system software and iSCSI drivers.
The following sections explain how to obtain documentation from Cisco Systems.
World Wide Web
You can access the most current Cisco documentation on the World Wide Web at the following URL:
Translated documentation is available at the following URL:
Cisco documentation and additional literature are available in a Cisco Documentation CD-ROM package, which is shipped with your product. The Documentation CD-ROM is updated monthly and may be more current than printed documentation. The CD-ROM package is available as a single unit or through an annual subscription.
Cisco documentation is available in the following ways:
•Registered Cisco Direct Customers can order Cisco product documentation from the Networking Products MarketPlace:
•Registered Cisco.com users can order the Documentation CD-ROM through the online Subscription Store:
•Nonregistered Cisco.com users can order documentation through a local account representative by calling Cisco corporate headquarters (California, USA) at 408 526-7208 or, elsewhere in North America, by calling 800 553-NETS (6387).
If you are reading Cisco product documentation on Cisco.com, you can submit technical comments electronically. Click Leave Feedback at the bottom of the Cisco Documentation home page. After you complete the form, print it out and fax it to Cisco at 408 527-0730.
You can e-mail your comments to firstname.lastname@example.org.
To submit your comments by mail, use the response card behind the front cover of your document, or write to the following address:
Attn: Document Resource Connection
170 West Tasman Drive
San Jose, CA 95134-9883
We appreciate your comments.
Obtaining Technical Assistance
Cisco provides Cisco.com as a starting point for all technical assistance. Customers and partners can obtain documentation, troubleshooting tips, and sample configurations from online tools by using the Cisco Technical Assistance Center (TAC) Web Site. Cisco.com registered users have complete access to the technical support resources on the Cisco TAC Web Site.
Cisco.com is the foundation of a suite of interactive, networked services that provides immediate, open access to Cisco information, networking solutions, services, programs, and resources at any time, from anywhere in the world.
Cisco.com is a highly integrated Internet application and a powerful, easy-to-use tool that provides a broad range of features and services to help you to
•Streamline business processes and improve productivity
•Resolve technical issues with online support
•Download and test software packages
•Order Cisco learning materials and merchandise
•Register for online skill assessment, training, and certification programs
You can self-register on Cisco.com to obtain customized information and service. To access Cisco.com, go to the following URL:
Technical Assistance Center
The Cisco TAC is available to all customers who need technical assistance with a Cisco product, technology, or solution. Two types of support are available through the Cisco TAC: the Cisco TAC Web Site and the Cisco TAC Escalation Center.
Inquiries to Cisco TAC are categorized according to the urgency of the issue:
•Priority level 4 (P4)—You need information or assistance concerning Cisco product capabilities, product installation, or basic product configuration.
•Priority level 3 (P3)—Your network performance is degraded. Network functionality is noticeably impaired, but most business operations continue.
•Priority level 2 (P2)—Your production network is severely degraded, affecting significant aspects of business operations. No workaround is available.
•Priority level 1 (P1)—Your production network is down, and a critical impact to business operations will occur if service is not restored quickly. No workaround is available.
Which Cisco TAC resource you choose is based on the priority of the problem and the conditions of service contracts, when applicable.
Cisco TAC Web Site
The Cisco TAC Web Site allows you to resolve P3 and P4 issues yourself, saving both cost and time. The site provides around-the-clock access to online tools, knowledge bases, and software. To access the Cisco TAC Web Site, go to the following URL:
All customers, partners, and resellers who have a valid Cisco services contract have complete access to the technical support resources on the Cisco TAC Web Site. The Cisco TAC Web Site requires a Cisco.com login ID and password. If you have a valid service contract but do not have a login ID or password, go to the following URL to register:
If you cannot resolve your technical issues by using the Cisco TAC Web Site, and you are a Cisco.com registered user, you can open a case online by using the TAC Case Open tool at the following URL:
If you have Internet access, it is recommended that you open P3 and P4 cases through the Cisco TAC Web Site.
Cisco TAC Escalation Center
The Cisco TAC Escalation Center addresses issues that are classified as priority level 1 or priority level 2; these classifications are assigned when severe network degradation significantly impacts business operations. When you contact the TAC Escalation Center with a P1 or P2 problem, a Cisco TAC engineer will automatically open a case.
To obtain a directory of toll-free Cisco TAC telephone numbers for your country, go to the following URL:
Before calling, please check with your network operations center to determine the level of Cisco support services to which your company is entitled; for example, SMARTnet, SMARTnet Onsite, or Network Supported Accounts (NSA). In addition, please have available your service agreement number and your product serial number.
This document is to be used in conjunction with the documents listed in the "Related Documentation" section.
Copyright © 2002, Cisco Systems, Inc.
All rights reserved.