Table Of Contents
Release Notes for Cisco iSCSI Driver Version 3.1.1 for Sun Solaris 2.6 or Sun Solaris 7 or 8
May 30, 2003
Note You can find the most current documentation on Cisco.com. This set of electronic documents may contain updates and modifications made after the hard-copy documents were printed.
These release notes support Cisco iSCSI Driver version 3.1.1 for Sun Solaris 2.6, 7 and 8.
For a list of software caveats that apply to version 3.1.1, see the "Caveats" section. The caveats are updated for every maintenance version and are located on Cisco.com and the Documentation CD-ROM.
These Release Notes describe the following topics:
The iSCSI Driver for Sun Solaris provides an IP host with the ability to access storage through an IP network. The iSCSI driver uses the iSCSI protocol to transport SCSI requests and responses over an IP network between the IP host and a Cisco SN 5400 or MDS 9000 Series system.
Architecturally, the iSCSI driver combines with the IP host's TCP/IP stack, network drivers, and network interface cards (NICs) to provide the same functions as a SCSI or Fibre Channel adapter driver with a host bus adapter (HBA).
The iSCSI driver provides a transport for SCSI requests and responses for storage devices; however, instead of providing a transport for directly attached devices, the driver transports the SCSI requests and responses between the IP host and a Cisco SN 5400 or MDS 9000 Series system via an IP network. The SN 5400 or MDS 9000 Series system, in turn, transports SCSI requests and responses between it and the storage devices attached to it.
Once the iSCSI driver is installed and started, the host proceeds with a discovery process for storage devices.
A more technical description of the driver's design and its features can be found in the readme file that accompanies the iSCSI driver in the downloaded driver archive file.
Note The iSCSI protocol is an IETF-defined protocol for IP storage (ips). For more information about the iSCSI protocol, refer to the IETF standards for IP storage at http://www.ietf.org.
This section describes the system requirements for version 3.1.1 and includes the following information:
Operating System Requirements
•The iSCSI Driver for Sun Solaris runs only on SPARC processor-based machines.
•The iSCSI Driver for Sun Solaris requires either Sun Solaris version 2.6, 7 or 8, with all Sun-recommended patches installed for the version being used.
•If you are using the Sun Gigabit Ethernet adapter, the SUNWged driver must have the most recent version of patch 106765 (for SUNWged version 2.0) or patch 108813 (for SUNWged version 3.0) installed.
•The iSCSI Driver for Sun Solaris supports both single and multiple processor machines.
•To ensure the best performance for the iSCSI driver, the extended windowing feature of TCP should be enabled on all IP hosts connecting to the SN 5400 Series system. In general, a larger window size enhances SN 5400 Series system throughput performance.
•To maximize performance, the receive and transmit flow control feature of the Gigabit Ethernet driver should be enabled on all IP hosts connecting to the SN 5400 Series system. If the IP hosts or the SN 5400 Series systems are connected to any Ethernet switches, the receive and transmit flow control attributes should be enabled on each of the connected switch ports.
•If you are using a 3Com Gigabit Ethernet Server network interface card in the IP host, the minimum supported revision level is "B" (3C985B-SX). Using a card with a lower revision level will significantly decrease performance.
Target iSCSI devices can be mounted automatically using the /etc/vfstab file.
The iSCSI init.d script looks for entries with the "mount at boot" field set to "iscsi" in the /etc/vfstab file, and attempts to mount those entries after the iSCSI daemon starts. This allows iSCSI devices to be automatically mounted as early as possible in the boot process.
For example, the following /etc/vfstab entries will fsck and mount the two iSCSI devices specified:#device device mount FS fsck mount mount#to mount to fsck point type pass at boot options/dev/dsk/c1t5d0s6 /dev/rdsk/c1t5d0s6 /mnt/t5 ufs 1 iscsi -/dev/dsk/c1t6d0s6 /dev/rdsk/c1t6d0s6 /mnt/t6 ufs 1 iscsi -
Note Due to network delays, targets may not always become available in the same order. This means that the order in which iSCSI devices are mounted may vary, and may not match the order of the devices listed in /etc/vfstab. You should not assume that mounts of iSCSI devices will occur in any particular order.
The "mount at boot" field in /etc/vfstab should never be set to "yes" for an iSCSI device, because the standard Solaris boot sequence mounts devices before the network is available. Mounts of iSCSI devices with the "mount at boot" field set to "yes" fail because the iSCSI devices are not available that early in the boot process. If a mount fails, a maintenance shell will start and the Solaris boot process will not complete until the shell is exited. A user at the console must exit from the shell to cause the boot process to complete without the listed devices mounted.
All iSCSI devices must be unmounted before the iSCSI driver stops. If the iSCSI driver stops while iSCSI devices are mounted, buffered writes may not be committed to disk and filesystem corruption may occur. Because Solaris will not unmount devices that are being used by a running process, all processes using the iSCSI devices must be killed before the devices can be unmounted.
The init.d script will attempt to kill all processes using iSCSI devices by sending them SIGTERM and then sending any remaining processes SIGKILL. The init.d script then unmounts all iSCSI devices, and kills the iSCSI daemon, terminating all connections to iSCSI devices.
Note The init.d script may not be able to successfully unmount filesystems because they are in use by processes that cannot be killed. Best practice is to manually stop all applications using the filesystem before stopping the driver.
Persistent Target Bindings
Persistent target binding ensures that a Solaris SCSI target always maps to the same physical storage device across system restarts. The iSCSI daemon stores bindings of iSCSI target iSCSI Names (or WWUIs) to Solaris target IDs in the file /etc/iscsi_bindings.
If any entry exists for a discovered target, the Solaris target ID from the entry is assigned to the target. If no entry exists for a discovered target, an entry is written to the file.
The iscsi_bindings file contains the Solaris target ID, the discovery address, and the iSCSI Name (or WWUI) of the iSCSI target, as shown in the following example:TargetId DiscoveryAddress TargetName0 10.100.100.156 iqn.1987-05.com.cisco.00.7e9d6f942e45736be69cb65c4c22e54c.disk_one1 10.100.100.156 iqn.1987-05.com.cisco.00.4d678bd82965df7765c788f3199ac15f.disk_two
A discovery connection to each DiscoveryAddress remains open; new targets that become available to the Solaris host after the driver is started are automatically logged into and added to the iscsi_bindings file (if no entry currently exists).
To use these new targets, Solaris administrative command, such as drvconfig, disks, or tapes, must be issued to add the required /dev entries. If the required target and LUN entries for the new target do not exist in /kernel/drv/sd.conf, you may have to add them and reboot the system before you can access the target.
The /etc/iscsi_bindings file contains persistent entries for all iSCSI targets ever logged into from this host. If a target is no longer available to a host, you can stop the iSCSI driver and manually edit the iscsi_bindings file and remove the entry so the obsolete target no longer consumes a Solaris target ID.
For more information about the iscsi-bindings file and persistent target binding, see the iSCSI driver readme file.
Target Port Failover
If the Cisco SN 5400 Series system has multiple Gigabit Ethernet ports configured to allow access to a set of iSCSI targets via multiple paths, the iSCSI driver discovers all IP addresses that can be used to reach each of the targets. When a specific target connection fails, the iSCSI driver attempts to connect to that target using the next IP address. Subsequent failures will cause the iSCSI driver to rotate through the list of addresses. This feature is enabled or disabled via an entry in the iscsi.conf file. The feature is enabled by default.
To effectively utilize this features, the Solaris system must also have multiple Gigabit Ethernet interfaces.
Starting and Stopping the iSCSI Driver
It is possible to manually stop and start the iSCSI driver. This may be necessary if you want to make configuration changes without rebooting the Solaris system. When stopping the iSCSI driver, the init.d script will attempt to unmount iSCSI devices. Because the init.d script may not be able to unmount busy filesystems, best practice is to manually stop all applications that are using iSCSI filesystems before stopping the driver.
To manually stop the iSCSI driver, enter:/etc/init.d/iscsi stop
To manually start the iSCSI driver, enter:/etc/init.d/iscsi start
Caution When installing the iSCSI driver, do not manually stop and restart the iSCSI driver. Always reboot the Solaris system after doing a "pkgadd" of the driver.
You can make configuration changes without having to stop the iSCSI driver or reboot the system, providing the required entries already exist in the /kernel/drv/sd.conf file. To make dynamic configuration changes, follow these instructions:
Step 1 Make the desired changes to the /etc/iscsi.conf file.
Step 2 When you are finished making changes, issue the reconfigure command. For example:/etc/init.d/iscsi reconfigure
This causes the iSCSI driver to re-read /etc/iscsi.conf and to create any new DiscoveryAddress connections it finds. Those discovery sessions discover targets and create new target connections.The script then runs the appropriate Solaris administrative commands (e.g. drvconfig or devfsadm) to add new iSCSI devices to the filesystem namespace.
Note Reconfiguration will not affect existing discovery or target sessions. For example, removing a DiscoveryAddress entry from the configuration file will not cause that session to be terminated.
All iSCSI devices should be unmounted prior to a system shutdown.
The Solaris /usr/sbin/reboot command should not be used to reboot the system while iSCSI devices are mounted. This reboot command will not execute the iSCSI shutdown script in /etc/rc0.d, and file system corruption can occur.
To safely reboot the Solaris system, use the following shutdown command:/usr/sbin/shutdown -i 6
SN 5400 Series System Software Requirements
The iSCSI Driver version 3.1.1 for Sun Solaris is interoperable to a Cisco SN 5400 Series system running software release 3.2.1 or later. This version of the driver is not interoperable with a Cisco SN 5400 Series system running software release 2.5.x or earlier.
MDS 9000 Series System Software Requirements
The iSCSI Driver version 3.1.1 for Sun Solaris is interoperable with a Cisco MDS 9000 Series system running SAN-OS Release 1.1(1) or later.
New and Changed Information
•The iSCSI driver version 3.1.1 iSCSI specification compliance has been upgraded to RFC.
•The iSCSI driver version 3.1.1 includes data and header digest support, configurable on either a per-target or global basis.
See the readme file for additional information about all new features.
This section describes how to obtain iSCSI driver software and upgrade an existing iSCSI driver installation, and includes the following information:
Obtaining the iSCSI Driver and Updated SN 5400 Series System Software
Registered Cisco.com users can download the most current SN 5400 Series system software, Cisco iSCSI drivers, readme files, release notes and example configuration files from Cisco.com. In addition, information about driver compatibility and other relevant driver information is available on Cisco.com. You can access software and related information by following these instructions:
Step 1 At http://www.cisco.com, log in to Cisco.com. Click Technical Support and Software Center.
Step 2 At the Software Center web page, under Software Products & Downloads, click Storage Networking Software.
Step 3 At the Storage Networking Software web page, click the appropriate link for your software.
Step 4 At the Software Download web page, click the file that you want to download.Another software download web page will be displayed with detailed information about the download file and Cisco's Software License Agreement. Follow the instructions on that and any subsequent web pages to download the software.
Step 5 To install and configure storage router software, see the appropriate storage router software configuration guide and release notes. To install and configure an iSCSI driver, see the readme file that accompanies the iSCSI driver (in the downloaded driver archive file) and the appropriate release notes.
Configuration guides and release notes are available online. You can access online documentation by following these instructions:
Step 1 At http://www.cisco.com, click Products & Services and Storage Networking Products.
Step 2 At the Cisco Storage Networking Products web page, click Cisco SN 5400 Series Storage Routers.
Step 3 At the Cisco SN 5400 Series Storage Routers web page, click Technical Documentation. On the Technical Documentation web page, choose the appropriate link for documentation, release notes, or other related information.
Installing iSCSI Driver Software
Refer to the readme file that accompanies the iSCSI driver (in the downloaded driver archive file) for complete installation and configuration procedures.
Upgrading to a New Version
To upgrade to a new version of iSCSI driver software, follow these instructions.
Note You must be super-user (root) to install and configure the iSCSI driver package.
Step 1 Unmount all iSCSI file systems and stop the old iSCSI driver. To manually stop the iSCSI driver, enter:/etc/init.d/iscsi stop
Step 2 Save the current /etc/iscsi.conf and /kernel/drv/sd.conf configuration files to another location.
Step 3 Remove the old iSCSI package.pkgrm CSCOiscsi
Step 4 Follow the instructions in the readme file to install the new iSCSI driver software package.
See the readme file for additional information about installing or upgrading iSCSI driver software.
Uninstalling iSCSI Driver Software
To uninstall the iSCSI driver software, follow these instructions:
Step 1 Unmount all iSCSI devices and stop the driver. For example:/etc/init.d/iscsi stop
You may want to save the /etc/iscsi.conf configuration file to another location before proceeding with the removal process.
Step 2 Remove the iSCSI driver. You must have super-user (root) authority to remove the driver.pkgrm CSCOiscsi
All configuration files installed by the package will be deleted.
Caveats describe unexpected behavior or defects in iSCSI software versions. Severity 1 caveats are the most serious caveats; severity 2 caveats are less serious.
This document describes open and resolved severity 1 and 2 caveats and selected caveats of other severities:
•The "Open Caveats" section lists open caveats that apply to the current version and may apply to previous versions.
•The "Resolved Caveats" section list caveats resolved in this version, but open in previous versions.
Within the sections, the caveats are sorted alphanumerically by caveat number.
Note If you have an account with Cisco.com, you can use Bug Navigator II to find caveats of any severity for any version. You can reach Bug Navigator II on Cisco.com at Service & Support:
There are no severity 1 or 2 caveats open against the iSCSI driver version 3.1.1. For a more complete list of caveats against this release, access Cisco.com as described in the section "Cisco.com" at the end of this document.
While running traffic to a tape drive, the Gigabit Ethernet connection from a Sun Solaris IP host to the SN 5420 Storage Router may be intermittently dropped and re-added for no apparent reason.
Workaround: None. However, the user should check the connections between the hosts and the SN 5420, and verify that the flow control is consistent between the hosts, switches, and the SN 5420s.
Error messages indicating that the iSCSI driver cannot claim bus 0 target <n> are seen on the Solaris host console or in the /var/adm/messages file. The iSCSI driver never gets connected to the targets being discovered. This can be caused by restarting the Solaris driver after a version upgrade or downgrade without rebooting the Solaris host.
Workaround: Reboot the Solaris host. The Solaris host must be rebooted after a version upgrade or downgrade to fully install the iSCSI driver. The iSCSI driver readme file for version 3.1.1 has been updated to include this information.
I/O stops to one or more iSCSI targets, and the iSCSI driver can not be stopped. Under very heavy load, with conditions that cause iSCSI connections to drop and restart frequently, iSCSI tasks can deadlock.
Workaround: None. Reboot the iSCSI host to clear the problem.
The Solaris host or Gigabit Ethernet may hang after several command timeout errors are seen. This can occur when a storage device is physically removed without stopping data traffic.
Workaround: None. The Solaris host must be rebooted. If a storage device is to be removed or replaced, all traffic to the device should be stopped.
When running simultaneously to a large number of targets and performing manual failovers of SCSI routing instances, the iSCSI driver on the Solaris host stopped attempting to login to the targets after doing the initial discovery connection. No additional commands were generated from the iSCSI driver. This issue is very rare and has not been reproduced.
Workaround: None. The Solaris host must be rebooted.
When running large amounts of data traffic to multiple devices from a Solaris multiprocessor host, the Solaris host experienced a panic due to a bad trap. Preceding the panic, there were several command aborts and connection drops and relogins. This issue is very rare and has not been reproduced.
Workaround: None. The Solaris host must be rebooted.
The following sections describe the related documentation available for the iSCSI Driver version 3.1.1 for Sun Solaris, and the Cisco SN 5400 and MDS 9000 Series systems. These documents consist of the iSCSI driver release notes and readme file, and the SN 5400 and MDS 9000 Series system hardware installation and software configuration guides.
The SN 5400 and MDS 9000 Series system hardware installation and software configuration documentation sets are available as printed manuals or electronic documents. The iSCSI driver readme file IS available in electronic format, as part of the software download package. See the "Obtaining the iSCSI Driver and Updated SN 5400 Series System Software" section for details.
This Release Notes document is the only document specific to iSCSI Driver version 3.1.1 for Sun Solaris. It is located on Cisco.com and the Documentation CD-ROM.
Each release of SN 5400 and MDS 9000 Series system software includes an associated Release Notes document, which is also available as an electronic document on Cisco.com and the Documentation CD-ROM.
Refer to the appropriate SN 5400 or MDS 9000 Series system hardware installation guide for hardware installation procedures. The Cisco SN 5428 Storage Router Hardware Installation Guide provides hardware installation procedures for SN 5428 Storage Routers. The Cisco SN 5428-2 Storage Router Hardware Installation Guide provides hardware installation procedures for SN 5428-2 Storage Routers. These documents are available as printed manuals. They are also available as electronic documents on Cisco.com and the Documentation CD-ROM
Refer to the appropriate SN 5400 or MDS 9000 Series system software configuration guide for software configuration information. The Cisco SN 5428 Storage Router Software Configuration Guide Release 3.2 provides configuration information for SN 5428 Storage Routers. The Cisco SN 5428-2 Storage Router Software Configuration Guide Release 3.2 provides configuration information for SN 5428-2 Storage Routers. These documents are available as printed manuals. They are also available as electronic documents on Cisco.com and the Documentation CD-ROM.
For documentation on the SN 5400 Series system web-based GUI, refer to the SN 5400 Series system web-based GUI online Help system.
Service and Support
For service and support for a product purchased from a reseller, contact the reseller, who offers a wide variety of Cisco service and support programs described in "Service and Support" of Cisco Information Packet shipped with your product.
Note If you purchased your product from a reseller, you can access Cisco.com as a guest. Cisco.com is Cisco Systems' primary real-time support channel. Your reseller offers programs that include direct access to Cisco.com services.
For service and support for a product purchased directly from Cisco, use Cisco.com.
Software Configuration Tips on the Cisco TAC Home Page
A variety of Cisco SN 5400 Series system software and iSCSI driver installation, configuration and usage tips are available on the Cisco Technical Assistance Center (TAC) Web Site.
You can access "tech tips" by following these instructions:
Step 1 At http://www.cisco.com, log in to Cisco.com. Click Technical Support, and select Hardware Support from the menu.
Step 2 At the Hardware Support web page, click Storage Networking Devices from the Hardware Support menu on the left side of the page.
Step 3 At the Storage Networking Devices web page, click the appropriate link for your system. For example, click the SN 5428 Storage Routers link.
Step 4 Click the Troubleshooting link, and then click the appropriate links for information about installing, configuring, and troubleshooting SN 5400 Series system software and iSCSI drivers.
Cisco provides several ways to obtain documentation, technical assistance, and other technical resources. These sections explain how to obtain technical information from Cisco Systems.
You can access the most current Cisco documentation on the World Wide Web at this URL:
You can access the Cisco website at this URL:
International Cisco web sites can be accessed from this URL:
Cisco documentation and additional literature are available in a Cisco Documentation CD-ROM package, which may have shipped with your product. The Documentation CD-ROM is updated monthly and may be more current than printed documentation. The CD-ROM package is available as a single unit or through an annual subscription.
Registered Cisco.com users can order the Documentation CD-ROM (product number DOC-CONDOCCD=) through the online Subscription Store:
You can find instructions for ordering documentation at this URL:
You can order Cisco documentation in these ways:
•Registered Cisco.com users (Cisco direct customers) can order Cisco product documentation from the Networking Products MarketPlace:
•Registered Cisco.com users can order the Documentation CD-ROM (Customer Order Number DOC-CONDOCCD=) through the online Subscription Store:
•Nonregistered Cisco.com users can order documentation through a local account representative by calling Cisco Systems Corporate Headquarters (California, U.S.A.) at 408 526-7208 or, elsewhere in North America, by calling 800 553-NETS (6387).
You can submit comments electronically on Cisco.com. On the Cisco Documentation home page, click Feedback at the top of the page.
You can email your comments to firstname.lastname@example.org.
You can submit your comments by mail by using the response card behind the front cover of your document or by writing to the following address:
Attn: Customer Document Ordering
170 West Tasman Drive
San Jose, CA 95134-9883
We appreciate your comments.
Obtaining Technical Assistance
Cisco provides Cisco.com, which includes the Cisco Technical Assistance Center (TAC) Website, as a starting point for all technical assistance. Customers and partners can obtain online documentation, troubleshooting tips, and sample configurations from the Cisco TAC website. Cisco.com registered users have complete access to the technical support resources on the Cisco TAC website, including TAC tools and utilities.
Cisco.com offers a suite of interactive, networked services that let you access Cisco information, networking solutions, services, programs, and resources at any time, from anywhere in the world.
Cisco.com provides a broad range of features and services to help you with these tasks:
•Streamline business processes and improve productivity
•Resolve technical issues with online support
•Download and test software packages
•Order Cisco learning materials and merchandise
•Register for online skill assessment, training, and certification programs
To obtain customized information and service, you can self-register on Cisco.com at this URL:
Technical Assistance Center
The Cisco TAC is available to all customers who need technical assistance with a Cisco product, technology, or solution. Two levels of support are available: the Cisco TAC website and the Cisco TAC Escalation Center. The avenue of support that you choose depends on the priority of the problem and the conditions stated in service contracts, when applicable.
We categorize Cisco TAC inquiries according to urgency:
•Priority level 4 (P4)—You need information or assistance concerning Cisco product capabilities, product installation, or basic product configuration.
•Priority level 3 (P3)—Your network performance is degraded. Network functionality is noticeably impaired, but most business operations continue.
•Priority level 2 (P2)—Your production network is severely degraded, affecting significant aspects of business operations. No workaround is available.
•Priority level 1 (P1)—Your production network is down, and a critical impact to business operations will occur if service is not restored quickly. No workaround is available.
Cisco TAC Website
You can use the Cisco TAC website to resolve P3 and P4 issues yourself, saving both cost and time. The site provides around-the-clock access to online tools, knowledge bases, and software. To access the Cisco TAC website, go to this URL:
All customers, partners, and resellers who have a valid Cisco service contract have complete access to the technical support resources on the Cisco TAC website. Some services on the Cisco TAC website require a Cisco.com login ID and password. If you have a valid service contract but do not have a login ID or password, go to this URL to register:
If you are a Cisco.com registered user, and you cannot resolve your technical issues by using the Cisco TAC website, you can open a case online at this URL:
If you have Internet access, we recommend that you open P3 and P4 cases through the Cisco TAC website so that you can describe the situation in your own words and attach any necessary files.
Cisco TAC Escalation Center
The Cisco TAC Escalation Center addresses priority level 1 or priority level 2 issues. These classifications are assigned when severe network degradation significantly impacts business operations. When you contact the TAC Escalation Center with a P1 or P2 problem, a Cisco TAC engineer automatically opens a case.
To obtain a directory of toll-free Cisco TAC telephone numbers for your country, go to this URL:
Before calling, please check with your network operations center to determine the level of Cisco support services to which your company is entitled: for example, SMARTnet, SMARTnet Onsite, or Network Supported Accounts (NSA). When you call the center, please have available your service agreement number and your product serial number.
Obtaining Additional Publications and Information
Information about Cisco products, technologies, and network solutions is available from various online and printed sources.
•The Cisco Product Catalog describes the networking products offered by Cisco Systems as well as ordering and customer support services. Access the Cisco Product Catalog at this URL:
•Cisco Press publishes a wide range of networking publications. Cisco suggests these titles for new and experienced users: Internetworking Terms and Acronyms Dictionary, Internetworking Technology Handbook, Internetworking Troubleshooting Guide, and the Internetworking Design Guide. For current Cisco Press titles and other information, go to Cisco Press online at this URL:
•Packet magazine is the Cisco monthly periodical that provides industry professionals with the latest information about the field of networking. You can access Packet magazine at this URL:
•iQ Magazine is the Cisco monthly periodical that provides business leaders and decision makers with the latest information about the networking industry. You can access iQ Magazine at this URL:
•Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering professionals involved in the design, development, and operation of public and private internets and intranets. You can access the Internet Protocol Journal at this URL:
•Training—Cisco offers world-class networking training, with current offerings in network training listed at this URL:
This document is to be used in conjunction with the documents listed in the "Related Documentation" section.
Copyright © 2003 Cisco Systems, Inc. All rights reserved.