Configuring IP Storage Services

Cisco MDS 9000 Series IPStorage (IPS) services extend the reach of Fibre Channel SANs by using open-standard, IP-based technology. The switch connects separated SAN islands using Fibre Channel over IP (FCIP) and it allows IP hosts to access Fibre Channel storage.


Note


The FCIP feature is available in the modules with IPS ports and available in Cisco MDS 9250i, MDS 9220i, and Cisco MDS 9700 Director switches.


The Cisco MDS 24/10 Port SAN Extension Module for MDS 9700 series switch allows you to use Fibre Channel and FCIP features. The Cisco MDS 24/10 port SAN Extension Module can be used in any of the Cisco MDS 9700 series switches.

This chapter includes the following sections:

Feature Information

This section briefly describes the new and updated features for releases, starting from Cisco MDS NX-OS Release 9.x.

Table 1. Feature Information Table
Feature Release Description
Configuring 40 Gbps Speed on Cisco MDS 24/10 port SAN Extension Module 9.3(1) This feature allows you to configure 40-Gbps speed on the IPS ports of Cisco MDS 24/10 port SAN Extension Module.

Supported Hardware

You can configure the FCIP and iSCSI features on the following types of hardware:

  • Cisco MDS 24/10 port SAN Extension Module in Cisco MDS 9700 Series Director Switches and Cisco MDS 9220i Fabric Switches support FCIP.

  • Cisco MDS 9250i Multiservice Fabric Switches support iSCSI.

Upgrading FCIP and iSCSI Interfaces

ISSU/D is disruptive for FCIP and iSCSI interfaces along with the update of the underlying IPS and GigabitEthernet interfaces supporting these upper layer interfaces. Fibre Channel interfaces (including FC interfaces on the same module or switch as the IPS and GigabitEthernet interfaces), FCoE Ethernet interfaces, and supervisor modules are all updated nondisruptively.

Within Cisco MDS 9700 Directors, the modules with IPS ports implement a 5 minute delay before the next module with IPS ports is upgraded.

Configuring IPS Ports on Cisco MDS 9220i Switch

Starting from Cisco MDS NX-OS Release 8.5(1), the Cisco MDS 9220i Fabric Switch is supported. Cisco MDS 9220i switch supports six IPS ports. Ports 1 through 4 support 1- and 10-Gbps speed and port 6 supports 40-Gbps speed. By default, the first two ports are operating at a 1-Gbps speed mode and are licensed. To enable the other ports and different speed modes, you must obtain a license. You cannot move the default license from the first two ports to any other ports. For information on port licensing, see the Cisco MDS 9000 Series Licensing Guide.

Cisco MDS 9220i switch supports three 1 or 10 Gbps IPS ports, one 1, 10, 25 Gbps IPS port, one 25 Gbps IPS port and one 40 Gbps IPS port. All these ports are handled by a single Service Engine. The below table provides the different speed combinations that can be configured on the IPS ports of Cisco MDS 9220i switch.

The IPStorage interfaces operating in 25G-speed-mode require FEC RS-IEEE. The Ethernet interface of the connected switch must also be configured to use FEC RS-IEEE. If an IPStorage interface remains stuck in the init state, configure the Ethernet interface on the connected switch to use FEC RS-IEEE.

Table 2. IPS Port Speed Configuration Matrix for Cisco MDS 9220i Switch
Speed mode Port 1 Port 2 Port 3 Port 4 Port 5 Port 6
1 Gbps 1 Gbps 1 Gbps 1 Gbps 1 Gbps Out-of-service Out-of-service
10 Gbps 10 Gbps 10 Gbps 10 Gbps 10 Gbps Out-of-service Out-of-service
25 Gbps Out-of-service Out-of-service Out-of-service 25 Gbps 25 Gbps Out-of-service
40 Gbps Out-of-service Out-of-service Out-of-service Out-of-service Out-of-service 40 Gbps

Note


25-Gbps speed is supported from Cisco MDS NX-OS Release 9.3(1).


Configuring 40 Gbps Speed on Cisco MDS 24/10 port SAN Extension Module

From Cisco MDS NX-OS Release 8.5(1), you can configure 40 Gbps on the IPS ports 9 and 10 of Cisco MDS 24/10 port SAN Extension Module. Cisco MDS 24/10 port SAN Extension Module supports eight 1 or 10 Gbps IPS ports and two 40 Gbps IPS ports. Ports 1 through 4 and port 9 and handled by Service Engine 1 and Ports 5 through 8 and port 10 and handled by Service Engine 2. Each Service Engine has the capability of 40 Gbps. You can only configure 40 Gbps speed on ports 9 and 10. When you configure 40 Gbps speed on port 9, ports 1 through 4 are moved to out-of-service because the Service Engine capability is 40 Gbps. However, ports 5 through 8 can be configured for 1 or 10 Gbps speed provided port 10 is not configured for 40 Gbps speed. Similarly, when you configure 40 Gbps speed on port 10, ports 5 through 8 are moved to out-of-service. However, ports 1through 4 can be configured for 1 or 10 Gbps speed provided port 9 is not configured for 40 Gbps speed. The following table provides the different speed combinations that can be configured on the IPS ports of Cisco MDS 24/10 port SAN Extension Module.

The following are the recommended configurations for achieving maximum throughput on the 40 Gbps IPS port:

  • The maximum bandwidth of FCIP tunnel is 10 Gbps. Therefore, configure four FCIP tunnels on the 40 Gbps IPS port. To create the FCIP tunnels, create four VLAN sub interfaces on the 40 Gbps IPS port. On each VLAN create an FCIP tunnel.

  • Configure the number of TCP connections to 5.

Table 3. IPS Port Speed Configuration Matrix for Cisco MDS 24/10 port SAN Extension Module
Service Engine 1 Service Engine 2
Ports 1 through 4 Port 9 Ports 5 through 8 Port 10
1 Gbps Out-of-service 1 Gbps Out-of-service
10 Gbps Out-of-service 10 Gbps Out-of-service
1 Gbps Out-of-service 10 Gbps Out-of-service
10 Gbps Out-of-service 1 Gbps Out-of-service
Out-of-service 40 Gbps 1 Gbps Out-of-service
Out-of-service 40 Gbps 10 Gbps Out-of-service
1 Gbps Out-of-service Out-of-service 40 Gbps
10 Gbps Out-of-service Out-of-service 40 Gbps
Out-of-service 40 Gbps Out-of-service 40 Gbps

Configuring IPStorage Interfaces for IPv4

Both FCIP and iSCSI rely on TCP/IP for network connectivity. On each Fibre Channel module with IPS ports, connectivity is provided in the form of IP storage ports on Cisco MDS 9250i switches and Cisco MDS 9700 series switches with 24/10 port SAN Extension modules that are appropriately configured. This section covers the steps required to configure IP for subsequent use by FCIP and iSCSI.

A new port mode, called IPS, is defined for IP storage ports on each Fibre Channel module with IPS ports. IP storage ports are implicitly set to IPS mode, so it can only be used to perform iSCSI and FCIP storage functions. IP storage ports do not bridge Ethernet frames or route other IP packets.

Each IPS port represents a single virtual Fibre Channel host in the Fibre Channel SAN. All the iSCSI hosts connected to this IPS port are merged and multiplexed through the single Fibre Channel host.

In large scale iSCSI deployments where the Fibre Channel storage subsystems require explicit LUN access control for every host device, use of proxy-initiator mode simplifies the configuration.


Tip


IPStorage ports on any Fibre Channel module with IPS ports should not be configured in the same IPS broadcast domain as the management IPS port—they should be configured in a different broadcast domain, either by using separate standalone hubs or switches or by using separate VLANs.


Basic IPStorage Configuration

The following figure shows an example of a basic IPStorage IP version 4 (IPv4) configuration.

Figure 1. IPStorage IPv4 Configuration Example

Note


The port on the Ethernet switch to which the IPStorage interface is connected should be configured as a host port (also known as access port) instead of a switch port. Spanning tree configuration for that port (on the Ethernet switch) should disabled. This helps avoid the delay in the management port coming up due to delay from Ethernet spanning tree processing that the Ethernet switch would run if enabled. For Cisco Ethernet switches, use either the switchport host command in Cisco IOS or the set port host command in Catalyst OS.


Fibre Channel Module with IPS Ports Core Dumps

IPS core dumps are different from the system’s kernel core dumps for other modules. When the Fibre Channel module with IPS port’s operating system (OS) unexpectedly resets, it is useful to obtain a copy of the memory image (called a IPS core dump) to identify the cause of the reset. Under that condition, the Fibre Channel module with IPS ports sends the core dump to the supervisor module for storage. Cisco MDS switches have two levels of IPS core dumps:

  • Partial core dumps (default)—Each partial core dump consists of four parts (four files). All four files are saved in the active supervisor module.

    In Cisco MDS 9700 Series Switches with 24/10 port SAN Extension Modules, each partial core dump consists of five parts (five files). All five files are saved in the active supervisor module.

    Use the show cores command to list these files.

  • Full core dumps—Each full core dump of Cisco MDS 9250i Switches and SSN-16 modules consists of 64 parts (64 files), and each full core dump of Cisco MDS 9700 Series Switches with 24/10 port SAN Extension Modules consists of 67 parts (67 files). The IPS core dump for MSM-18/4 modules consists of 32 parts. This dump cannot be saved on the supervisor module because of its large space requirement. They are copied directly to an external TFTP server.

    Use the system cores tftp: command to configure an external TFTP server to copy the IPS core dump (and other core dumps).

To configure IPS core dumps on the Fibre Channel module with IPS ports, follow these steps:

Procedure

  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# ips core dump full

Configures a dump of the full core generation for all Fibre Channel module with IPS ports in the switch.

ips core dump full’ successfully set for module 9

Step 3

switch(config)# no ips core dump full

Configures a dump of the partial core (default) generation for the Fibre Channel module with IPS ports in slot 9.

 ips core dump partial’ successfully set for module 9

What to do next

Configuring Autonegotiation

By default, autonegotiation is enabled on all the IPStorage interface. You can enable or disable autonegotiation for a specified IPStorage interface. When autonegotiation is enabled, the port automatically detects the speed or pause method, and duplex of incoming signals based on the link partner. You can also detect link up conditions using the autonegotiation feature.

Configuring the MTU Frame Size

You can configure the interfaces on a switch to transfer large (or jumbo) frames on a port. The default IP maximum transmission unit (MTU) frame size is 1500 bytes for all Ethernet ports. By configuring jumbo frames on a port, the MTU size can be increased up to 9000 bytes.


Note


The minimum MTU size is 576 bytes.



Tip


MTU changes are disruptive, all FCIP links and iSCSI sessions flap when the software detects a change in the MTU size.


Configuring Promiscuous Mode

You can enable or disable promiscuous mode on a specific IPStorage interface. By enabling the promiscuous mode, the IPStorage interface receives all the packets and the software then filters and discards the packets that are not destined for that IPStorage interface.

About VLANs for IPStorage

Virtual LANs (VLANs) create multiple virtual Layer 2 networks over a physical LAN network. VLANs provide traffic isolation, security, and broadcast control.

IPStorage ports automatically recognize Ethernet frames with IEEE 802.1Q VLAN encapsulation. If you need to have traffic from multiple VLANs terminated on one IPStorage port, configure subinterfaces—one for each VLAN.

If the Fibre Channel module with IPS ports or MPS-14/2 module is connected to a Cisco Ethernet switch, and you need to have traffic from multiple VLANs coming to one IPS port, verify the following requirements on the Ethernet switch:

  • The encapsulation is set to 802.1Q and not ISL, which is the default.

  • Use the VLAN ID as a subscription to the IPStorage interface name to create the subinterface name:

     <slot-number> / <port-number>
    <VLAN-ID>

Interface Subnet Requirements

IPStorage interfaces (major), subinterfaces (VLAN ID), and management interfaces (mgmt 0) can be configured in the same or different subnet depending on the configuration (see the following table).

Table 4. Subnet Requirements for Ethernet Interfaces Within the Same Switch
Interface 1 Interface 2 Same Subnet Allowed Notes
IPStorage 1/1 IPStorage 1/2 Yes Two major interfaces can be configured in the same or different subnets.
IPStorage 1/1.100 IPStorage 1/2.100 Yes 1 Two subinterfaces with the same VLAN ID can be configured in the same or different subnets.
IPStorage 1/1.100 IPStorage 1/2.200 No Two subinterfaces with different VLAN IDs cannot be configured in the same subnet.
IPStorage 1/1 IPStorage 1/1.100 No A subinterface cannot be configured on the same subnet as the major interface.
mgmt0 IPStorage 1/1.100 No The mgmt0 interface cannot be configured in the same subnet as the IPStorage interfaces or subinterfaces.
mgmt0 IPStorage 1/1 No
1 Except IPSec enabled interfaces, which should not be configured in the same IP subnet.

Note


The configuration requirements in the above table applies to Ethernet PortChannels also.


Verifying IPStorage Connectivity

After the IPStorage interfaces are connected with valid IP addresses, verify the interface connectivity on each switch. Ping the IP host using the IP address of the host to verify that the static IP route is configured correctly.


Note


If the connection fails, verify the following, and ping the IP host again:

  • The IP address for the destination (IP host) is correctly configured.

  • The host is active (powered on).

  • The IP route is configured correctly.

  • The IP host has a route to get to the IPStorage interface subnet.

  • The IPStorage interface is in the up state.


IPStorage IPv4-ACL Guidelines


Tip


If IPv4-ACLs are already configured in a IPStorage interface, you cannot add this interface to an Ethernet PortChannel group.


Follow these guidelines when configuring IPv4-ACLs for IPStorage interfaces:

  • Only use Transmission Control Protocol (TCP) or Internet Control Message Protocol (ICMP).


    Note


    Other protocols such as User Datagram Protocol (UDP) and HTTP are not supported in IPStorage interfaces. Applying an ACL that contains rules for these protocols to a IPStorage interface is allowed but those rules have no effect.


  • Apply IPv4-ACLs to the interface before you enable an interface. This ensures that the filters are in place before traffic starts flowing.

  • Be aware of the following conditions:

    • If you use the log-deny option, a maximum of 50 messages are logged per second.

    • The established , precedence , and fragments options are ignored when you apply IPv4-ACLs (containing these options) to IPStorage interfaces.

    • If an IPv4-ACL rule applies to a preexisting TCP connection, that rule is ignored. For example if there is an existing TCP connection between A and B, and an IPv4-ACL specifies dropping all packets whose source is A and destination is B is subsequently applied, it will have no effect.

Configuring IPStorage High Availability

Virtual Router Redundancy Protocol (VRRP) and Ethernet PortChannels are two IPStorage features that provide high availability for iSCSI and FCIP services.

VRRP for iSCSI and FCIP Services

VRRP provides a redundant alternate path to the IPStorage port for iSCSI and FCIP services. VRRP provides IP address failover protection to an alternate IPStorage interface so the IP address is always available (see the below figure).

Figure 2. VRRP Scenario

In the above figure, all members of the VRRP group must be IPStorage ports. VRRP group members can be one or more of the following interfaces:

  • One or more interfaces in the same Fibre Channel module with IPS ports or MSM-18/4 module

  • Interfaces across Fibre Channel module with IPS ports or MSM-18/4 modules in one switch

  • Interfaces across Fibre Channel module with IPS ports or MSM-18/4 modules in different switches

  • IPStorage subinterfaces

  • Ethernet PortChannels and PortChannel subinterfaces

You can configure no more than seven VRRP groups, both IPv4 and IPv6, on a IPStorage interface, including the main interface and all subinterfaces.

Configuring VRRP for IPStorage Interfaces

To configure VRRP for IPStorage interfaces using IPv4 or IPv6, follow these steps:

Procedure


Step 1

switch1# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 2/2

Enters the interface configuration mode on the IPStorage interface (slot 2, port 2).

Step 3

To configure VRRP for IPStorage interfaces using IPv4:

  1. switch(config-if)# ip address 10.1.1.10 255.255.255.0

    Assigns the IPv4 address (10.1.1.10) and subnet mask (255.255.255.0) for the IPStorage interface.

  2. switch(config-if)# no shutdown

    Enables the selected interface.

  3. switch(config-if)# vrrp 100

    Creates VR ID 100.

  4. switch(config-if-vrrp) # address 10.1.1.100

    Configures the virtual IPv4 address (10.1.1.100) for the selected VRRP group (identified by the VR ID).

    Note

     

    The virtual IPv4 address must be in the same subnet as the IPv4 address of the IPStorage interface. All members of the VRRP group must configure the same virtual IPv4 address.

  5. switch(config-if-vrrp) # priority 10

    Configures the priority for the selected interface within this VRRP group.

    Note

     

    The interface with the highest priority is selected as the master.

  6. switch(config-if-vrrp) # no shutdown

    Enables the VRRP protocol on the selected interface.

Step 4

To configure VRRP for IPStorage interfaces using IPv6:

  1. switch(config-if)# ipv6 address 2001:0db8:800:200c::417a/64

    Assigns the IPv6 address for the IPStorage interface.

  2. switch(config-if)# no shutdown

    Enables the selected interface.

  3. switch(config-if)# vrrp ipv6 100

    Creates VR ID 100.

  4. switch(config-if-vrrp-ipv6) # address 2001:0db8:800:200c::417a

    Assigns single primary link-local IPv6 address or one of the multiple secondary IPv6 addresses.

    Note

     

    If this IPv6 address is the same as the physical IPv6 address, this switch is automatically the owner of this IPv6 address.

  5. switch(config-if-vrrp-ipv6) # priority 10

    Configures the priority for the selected interface within this VRRP group.

    Note

     

    The interface with the highest priority is selected as the master.

  6. switch(config-if-vrrp-ipv6) # no shutdown

    Enables the VRRP protocol on the selected interface.

Note

 

The VRRP preempt option is not supported on IPS interfaces. However, if the virtual IPv4 IP address is also the IPv4 IP address for the interface, then preemption is implicitly applied.


Configuring CDP

The Cisco Discovery Protocol (CDP) is supported on the management Ethernet interface on the supervisor module and the IPStorage interfaces on the Fibre Channel module with IPS ports or MSM-18/4 module.

For information about configuring CDP, refer to the Cisco MDS 9000 Series NX-OS Fundamentals Configuration Guide.

Changing Link Speed on IPStorage Interfaces

This sections includes the following topics:

Changing Link Speed on Cisco MDS 9250i Multiservice Fabric Switch

The Cisco MDS 9250i Multiservice Fabric Switch has two IPStorage interfaces that support 1 Gbps and 10 Gbps link speeds. By default, IPStorage interfaces are configured at 10 Gbps link speed.

Switching between different link speeds is supported on Cisco 10 Gbps IPStorage platforms starting from Cisco MDS NX-OS Release 6.2(13). An ISSD to a release earlier than Cisco MDS NX-OS Release 6.2(13) when any of the IPStorage ports are configured at 1 Gbps, is disallowed. Reconfigure such ports back to the default link speed of 10 Gbps before attempting such a downgrade.

Configuring 1 Gbps Link Speed on an IPStorage Interface

To configure 1 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch1# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage slot-number/port-number-range

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# shutdown

Administratively disables the interface and stops traffic through the interface.

Step 4

switch(config-if)# switchport speed 1000 2

Example:
 This speed change will disrupt FCIP/iSCSI
traffic for 5 mins on all  IPStorage ports. If FCIP tunnels are configured please make sure
max-bw  <= 1000 Mbps and tcp-connections set to 2. Do you want to  continue(y/n) ? [n]

Sets the link speed of the interface and all subinterfaces to 1000 Mbps (1 Gbps).

Note

 

This command causes all IPStorage ports on the selected FCIP engine to be reset. This may cause traffic disruption for up to 5 minutes. By default, n is selected. Press Enter to abort the command. Enter y and press Enter to continue.

3

Step 5

switch(config-if) # no shutdown

Administratively enables the interface.

Step 6

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Step 7

switch# show ips status

Displays the operational speed of the IPStorage port.

2 Configuring the link speed of an interface generates the following port software failure syslog message: %IF_DOWN_SOFTWARE_FAILURE: %$VSAN 1%$ Interface fcip is down (Port software failure)
3 If the conditions specified in the warning message are not met, the configured link speed is still applied. However, issues such as packet drops, retransmissions, and FCIP tunnel flaps may occur.

Configuring 10 Gbps Link Speed on an IPStorage Interface

To configure 10 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch1# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage slot-number/port-number-range

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# shutdown

Administratively disables the interface and stops traffic through the interface.

Step 4

switch(config-if)# switchport speed 10000 4

Sets the link speed of the interface and all subinterfaces to 10000 Mbps (10 Gbps).

Note

 

This command causes all IPStorage ports on the selected FCIP engine to be reset. This may cause traffic disruption for up to 5 minutes. By default, n is selected. Press Enter to abort the command. Enter y and press Enter to continue.

5

Step 5

switch(config-if) # no shutdown

Administratively enables the interface.

Step 6

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Step 7

switch# show ips status

Displays the operational speed of the IPStorage port.

4 Configuring the link speed of an interface generates the following port software failure syslog message:%IF_DOWN_SOFTWARE_FAILURE: %$VSAN 1%$ Interface fcip is down (Port software failure)
5 If the conditions specified in the warning message are not met, the configured link speed is still applied. However, issues such as packet drops, retransmissions, and FCIP tunnel flaps may occur.
What to do next

If there is a mismatch between the configured link speed and the small form-factor pluggable (SFP) speed capabilities, the port goes into an Error Disabled state and a corresponding syslog message is logged. In such a scenario, either the configured link speed or the SFP should be changed. If the link speed is changed, even if the port is already enabled, the shutdown and no shutdown commands must be explicitly issued for the change to be applied.

For more information about supported 1 Gbps SFPs for a Cisco MDS 9250i Multiservice Fabric Switch, see the Cisco MDS 9000 Family Pluggable Transceivers Data Sheet .

For information about configuring FCIP tunnels with IPStorage interfaces at 1 Gbps speed, see Configuring FCIP.

Changing Link Speed on Cisco MDS 9220i Switch


Note


For Cisco MDS 9220i switch, delete all the FCIP related configurations, such as profiles, tunnels, and so on, before switching from 1 or 10 Gbps speed to 40 Gbps speed or vice versa.


Configuring 1 Gbps Link Speed on an IPStorage Interface

To configure 1 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 1G-speed-mode

Example:
This speed change will disrupt FCIP/iSCSI
traffic for 60 seconds on  selected IPStorage ports.
If FCIP tunnels are configured please
make sure  max-bw <= 1000 Mbps and tcp-connections set to 2.
Do you wish to continue(y/n)? [n]

Sets the link speed of the interface IPS1/1-4 and all subinterfaces to 1000 Mbps (1 Gbps) and administratively enables the interface.

Sets IPS1/5-6 out-of-service.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Configuring 10 Gbps Link Speed on an IPStorage Interface

To configure 10 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 10G-speed-mode

Example:
This speed change will disrupt FCIP/iSCSI traffic 
for 60 seconds on select IPStorage ports.
Do you wish to continue(y/n)? [n]

Sets the link speed of the interface IPS1/1-4 and all subinterfaces to 10000 Mbps (10 Gbps) and administratively enables the interface.

Sets IPS1/5-6 out-of-service.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Configuring 25 Gbps Link Speed on an IPStorage Interface

To configure 25 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

Enters IPStorage interface configuration mode.

Note

 

Configure the number of TCP connections to 5 for attaining maximum throughput.

Step 3

switch(config-if)# 25G-speed-mode

Sets the link speed of the interface IPS1/4-5 to 25000 Mbps (25 Gbps) and administratively enables the interface IPS1/4-5.

Sets IPS1/1-3 and IPS1/6 to out-of-service.

Note

 

Configure the connected Ethernet interface to use FEC RS-IEEE when using IPStorage ports in 25g mode.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Configuring 40 Gbps Link Speed on an IPStorage Interface

To configure 40 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

Enters IPStorage interface configuration mode.

Note

 

Configure the number of TCP connections to 5 for attaining maximum throughput.

Step 3

switch(config-if)# 40G-speed-mode

Sets the link speed of the interface IPS1/6 to 40000 Mbps (40 Gbps) and administratively enables the interface.

Sets IPS1/1-5 to out-of-service.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Changing Link Speed on Cisco MDS 24/10 port SAN Extension Module


Note


For Cisco MDS 24/10 port SAN Extension Module, delete all the FCIP related configurations, such as profiles, tunnels, and so on, before switching from 1 or 10 Gbps speed to 40 Gbps speed or vice versa.


Configuring 1 Gbps Link Speed on an IPStorage Interface

To configure 1 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage slot-number/port-number-range, IPStorage slot-number/port-number

Enters IPStorage interface configuration mode.

Note

 

The values for port-number-range can be 1-4 or 5-8. The value for port number can be 9 if port-number-range is 1-4 and the value for port number can be 10 if port-number-range is 5-8.

Step 3

switch(config-if)# 1G-speed-mode

Example:
This speed change will disrupt FCIP/iSCSI
traffic for 60 seconds on  selected IPStorage ports.
If FCIP tunnels are configured please make sure  
max-bw <= 1000 Mbps and tcp-connections set to 2.
 Do you wish to continue(y/n)? [n] 

Sets the link speed of the interface and all subinterfaces to 1000 Mbps (1 Gbps) and administratively enables the interface.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Configuring 10 Gbps Link Speed on an IPStorage Interface

To configure 10 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage slot-number/port-number-range, IPStorage slot-number/port-number

Enters IPStorage interface configuration mode.

Note

 

The values for port-number-range can be 1-4 or 5-8. The value for port number can be 9 if port-number-range is 1-4 and the value for port number can be 10 if port-number-range is 5-8.

Step 3

switch(config-if)# 10G-speed-mode

Example:
This speed change will disrupt FCIP/iSCSI traffic for 
60 seconds on selected IPStorage ports.
 Do you wish to continue(y/n)? [n] 

Sets the link speed of the interface and all subinterfaces to 10000 Mbps (10 Gbps) and administratively enables the interface.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Configuring 40 Gbps Link Speed on an IPStorage Interface

To configure 40 Gbps link speed on an IPStorage interface, follow these steps:

Procedure
  Command or Action Purpose

Step 1

switch# config terminal

Enters configuration mode.

Step 2

switch(config)# interface IPStorage slot-number/port-number-range, IPStorage slot-number/port-number

Enters IPStorage interface configuration mode.

Note

 

The values for port-number-range can be 1-4 or 5-8. The value for port number can be 9 if port-number-range is 1-4 and the value for port number can be 10 if port-number-range is 5-8.

Step 3

switch(config-if)# 40G-speed-mode

Sets the link speed of the interface to 40 Gbps and interfaces in the range to out-of-service.

Step 4

switch(config-if)# end

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Displaying Statistics

This section provides examples to verify IPStorage interfaces and TCP/IP statistics on the IPStorage ports.

Use the show interface IPStorage command on each switch to verify that the interfaces are up and functioning as desired.

Displaying the IPStorage Interface

switch# show interface IPStorage 4/1
IPStorage4/1 is up
Hardware is IPStorage, address is 008e.7339.39e7
Internet address is 10.197.141.81/24
MTU 2500 bytes
Port mode is IPS
Speed is 10 Gbps
Beacon is turned off
Auto-Negotiation is turned on
5 minutes input rate 77012744 bits/sec, 9626593 bytes/sec, 112755 frames/sec
5 minutes output rate 2762915176 bits/sec, 345364397 bytes/sec, 175258 frames/sec
71187036 packets input, 6078261484 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
110617842 packets output, 217860230652 bytes, 0 underruns
0 output errors, 0 collisions, 0 fifo
0 carrier errors
switch# show interface IPStorage 5/1-10 brief
 
--------------------------------------------------------------------------------
Interface Status IP Address Speed MTU
--------------------------------------------------------------------------------
IPStorage5/1 up 1.1.1.1/24 10 Gbps 2500
IPStorage5/2 up 2.2.2.2/24 10 Gbps 2500
IPStorage5/3 up 3.3.3.3/24 10 Gbps 2500
IPStorage5/4 up 4.4.4.4/24 10 Gbps 2500
IPStorage5/5 up 6811::3456/64 1 Gbps 2300
IPStorage5/6 up 9.9.9.1/24 1 Gbps 2500
IPStorage5/7 up 7.7.7.1/24 1 Gbps 2500
IPStorage5/8 up 8.8.8.1/24 1 Gbps 2500
IPStorage5/9 outOfServc -- auto 1500
IPStorage5/10 outOfServc -- auto 1500

Displaying Ethernet MAC Statistics

The show ips stats mac interface ips command takes the IPStorage interface as a parameter and returns the statistics for that interface.


Note


Use the physical interface, not the subinterface, to display Ethernet MAC statistics.


Displaying IPStorage Interface MAC Statistics

switch# show ips stats mac interface ips 4/1
 
DPP HW IPStorage port 4 (octeon port 19) statistics
 
Rx stats : 
  dropped         : 0          octs,   0          pkts 
  l2 red pkts     : 0          mcast,  0          bcasts
  l3 red pkts     : 0          mcast,  0          bcasts
  oversize        : 0          pkts,   0          crcpkts
  runt            : 0          pkts,   0          crcpkts
  inband          : 0          octs,   0          pkts,      0      err
  pci raw         : 0          pkts
  fcs_align_err   : 0          pkts
 
  total           : 0          octs,   0          pkts
 
  length of [pkts]:- 
   [64B]          : 0            [65B-127B]       : 0           
   [128B-255B]    : 0            [256B-511B]      : 0         
   [512B-1023B]   : 0            [1024B-1518B]    : 0         
   [1519B-MAX]    : 0         
 
Tx stats : 
  total           : 0          octs,   0          pkts
  dropped         : 0          octs,   0          pkts
 
Hw-intf stats : 
  total           : 0            Rx_octs,   0            Rx_pkts
  dropped         : 0            Rx_pkts
  total           : 0            Tx_octs,   0            Tx_pkts
  dropped         : 0            Tx_pkts

Displaying TCP Statistics

Use the show ips stats tcp interface ips to display and verify TCP statistics. This command takes the main Ethernet interface as a parameter, and shows TCP stats along with the connection list and TCP state. The detail option shows all information maintained by the interface. See the following examples.

Displaying TCP Statistics

switch# show ips stats tcp interface ips 4/1
 
TCP statistics for port IPStorage4/1
Connection Stats
0 active openings, 3 accepts
0 failed attempts, 12 reset received, 3 established
Segment stats
163 received, 355 sent, 0 retransmitted
0 bad segments received, 0 reset sent
TCP Active Connections
Local Address Remote Address State Send-Q Recv-Q
0.0.0.0:3260 0.0.0.0:0 LISTEN 0 0

Displaying Detailed TCP Statistics

switch# show ips stats tcp interface ips 4/1 detail
 
TCP Statistics for port IPStorage4/1
TCP send stats
355 segments, 37760 bytes
222 data, 130 ack only packets
3 control (SYN/FIN/RST), 0 probes, 0 window updates
0 segments retransmitted, 0 bytes
0 retransmitted while on ethernet send queue, 0 packets split
0 delayed acks sent
TCP receive stats
163 segments, 114 data packets in sequence, 6512 bytes in sequence
0 predicted ack, 10 predicted data
0 bad checksum, 0 multi/broadcast, 0 bad offset
0 no memory drops, 0 short segments
0 duplicate bytes, 0 duplicate packets
0 partial duplicate bytes, 0 partial duplicate packets
0 out-of-order bytes, 1 out-of-order packets
0 packet after window, 0 bytes after window
0 packets after close
121 acks, 37764 ack bytes, 0 ack toomuch, 4 duplicate acks
0 ack packets left of snd_una, 0 non-4 byte aligned packets
8 window updates, 0 window probe
30 pcb hash miss, 0 no port, 0 bad SYN, 0 paws drops
TCP Connection Stats
0 attempts, 3 accepts, 3 established
3 closed, 2 drops, 0 conn drops
0 drop in retransmit timeout, 1 drop in keepalive timeout
0 drop in persist drops, 0 connections drained
TCP Miscellaneous Stats
115 segments timed, 121 rtt updated
0 retransmit timeout, 0 persist timeout
12 keepalive timeout, 11 keepalive probes
TCP SACK Stats
0 recovery episodes, 0 data packets, 0 data bytes
0 data packets retransmitted, 0 data bytes retransmitted
0 connections closed, 0 retransmit timeouts
TCP SYN Cache Stats
15 entries, 3 connections completed, 0 entries timed out
0 dropped due to overflow, 12 dropped due to RST
0 dropped due to ICMP unreach, 0 dropped due to bucket overflow
0 abort due to no memory, 0 duplicate SYN, 0 no-route SYN drop
0 hash collisions, 0 retransmitted
TCP Active Connections
Local Address Remote Address State Send-Q Recv-Q
0.0.0.0:3260 0.0.0.0:0 LISTEN 0 0

Use the show ips stats icmp interface ips to display and verify IP statistics. This command takes the main Ethernet interface as a parameter and returns the ICMP statistics for that interface.

Displaying ICMP Statistics

switch# show ips stats icmp interface ips 4/1
 
ICMP Statistics for port IPStorage4/1
0 ICMP messages received
0 ICMP messages dropped due to errors
ICMP input histogram
0 destination unreachable
0 time exceeded
0 parameter problem
0 source quench
0 redirect
0 echo request
0 echo reply
0 timestamp request
0 timestamp reply
0 address mask request
0 address mask reply
ICMP output histogram
0 destination unreachable
0 time exceeded
0 parameter problem
0 source quench
0 redirect
0 echo request
0 echo reply
0 timestamp request
0 timestamp reply
0 address mask request
0 address mask reply

Displaying IPStorage Ports Speed

switch# show ips status
Port 1/1 READY 10G
Port 1/2 READY 1G

Default Settings for IPStorage Services Parameters

The following table lists the default settings for IPStorage services parameters.

Table 5. Default IPStorage Parameters
Parameters Default
IPS core size Partial