Guest

Cisco Catalyst 6000 Series Switches

Configuring and Troubleshooting IP MLS on Catalyst 6500/6000 Switches with an MSFC

Document ID: 10566

Updated: Aug 30, 2005

   Print

Introduction

This document is a guide on verifying and how to read Multilayer Switching (MLS) commands on the Catalyst 6500/6000. The document contains a very brief review of what MLS is, and also gives examples of how to use MLS. Based on these examples, this document shows how to verify the MLS operation, and provides brief troubleshooting tips for configuring MLS.

This document applies only to the Catalyst 6500/6000 series switch equipped with the following hardware:

  • Supervisor Engine 1A running Catalyst OS (CatOS) Software

  • Policy Feature Card (PFC)

  • Multilayer Switch Feature Card (MSFC)

Note: This document is not valid when using any other hardware configuration such as Supervisor Engine 2 or Multilayer Switch Module (MSM). It is also not valid when running Cisco IOS® Software on both the Supervisor Engine 1A and MSFC.

For similar information regarding troubleshooting unicast routing on a Catalyst 6500/6000 series switch with Supervisor Engine 2 and running CatOS software, refer to Troubleshoot Unicast IP Routing Involving CEF on Catalyst 6500/6000 Series Switches with a Supervisor Engine 2 and Running CatOS System Software.

For a more complete description of MLS terminology and operation, see the Related Information

Related Cisco Support Community Discussions section.

Before You Begin

Conventions

For more information on document conventions, see the Cisco Technical Tips Conventions.

Prerequisites

There are no specific prerequisites for this document.

Components Used

The information in this document is based on the software and hardware versions below.

  • Catalyst 6500/6000 with an MSFC

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Overview of MLS on the MSFC

The MSFC is the second generation routing engine for the Catalyst 6500/6000 series switch that can route 15 million packets per second. The MSFC only works with Supervisor Engines that have the PFC. The MSFC performs an internal MLS with the PFC, which acts similar to the NetFlow Feature Card (NFFC) on the Catalyst 5000. This internal MLS is not visible and is purely limited to the switch: you have nothing to configure to make it work and it supports hardware shortcuts for IP, IPX, and IP multicast. The configuration of the MSFC is similar to the configuration of an RSM or an RSFC using VLAN interfaces. You can access it using session 15 (for MSFC on the Supervisor Engine in slot 1) or session 16 (for MSFC on the Supervisor Engine in slot 2).

The principle is similar to Multilayer Switching Protocol (MLSP) on the Catalyst 5000. The first packet is routed by the MSFC, and the PFC creates a shortcut that is used by all subsequent packets of the same flow. Unlike MLSP on the Catalyst 5000, which requires IP communication between the MLS-SE and the MLS-RP, MLS on the Catalyst 6500/6000 works by communication between the MSFC and PFC over a serial channel (SCP).

The PFC cannot be the MLS-SE for a Catalyst 5000 MLS environment; however, the MSFC can be MLS-RP for other Catalyst 5000s in the network. In that case, you should configure the MSFC using the same mls rp ip command as you would for any Cisco IOS router used as the MLS RP.

mls.gif

Example 1: MSFC Routing Between Two VLANs

The MLS on the Catalyst 6500/6000 for unicast IP is plug and play. You do not need to configure it. Below is a sample configuration where tamer is a Catalyst 6500/6000, which has an MSFC called tamer-msfc. Routing between VLAN 11 and 12 is configured on the MSFC without a single command related to MLS. Keep in mind that the Supervisor Engine will not have any MLS-specific configuration.

tamer-msfc#wr t 
Building configuration... 

Current configuration: 
! 
version 12.1 
no service pad 
service timestamps debug uptime 
service timestamps log uptime 
no service password-encryption 
! 
hostname tamer-msfc 
! 
boot system flash bootflash:c6msfc-ds-mz.121-1.E2.bin 
! 
ip subnet-zero 
ip cef 
! 
interface Vlan11 
 ip address 11.1.1.2 255.255.255.0 
! 
interface Vlan12 
 ip address 12.1.1.2 255.255.255.0 
! 
router eigrp 1 
 network 11.0.0.0 
 network 12.0.0.0 
 no auto-summary 
! 
ip classless 
no ip http server 
! 
! 
line con 0 
 transport input none 
line vty 0 4 
 login 
! 
end

Reading show mls Entries and Basic Checking

Below is the output that you should get when issuing the show mls command on the Supervisor Engine.

tamer (enable) show mls 
Total packets switched = 8      

!--- Number of packet shortcuted by MLS.

Total Active MLS entries = 0      

!--- Number of flows currently existing.

MSFC 11.1.1.2 (Module 15) entries = 0 
IP Multilayer switching aging time = 256 seconds 
IP Multilayer switching fast aging time = 0 seconds, packet threshold = 0 
IP Current flow mask is Destination flow 
Active IP MLS entries = 0 
Netflow Data Export version: 7 
Netflow Data Export disabled 
Netflow Data Export port/host is not configured. 
Total packets exported = 0 

IP MSFC ID      Module XTAG MAC               Vlans 
--------------- ------ ---- ----------------- ---------------- 
11.1.1.2        15     1    00-d0-d3-9c-9e-3c 12,11      

!--- MSFC recognized by the switch.

The show mls command output should always have an IP MSFC ID. If you do not see the IP MSFC ID when you issue the show mls command, verify the following:

  • MSFC is up and running (not stuck in ROMmon mode, for example).

  • MLS is still enabled on the MSFC.

You can verify this by issuing the following commands on the MSFC:

tamer-msfc#show mls status 
MLS global configuration status: 
global mls ip:                     enabled     

!--- Should be enabled for unicast IP.

global mls ipx:                    enabled 
global mls ip multicast:           disabled 
current ip flowmask for unicast:   destination only 
current ipx flowmask for unicast:  destination only 

By issuing the show mls status command, you can determine whether MLS is enabled for IP, IPX, and IP multicast. These features should always be enabled by default; however, they can be disabled by issuing the following command in configuration mode:

no mls ip

The no mls ip command should only be used for debugging purposes. The command is also available as a hidden command in global configuration mode. You can also disable MLS on a per-VLAN interface basis by issuing the no mls ip command in interface configuration mode

Note: Do not issue the show mls rp command on the MSFC. This command output indicates that MLS is disabled. However, the show mls command output issued on the Supervisor Engine above indicated that MLS was working correctly. This discrepancy occurs because the show mls rp command should be used when doing MLS-rp in conjunction with a Catalyst 5000 switch.

tamer-msfc#show mls rp 
ip multilayer switching is globally disabled 
ipx multilayer switching is globally disabled 
ipx mls inbound acl override is globally disabled 
mls id is 00d0.d39c.9e04 
mls ip address 0.0.0.0 
mls ip flow mask is unknown 
mls ipx flow mask is unknown 
number of domains configured for mls 0

A candidate packet is a packet that can potentially initiate the setup of a MLS shortcut. Its destination MAC address is equal to the MAC address of a router running MLS. In this case, the MAC address of the MSFC is 00-d0-d3-9c-9e-3c (seen by issuing the show mls command). To verify that the switch knows that this MAC address is a router MAC address, issue the show cam mac_address command, as shown below.

tamer (enable) show cam 00-d0-d3-9c-9e-3c 
* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry. 
X = Port Security Entry 
VLAN  Dest MAC/Route Des    [CoS]  Destination Ports or VCs / [Protocol Type] 
----  ------------------    -----  ------------------------------------------- 
11    00-d0-d3-9c-9e-3c R#          15/1 
12    00-d0-d3-9c-9e-3c R#          15/1 
Total Matching CAM Entries Displayed = 2

This output confirms that the switch knows that this MAC address is a router entry linked to port 15/1 (MSFC port in slot 1).

If you still do not see the MSFC when you issue the command show mls on the switch Supervisor Engine, issue the following command:

tamer (enable) show mls rlog l2 
SWLOG at 815d0c50: magic 1008, size 51200, cur 815d4170, end 815dd460 
Current time is: 08/08/00,17:13:25 

118 08/08/00,17:13:16:(RouterConfig)Router_cfg: router_add_MAC_to_earl 00-d0-d3-
9c-9e-3c  added for mod 15/1 Vlan 12 Earl AL =0 
117 08/08/00,17:13:16:(RouterConfig)Router_Cfg: Process add mls entry for mod 15
/1 vlan 12, i/f 1, proto 0, LC 3 
116 08/08/00,17:13:16:(RouterConfig)Router_cfg: router_add_MAC_to_earl 00-d0-d3-
9c-9e-3c  added for mod 15/1 Vlan 11 Earl AL =0 
115 08/08/00,17:13:16:(RouterConfig)Router_Cfg: Process add mls entry for mod 15
/1 vlan 11, i/f 1, proto 0, LC 3

This command shows you the messages the switch is receiving from the MSFC, and that the router entries are added.

Reading show mls Output Tables

Issue the show mls entry command to view the full MLS table with all the shortcuts. The output below shows all of the flows received.

tamer (enable) show mls entry 
Destination-IP  Source-IP       Prot  DstPrt SrcPrt Destination-MAC   Vlan EDst 
--------------- --------------- ----- ------ ------ ----------------- ---- ---- 
MSFC 11.1.1.2 (Module 15): 
10.68.5.1       -               -     -      -      00-d0-00-3f-a3-ff 8    ARPA 
12.1.1.1        -               -     -      -      00-00-0c-8c-70-88 12   ARPA 
11.1.1.1        -               -     -      -      00-00-0c-09-50-66 11   ARPA 
ESrc DPort  SPort  Stat-Pkts  Stat-Bytes  Uptime   Age 
---- ------ ------ ---------- ----------- -------- -------- 
ARPA 1/3    7/3    4          400         00:00:02 00:00:02 
ARPA 7/4    7/3    4          400         00:00:08 00:00:08 
ARPA 7/3    7/4    9          900         00:00:08 00:00:08 


Destination-IPX           Destination-MAC   Vlan EDst ESrc  Port  Stat-Pkts  
------------------------- ----------------- ---- ---- ---- ----- ---------- 
Stat-Bytes  Uptime   Age 
----------- -------- -------- 

MSFC 11.1.1.2 (Module 15): 

Total entries displayed: 2 
tamer (enable)

Note: One flow is created per destination. A ping from 12.1.1.1 to 11.1.1.1 will create two flows (one for each direction), as indicated by the last two lines in the output shown above.

The following are some descriptions of the information that is found in the table:

  • Destination IP, source IP, Prot, DstPrt, and SrcPrt are the fields used to create shortcuts. In this case, destination only flow is used. Only the destination IP address of a flow is cached. This can be changed by modifying the flowmask, which is described later in this document.

  • Destination MAC is the MAC address that will be used to rewrite the destination MAC of the packet. The source MAC address is rewritten with the MAC address of the MSFC.

  • VLAN indicates the destination VLAN needed to reach that IP address. The destination VLAN is important, for example, if the packet needs to be sent on a trunk.

  • DPort and Sport are the destination and source port of the flow.

  • Stat-Pkts and Stat-Bytes give you statistics on the number of packets that have used this shortcut since the creation of the flow.

  • Uptime is the time since the flow has been created.

  • Age is the elapsed time since the flow was last used.

Change the flow to destination-source. The show mls entry command output displays both the source IP address and destination IP address are caching. A different flow for each source IP address communicating to the same destination IP address is now created, as shown below.

tamer (enable) set mls flow destination-source 
Configured IP flowmask is set to destination-source flow. 
Warning: Configuring more specific flow mask may increase the number of MLS 
 entries dramatically. 
tamer (enable) 2000 Aug 09 17:05:12 %MLS-5-FLOWMASKCHANGE:IP flowmask changed 
 from DEST to DEST-SRC 

tamer (enable) show mls entry 
Destination-IP  Source-IP       Prot  DstPrt SrcPrt Destination-MAC   Vlan EDst 
--------------- --------------- ----- ------ ------ ----------------- ---- ---- 
MSFC 11.1.1.2 (Module 15): 
11.1.1.1        12.1.1.1        -     -      -      00-00-0c-09-50-66 11   ARPA 
11.1.1.1        10.68.5.1       -     -      -      00-00-0c-09-50-66 11   ARPA 
10.68.5.1       11.1.1.1        -     -      -      00-d0-00-3f-a3-ff 8    ARPA 
12.1.1.1        11.1.1.1        -     -      -      00-00-0c-8c-70-88 12   ARPA 
MSFC 0.0.0.0 (Module 16): 

ESrc DPort  Sport  Stat-Pkts  Stat-Bytes  Uptime   Age 
---- ------ ------ ---------- ----------- -------- -------- 
ARPA 7/3    7/4    4          400         00:00:02 00:00:02 
ARPA 7/3    1/3    4          400         00:00:32 00:00:32 
ARPA 1/3    7/3    4          400         00:00:32 00:00:32 
ARPA 7/4    7/3    4          400         00:00:02 00:00:02 


Destination-IPX           Destination-MAC   Vlan EDst ESrc  Port  
------------------------- ----------------- ---- ---- ---- ----- ---------- 
Stat-Pkts  Stat-Bytes  Uptime   Age 
----------- -------- -------- 

MSFC 11.1.1.2 (Module 15): 
MSFC 0.0.0.0 (Module 16): 


Total entries displayed: 4
tamer (enable)

The third option is set MLS to full flow. Perform a few pings and Telnet sessions to see how the different flows are created for each TCP port. Below is how the MLS table should look after making a few pings and Telnet sessions. Using full flow, the number of flows created increases very rapidly. The TCP port information is cached, and appears in the MLS table.

tamer (enable) set mls flow full 
Configured IP flowmask is set to full flow. 
Warning: Configuring more specific flow mask may increase the number of MLS 
 entries dramatically. 
Tamer (enable) 2000 Aug 09 17:30:01 %MLS-5-FLOWMASKCHANGE:IP flowmask changed
 from DEST to FULL 

tamer (enable) 
tamer (enable) show mls entry 
Destination-IP  Source-IP       Prot  DstPrt SrcPrt Destination-MAC   Vlan EDst 
--------------- --------------- ----- ------ ------ ----------------- ---- ---- 
MSFC 11.1.1.2 (Module 15): 
12.1.1.1        11.1.1.1        ICMP  -      -      00-00-0c-8c-70-88 12   ARPA 
11.1.1.1        12.1.1.1        TCP   11001  Telnet 00-00-0c-09-50-66 11   ARPA 
12.1.1.1        11.1.1.1        TCP*  Telnet 11001  00-00-0c-8c-70-88 12   ARPA 
11.1.1.1        10.68.5.1       TCP   11002  Telnet 00-00-0c-09-50-66 11   ARPA 
10.68.5.1       11.1.1.1        ICMP  -      -      00-d0-00-3f-a3-ff 8    ARPA 
10.68.5.1       11.1.1.1        TCP*  Telnet 11002  00-d0-00-3f-a3-ff 8    ARPA 
11.1.1.1        10.68.5.1       ICMP  -      -      00-00-0c-09-50-66 11   ARPA 
11.1.1.1        12.1.1.1        ICMP  -      -      00-00-0c-09-50-66 11   ARPA 

ESrc DPort  Sport  Stat-Pkts  Stat-Bytes  Uptime   Age 
---- ------ ------ ---------- ----------- -------- -------- 
ARPA 7/4    7/3    4          400         00:00:30 00:00:30 
ARPA 7/3    7/4    16         688         00:00:26 00:00:24 
ARPA 7/4    7/3    18         757         00:00:26 00:00:24 
ARPA 7/3    1/3    61         4968        00:00:16 00:00:06 
ARPA 1/3    7/3    4          400         00:00:33 00:00:33 
ARPA 1/3    7/3    69         2845        00:00:17 00:00:06 
ARPA 7/3    1/3    4          400         00:00:33 00:00:33 
ARPA 7/3    7/4    4          400         00:00:32 00:00:31 

  
Destination-IPX           Destination-MAC   Vlan EDst ESrc  Port  Stat-Pkts  
------------------------- ----------------- ---- ---- ---- ----- ---------- 
Stat-Bytes  Uptime   Age 
----------- -------- -------- 

MSFC 11.1.1.2 (Module 15): 

Total entries displayed: 8

Notes

  • In a live network, the number of flows created can be up to several thousands. Issue the show mls entry ip [destination|source] command to display specific flow instead of displaying the full flow table, as shown below.

    tamer (enable) show mls entry ip destination 12.1.1.1 
    
    Destination-IP  Source-IP       Prot  DstPrt SrcPrt Destination-MAC   Vlan EDst 
    --------------- --------------- ----- ------ ------ ----------------- ---- ---- 
    
    MSFC 11.1.1.2 (Module 15): 
    12.1.1.1        -               -     -      -      00-00-0c-8c-70-88 12   ARPA 
    
    
    ESrc DPort  Sport  Stat-Pkts  Stat-Bytes  Uptime   Age 
    ---- ------ ------ ---------- ----------- -------- -------- 
    ARPA 7/4    7/3    4          400         00:00:30 00:00:30 
    
  • You can verify the statistics for the flow by issuing the command show mls statistics, as shown below.

    tamer (enable) show mls statistics entry ip 15 
                                      Last    Used 
    Destination IP  Source IP       Prot DstPrt SrcPrt Stat-Pkts  Stat-Bytes 
    --------------- --------------- ---- ------ ------ ---------- --------------- 
    MSFC 11.1.1.2 (Module 15): 
    12.1.1.1        11.1.1.1        ICMP -      -      9          900 
    11.1.1.1        10.68.5.1       TCP  11005  Telnet 20         913 
    11.1.1.1        10.68.5.1       TCP  11004  Telnet 0          0 
    10.68.5.1       11.1.1.1        ICMP -      -      4          400 
    10.68.5.1       12.1.1.1        ICMP -      -      9          900 
    12.1.1.1        10.68.5.1       ICMP -      -      4          400 
    11.1.1.1        10.68.5.1       ICMP -      -      4          400 
    11.1.1.1        12.1.1.1        ICMP -      -      9          900

Troubleshooting a Specific Connection

If you are having a connectivity problem to a specific IP address or between two specific hosts, try the following to troubleshoot:

  • Issue the show mls entry ip [destination|source] command to see if the flow has been created.

  • Issue the show mls statistics entry [source|destination] command several times in a row to see if the counters of stat-pakts for that shortcut are increasing.

  • Verify the relevant flow.

For example, for an FTP session of a big file between TFTP server 12.1.1.1 and TFTP client 11.1.1.1, you need to verify the following two flows:

  • One with destination 12.1.1.1 that should be hit only by the TFTP acknowledgment (small packet) (source of the flow 12.1.1.1 if destination-source flow used).

  • One with destination 11.1.1.1 that should be hit by a lot of big packets (the actual file transfer) (source of the flow 11.1.1.1 if destination-source flow is used).

    This is an example of TFTP between 12.1.1.1. and 11.1.1.1 of a file of about 7.6 MB. Following is the MLS stat table before the start of the TFTP:

    tamer (enable) show mls statistics entry 
                                      Last    Used 
    Destination IP  Source IP       Prot DstPrt SrcPrt Stat-Pkts  Stat-Bytes 
    --------------- --------------- ---- ------ ------ ---------- --------------- 
    MSFC 11.1.1.2 (Module 15): 
    12.1.1.1        11.1.1.1        ICMP -      -      4          400 
    11.1.1.1        12.1.1.1        ICMP -      -      4          400 
    12.1.1.1        11.1.1.1        TCP  11000  Telnet 20         894
    

The TFTP has just started. The two additional flows created for TFTP traffic (UDP port 69) are shown below.

tamer (enable) show mls statistics entry 
                                  Last    Used 
Destination IP  Source IP       Prot DstPrt SrcPrt Stat-Pkts  Stat-Bytes 
--------------- --------------- ---- ------ ------ ---------- --------------- 
MSFC 11.1.1.2 (Module 15): 
12.1.1.1        11.1.1.1        ICMP -      -      4          400 
12.1.1.1        11.1.1.1        UDP  69     50532  343        10997 
11.1.1.1        12.1.1.1        ICMP -      -      4          400 
11.1.1.1        12.1.1.1        UDP  50532  69     343        186592 
12.1.1.1        11.1.1.1        TCP  11000  Telnet 20         894

The TFTP transfer just ended. Roughly 8.1 MB are transferred from server to client in 14,903 packets, which creates an average size of 544 bytes per packet. In the other direction, the same amount of packets are received with an average size of 476,949 divided by 14,904, which makes 33 bytes.

Tamer (enable) show mls statistics entry 
                                  Last    Used 
Destination IP  Source IP       Prot DstPrt SrcPrt Stat-Pkts  Stat-Bytes 
--------------- --------------- ---- ------ ------ ---------- --------------- 
MSFC 11.1.1.2 (Module 15): 
12.1.1.1        11.1.1.1        ICMP -      -      4          400 
12.1.1.1        11.1.1.1        UDP  69     50532  14904      476949 
11.1.1.1        12.1.1.1        ICMP -      -      4          400 
11.1.1.1        12.1.1.1        UDP  50532  69     14903      8107224 
12.1.1.1        11.1.1.1        TCP  11000  Telnet 20         894

These tables should give you an idea of what your traffic pattern should look like.

Example 2: Two MSFCs in the Same Chassis Configured for HSRP

Below is the running configuration of the two MSFCs configured for HSRP and the output of the show standby command. The MSFC in slot 15 is active for VLAN 12, and the MSFC in slot 16 is active for VLAN 11.

Slot 15 Slot 16
tamer-msfc#wr t 
Building configuration... 

Current configuration: 
! 
version 12.1 
no service pad 
service timestamps debug uptime 
service timestamps log uptime 
no service password-encryption 
! 
hostname tamer-msfc 
! 
boot system flash bootflash:
  c6msfc-ds-mz.121-1.E2.bin 
! 
ip subnet-zero 
ip cef 
! 
! 
! 
! 
interface Vlan1 
 ip address 10.200.11.120 255.255.252.0 
! 
interface Vlan8 
 ip address 10.68.5.2 255.255.252.0 
! 
interface Vlan11 
 ip address 11.1.1.2 255.255.255.0 
 no ip redirects 
 standby 11 preempt 
 standby 11 ip 11.1.1.3 
! 
interface Vlan12 
 ip address 12.1.1.2 255.255.255.0 
 no ip redirects 
 standby 12 priority 105 preempt 
 standby 12 ip 12.1.1.3 
! 
router eigrp 1 
 network 10.0.0.0 
 network 11.0.0.0 
 network 12.0.0.0 
 no auto-summary 
! 
ip classless 
! 
line con 0 
 transport input none 
line vty 0 4 
 login 
! 
end
tamer-msfc-2#wr t 
Building configuration... 

Current configuration: 
! 
version 12.1 
no service pad 
service timestamps debug uptime 
service timestamps log uptime 
no service password-encryption 
! 
hostname tamer-msfc-2 
! 
boot system flash
  bootflash:c6msfc-jsv-mz.121-2.E.bin 
! 
ip subnet-zero 
!
! 
! 
! 
interface Vlan1 
 ip address 10.200.11.121 255.255.252.0 
! 
interface Vlan8 
 ip address 10.68.5.4 255.255.252.0 
! 
interface Vlan11 
 ip address 11.1.1.4 255.255.255.0 
 no ip redirects 
 standby 11 priority 105 preempt 
 standby 11 ip 11.1.1.3 
! 
interface Vlan12 
 ip address 12.1.1.4 255.255.255.0 
 no ip redirects 
 standby 12 preempt 
 standby 12 ip 12.1.1.3 
! 
router eigrp 1 
 network 10.0.0.0 
 network 11.0.0.0 
 network 12.0.0.0 
 no auto-summary 
! 
ip classless!  
! 
line con 0 
 transport input none 
line vty 0 4 
 login 
! 
end


tamer-msfc>show standby 
Vlan11 - Group 11 
  Local state is Standby, priority 100,
    may preempt 
  Hellotime 3 holdtime 10 
  Next hello sent in 00:00:00.814 
  Hot standby IP address is 11.1.1.3 configured 
  Active router is 11.1.1.4 expires in 00:00:09 
  Standby router is local 
  Standby virtual MAC address is 0000.0c07.ac0b 
  4 state changes, last state change 00:06:36 
Vlan12 - Group 12 
  Local state is Active, priority 105,
    may preempt 
  Hellotime 3 holdtime 10 
  Next hello sent in 00:00:02.380 
  Hot standby IP address is 12.1.1.3 configured 
  Active router is local 
  Standby router is 12.1.1.4 expires in 00:00:09 
  Standby virtual MAC address is 0000.0c07.ac0c 
  2 state changes, last state change 00:12:22
tamer-msfc-2#show standby 
Vlan11 - Group 11 
  Local state is Active, priority 105,
    may preempt 
  Hellotime 3 holdtime 10 
  Next hello sent in 00:00:02.846 
  Hot standby IP address is 11.1.1.3 configured 
  Active router is local 
  Standby router is 11.1.1.2 expires in 00:00:08 
  Standby virtual MAC address is 0000.0c07.ac0b 
  2 state changes, last state change 00:07:02 
Vlan12 - Group 12 
  Local state is Standby, priority 100,
    may preempt 
  Hellotime 3 holdtime 10 
  Next hello sent in 00:00:02.518 
  Hot standby IP address is 12.1.1.3 configured 
  Active router is 12.1.1.2 expires in 00:00:07,
   priority 105 
  Standby router is local 
  Standby virtual MAC address is 0000.0c07.ac0c 
  4 state changes, last state change 00:04:08

All the information in the previous example is still valid. To verify what has changed after configuring HSRP, view the output of the MLS commands below.

Issuing show mls Commands

tamer (enable) show mls 
Total packets switched = 29894 
Total Active MLS entries = 0 
  MSFC 11.1.1.2 (Module 15) entries = 0 
  MSFC 10.200.11.121 (Module 16) entries = 0 
IP Multilayer switching aging time = 256 seconds 
IP Multilayer switching fast aging time = 0 seconds, packet threshold = 0 
IP Current flow mask is Full flow 
Active IP MLS entries = 0 
Netflow Data Export version: 7 
Netflow Data Export disabled 
Netflow Data Export port/host is not configured. 
Total packets exported = 0 
IP MSFC ID      Module XTAG MAC               Vlans 
--------------- ------ ---- ----------------- ---------------- 
11.1.1.2        15     1    00-d0-d3-9c-9e-3c 12,11,8,1 
                            00-00-0c-07-ac-0c 12 

10.200.11.121   16     2    00-d0-bc-f0-07-b0 1,8,11,12 
                            00-00-0c-07-ac-0b 11 
  

IPX Multilayer switching aging time = 256 seconds 
IPX flow mask is Destination flow 
IPX max hop is 15 
Active IPX MLS entries = 0 

IPX MSFC ID     Module XTAG MAC               Vlans 
--------------- ------ ---- ----------------- ---------------- 
11.1.1.2        15     1    -                 - 

10.200.11.121   16     2    -                 -
  • There are now two MLS routers seen by the PFC.

  • For each router seen, the MAC address used by the HSRP group is 00-00-0c-07-ac-xx. These MAC addresses are the virtual MAC addresses used by HSRP. You are only seeing the MAC address of group 11 linked to the router which is active for that group (slot 15 for VLAN 12 and slot 16 for VLAN 11). That means that in addition to packets with destination MAC address being the MSFC MAC address, candidate packets are also considered, which are packets with destination MAC being the HSRP address.

  • As stated in the first example, you also need to see these HSRP address in the Layer 2 CAM table pointing to the MSFC.

tamer (enable) show cam 00-00-0c-07-ac-0c 
* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry. 
X = Port Security Entry 

VLAN  Dest MAC/Route Des    [COs]  Destination Ports or VCS / [Protocol Type] 
----  ------------------    -----  ------------------------------------------- 
12    00-00-0c-07-ac-0c R#          15/1 
Total Matching CAM Entries Displayed = 1 
tamer (enable) 

tamer (enable) show cam 00-00-0c-07-ac-0b 
* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry. 
X = Port Security Entry 

VLAN  Dest MAC/Route Des    [COs]  Destination Ports or VCS / [Protocol Type] 
----  ------------------    -----  ------------------------------------------- 
11    00-00-0c-07-ac-0b R#          16/1 
Total Matching CAM Entries Displayed = 1 
tamer (enable)

Issuing show mls entry Commands

tamer (enable) show mls entry
Destination-IP  Source-IP       Prot  DstPrt SrcPrt Destination-MAC   Vlan EDst 
--------------- --------------- ----- ------ ------ ----------------- ---- ---- 
MSFC 11.1.1.2 (Module 15): 
11.1.1.1        12.1.1.1        ICMP  -      -      00-00-0c-09-50-66 11   ARPA 
MSFC 10.200.11.121 (Module 16): 
12.1.1.1        11.1.1.1        ICMP  -      -      00-10-7b-3b-af-3b 12   ARPA 

ESrc DPort  Sport  Stat-Pkts  Stat-Bytes  Uptime   Age 
---- ------ ------ ---------- ----------- -------- -------- 
ARPA 7/3    7/4    4          400         00:00:03 00:00:03 
ARPA 7/4    7/3    4          400         00:00:04 00:00:03 


Destination-IPX           Destination-MAC   Vlan EDst ESrc  Port  Stat-Pkts  
------------------------- ----------------- ---- ---- ---- ----- ---------- 
Stat-Bytes  Uptime   Age 
----------- -------- -------- 

MSFC 11.1.1.2 (Module 15): 
MSFC 10.200.11.121 (Module 16):

  • There are now two shortcut tables: one for flows created by the first MSFC, and one for flows created by the second MSFC.

  • By pinging between 11.1.1.1 and 12.1.1.1, which are two PCs configured with the HSRP address as their default gateway, packets from 12.1.1.1 to 11.1.1.1 coming in VLAN 12 on the switch were shortcuted by the MSFC in slot 15 (as it is the active HSRP router for VLAN 12), and a shortcut for destination 11.1.1.1 is created by the MSFC in slot 15. The shortcuts for packets from 11.1.1.1 to 12.1.1.1 have been created on the other end by the MSFC in slot 16.

Troubleshooting Tips

Creation of Flows

If a flow has not been created, use the following troubleshooting tips:

  • Does the Supervisor Engine see the MSFC with all expected MAC addresses when issuing the show mls command?

    • If yes, go to the next step.

    • If no, make sure that the MSFC is not stuck in ROMmon mode. Verify that MLS is enabled on the MSFC by issuing the command show mls status.

  • Is the MAC address of the MSFC present when issuing the show cam command? Does it show up as a router cam entry (R#)?

  • Verify that you have not configured a feature on the MSFC that disables MLS.

    • Check the features that may impact MLS by reviewing the release notes for the software version you are running.

    • Examples of restrictions are shown in the table below.

      IP Router Command Restrictions

      Command Behavior
      clear ip route Clears all MLS cache entries for all switches performing Layer 3 switching for this MSFC.
      ip routing The no form purges all MLS cache entries and disables IP MLS on this MSFC.
      ip security (all forms of this command) Disables IP MLS on the interface.
      ip tcp compression-connections Disables IP MLS on the interface.
      ip tcp header-compression Disables IP MLS on the interface.

  • Verify that you have enabled an access list that requests software handling instead of being processed by hardware shortcuts. Refer to Hardware and Software Handling of IOS ACLs for more information.

If you have gone through all the tips listed above, and are still having problems, verify that the MSFC is still being hit by a lot of packets.

  • Something may be causing the entries to be purged continuously. Some possible causes of purging the flow table may include the following:

    • Route flapping or any Layer 3 instability.

    • ARP cache changes on the MSFC.

    • Flowmask changed on the Supervisor Engine.

    • Destination VLAN is deleted.

    • VLAN interface is shutdown on the MSFC.

  • Some other reasons for software forwarding (packet hitting the MSFC) instead of hardware shortcuts may include the following:

    • Packet with IP option set.

    • Packet with TTL less than or equal to 1.

    • Packet that needs to be fragmented.

  • You can have up to 128 K of flow, however, a hashing algorithm is used. If you exceed 32 K flows, you may begin to have hash collision that will cause packets being routed by software.

    • A way to avoid having too many flows is to configure a fast aging for MLS flow.

  • Remember that you can only have MLS for IP, IPX, and IP Multicast. If you have other types of traffic (for example, AppleTalk), they will be software routed, and can cause CPU peaks on the MSFC or excessive packet hitting on the MSFC.

  • As specified, IP MLS and IPX MLS are enabled by default, however, IP multicast MLS is not enabled by default. If you are using IP multicast, make sure to enable MLS for multicast as specified in the configuration guide.

Note: A Spanning Tree Topology Change Notification (TCN) or flapping ports on a Catalyst 6500/6000 series switch will not cause the MLS flow table to be cleared, as was the case for MLS on Catalyst 5000 switches.

IP Accounting excludes MLS traffic

In the Cisco Catalyst 6500 series, Multiple Layer Switching (MLS) is deployed in such a way that once a flow is established, traffic is directly switched at PFC (hardware-switched) and is not processed by the MSFC, hence the lack of continuous accounting. Only new or process-switched flows (software-switched) is recorded by IP accounting when enabled, and even then only until the entry is entered into the database. Thus, the previous warning message is normally displayed when you enable IP accounting on such a platform.

6500(config)#int fa8/40
6500(config-if)#ip accounting
Accounting will exclude mls traffic when mls is enabled.

NetFlow accounting is the preferred method. Refer to Configuring NetFlow for more information regarding to NetFlow.

Unable to configure interface-full Flowmask

C6500#mls flow ip interface-full
%  Unable to configure flow mask for ip protocol: interface-full. Reset
to the default flow mask type: none

Issue the show fm fie flowmask detail command and verify if NAT is enabled and uses Intf Full Flow mask.

C6500#show fm fie flowmask detail

!--- Part of the output not shown

Primary Flowmasks registered by Features
----------------------------+------------------------+------------------
---
          Feature                   Flowmask             Flowmask Status
----------------------------+------------------------+------------------
---
 IP_ACCESS_INGRESS               Intf Full Flow		Disabled/Unused
 IP_ACCESS_EGRESS                Intf Full Flow		Disabled/Unused
 NAT_INGRESS                     Intf Full Flow            Enabled
 NAT_EGRESS                      Intf Full Flow            Enabled

!--- Remaining part of the output not shown

If NAT uses the Intf Full Flow mask and when you try to configure the interface-full for Netflow, there is an issue, as there is a flowmask conflict. If you want to use Netflow stat, you can try to use interface-destination-source (mls flow ip interface-destination-source command), which does not conflict with the Netflow mask usage.

NDE assumes that all flows are created with the same flow mask. Due to this restriction, NDE cannot be enabled with certain features that require conflicting flow masks. One specific case is hardware-accelerated NAT. NDE and hardware-accelerated NAT are mutually exclusive.

NDE fails if any of these events occur:

  • Hardware-accelerated NAT is enabled.

  • Two or more features with conflicting flow masks have been configured on the switch.

Conversely, once NDE is successfully configured, NAT cannot be configured to work in the hardware and two different features with conflicting flow mask requirements cannot be configured on the switch.

Related Information

Updated: Aug 30, 2005
Document ID: 10566