Guest

Cisco Catalyst 4000 Series Switches

Configuration and Overview of the Router Module for the Catalyst 4500/4000 Family (WS-X4232-L3)

Document ID: 6198

Updated: Jan 07, 2007

   Print

Introduction

This document describes the WS-X4232-L3 router module for the Cisco Catalyst 4500/4000 Series Switches. In addition to a description of the architecture and configuration of the WS-X4232-L3, this document also provides a sample configuration that uses a Catalyst 4500/4000 Series Switch and the router module.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these software and hardware versions:

  • Cisco Catalyst OS (CatOS) release 5.5(1) or later

  • Cisco IOS® Software Release 12.0(7)W5(15d)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

The Cisco IOS Software image file name for the WS-X4232-L3 begins with "cat4232-". You can find the file in the the Catalyst 4232 section of the Download Software Area (registered customers only) for LAN switching software.

Note: There is support for the router module when you use it in conjunction with the Supervisor Engine 1 and Supervisor Engine 2. However, there is no support for the router module when you use it in conjunction with Supervisor Engine 2+, 3, 4, or 5.

Note: Refer to the Features section of Installation and Configuration Note for the Catalyst 4000 Layer 3 Services Module for more information on the software features that have support on the router module (WS-X4232-L3).

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Architecture Overview

The WS-X4232-L3 module has 32 Fast Ethernet ports and two Gigabit Ethernet ports.

These two Gigabit Ethernet ports correspond to interfaces gigabit 1 and gigabit 2 in the router configuration. These Gigabit Ethernet ports are routed ports.

Internally, the module has two Gigabit Ethernet interfaces (gigabit 3 and gigabit 4) that connect the router to the switch backplane. The switch backplane uses the first two ports in that slot to connect to the router module. When you insert the WS-X4232-L3 module in slot 3, Gigabit Ethernet interfaces 3 and 4 connect to the backplane ports 3/1 and 3/2. Ports 3/1 and 3/2 are Layer 2 ports with configuration on the switch Supervisor Engine. Gigabit Ethernet interfaces 3 and 4 are Layer 3 ports with configuration on the router module.

There are 32 Fast Ethernet ports on the router module. These ports are Layer 2 ports and do not perform any Layer 3 functions. Although the ports have a physical location on the router module, you must configure the ports on the switch Supervisor Engine.

This diagram provides a visual explanation of the architecture. For this setup, install the router module in slot 2 of the Catalyst switch.

28-a.gif

Configuration of the WS-X4232-L3

Supervisor Engine

The show port command displays the two gigabit ports and the 32 10/100 Mbps ports with the numbers 1 through 34.

Note: The two gigabit ports that you see from the Supervisor Engine are not the two ports that you see on the front panel. The ports that you see from the Supervisor Engine are the two switched ports that connect to the route engine. You need to configure the physical ports as switch ports. This configuration is similar to the configuration of the Multilayer Switch Module (MSM) on the Catalyst 6500/6000 Series Switches. The more common configuration for these ports is to set them as Gigabit EtherChannel (GEC) and trunking. This way, you can route between all VLANs on the router.

Note: You can access the router module from the Supervisor Engine if you issue the session module# command. This action is similar to access of the Route Switch Module (RSM) in a Catalyst 5500/5000 Series Switch.

Router

If you see a router prompt, look for four Gigabit Ethernet interfaces with numbers from 1 to 4 (gigabit 1, gigabit 2, gigabit 3, and gigabit 4) and a Fast Ethernet out-of-band interface.

This is the default configuration:

Router#show run 
Building configuration...  

Current configuration:  
!  
version 12.0  
service config  
no service pad  
service timestamps debug uptime  
service timestamps log uptime  
no service password-encryption  
!  
hostname Router  
!  
!  
ip subnet-zero  
!  
!  
!  
interface FastEthernet1  
 no ip address  
 no ip directed-broadcast  
 shutdown  
!  
interface gigabitEthernet1  
 no ip address  
 no ip directed-broadcast 

!--- Output suppressed.

Note: In this configuration, gigabit 3 and gigabit 4 are the connections that goes to the backplane. Gigabit 1 and gigabit 2 are the user ports on the front panel (routed ports). Most of the time, as on an MSM, you configure port 3 and port 4 to be part of the same interface port channel. Also, you configure subinterfaces on that channel (with Inter-Switch Link Protocol [ISL] or IEEE 802.1Q encapsulation). As on the MSM, the configuration of gigabit 3 and gigabit 4 on the router module needs to be consistent with the configuration of port slot/1 and slot/2 on the switch side. You can check the traffic between the router and the switch if you issue the show interface port-channel or show interface gigabitethernet commands.

Access List Support on the WS-X4232-L3

There is support for access control lists (ACLs) on the WS-X4232-L3 router module, but the sample configuration that this document discusses does not support ACLs. Refer to Configuring ACLs on the WS-X4232-L3 Router Module for the Catalyst 4000 Family for more information on ACL configurations with support for the WS-X4232-L3 module.

Sample Configurations

The sample configuration contains the elements in this list. (See the Network Diagram.)

  • Bang—A Catalyst 4500/4000 Series Switch with a router module in slot 3.

  • Liki—A router that attaches to Gigabit Ethernet 1 on the router module.

  • Donald—A router that attaches in VLAN 2 on port 3/3 of Bang. Port 3/3 is one of the Layer 2 ports of the router module.

  • Daniella—A router that attaches in VLAN 3 on port 2/3 of Bang.

This configuration includes a GEC connection between the router module and the Catalyst 4500/4000 Series Switch. You configure trunking on the GEC to allow multiple VLANs to pass to the router for interVLAN routing. This GEC configuration is the standard configuration. All the commands specific to this setup are moved into the port-channel subinterfaces.

When you use the Layer 3 module, remember that all traffic that reaches the router on the native VLAN is routed in software. This situation has an adverse effect on the performance of the switch. The microcode on the WS-X4232-L3 does not process 802.1Q packets that come in on the native VLAN without tags. Instead, the packets go to the CPU, and the CPU processes the packets. This process results in high CPU utilization if the CPU receives packets without tags at a high rate on the native VLAN subinterfaces. Therefore, create a dummy VLAN (which does not contain any user traffic) as the native VLAN. In this configuration example (the Network Diagram), VLAN 99 serves as the native VLAN. Configure only the native VLAN on the GEC between the router and the switch. Do not configure any other ports on the switch in this dummy VLAN.

Note: Create a dummy VLAN as the native VLAN on the trunk links between the router and the switch. The CPU routes in software all the traffic that sends on the native VLAN, which has an adverse effect on the performance of the switch. Create an additional VLAN that you do not use anywhere else in the network and make this VLAN the native VLAN for the trunk links between the router and the switch.

Network Diagram

28-b.gif

The Switch Supervisor Engine Configuration and Router Module Configuration sections of this document present configurations and output of some show commands. The configurations are on the Supervisor Engine of the Catalyst 4500/4000 Series Switch and the router module. This allows routing between the three subnets (VLAN 1, VLAN 2, and the router that attaches to Gigabit Ethernet 1).

Switch Supervisor Engine Configuration

The router switch card shows 34 ports in the show module command. These 34 ports include 32 switched ports to the front panel and 2 gigabit switched ports that directly connect to two of the router ports. Here is a sample:

bang> (enable) show module
Mod Slot Ports Module-Type                Model                Sub Status  
--- ---- ----- ------------------------- ------------------- ---  --------  
1   1    0     Switching  Supervisor      WS-X4012             no  ok  
2   2    34    10/100/1000  Ethernet      WS-X4232             no  ok  
3   3    34    Router Switch  Card        WS-X4232-L3          no  ok

Mod Module-Name         Serial-Num  
--- ------------------- --------------------  
1                        JAB02380AYG  
2                        JAB03210B6Y  
3                        JAB0417055S  

Mod MAC-Address(es)                         Hw     Fw          Sw  
--- -------------------------------------- ------ ---------- -----------------  
1   00-50-73-2a-f3-00 to 00-50-73-2a-f6-ff 1.0     4.5(1)     5.5(1)  
2   00-50-73-42-a9-68 to 00-50-73-42-a9-89 1.6  
3   00-01-42-06-73-a8 to 00-01-42-06-73-c9 1.0     12.0(7)W5( 12.0(7)W5(14.90

The only configuration added on the Catalyst 4000 side relates to the GEC trunk to the router module, as this sample shows:

bang> (enable) show config

# ***** NON-DEFAULT CONFIGURATION *****  
!  
! 
!  
!  
!  
set port channel all distribution mac both  
!  
#ip  
set interface sl0 down  
set interface me1 down  
!  
#set boot command  
set boot config-register 0x102  
set boot system flash bootflash:cat4000.5-5-1.bin  
!  
#port channel  
set port channel 3/1-2 156  
!  
#module 1 : 0-port Switching Supervisor  
!  
#module 2 : 34-port 10/100/1000 Ethernet  

set VLAN 3    2/3

!  
#module 3 : 34-port Router Switch Card  
set VLAN 2    3/3
set VLAN 99   3/1-2

!--- This interface has a configuration for 802.1Q routing. 
!--- The interface uses VLAN 99 as the native VLAN. The native VLAN on the 
!--- router switch must match the one that you have configured on the router.
!--- VLAN 99 is a dummy native VLAN. For more information, 
!--- see the note in the Sample Configurations section.


set trunk 3/1  nonegotiate dot1q 1-1005


!--- Note: Trunk mode needs to be in no-negotiate status
!--- because the router module does not support Dynamic Trunking Protocol (DTP).


set trunk 3/2  nonegotiate dot1q 1-1005
set port channel 3/1-2 mode on


!--- Note: You need to force the channel mode to on because
!--- the router module does not support Port Aggregation Protocol (PAgP).
  
end

On the switch, the show cdp neighbor command displays the router module as if the module were an external router that connects by a GEC trunk on gigabit ports 3/1 and 3/2. Here is a sample:

bang> (enable) show cdp neighbor
* - indicates vlan mismatch.  
# - indicates duplex mismatch.  
Port     Device-ID                        Port-ID                    Platform  
-------- ------------------------------- -------------------------  ------------  
 2/3     daniella                         Ethernet0                  cisco 2500  
                                         
                                         
                                          
 3/3     donald                           Ethernet0                  cisco 2500

bang> (enable) show trunk 
* - indicates vtp domain mismatch  
Port      Mode          Encapsulation  Status        Native  vlan  
--------  -----------  -------------  ------------   -----------  
 3/1      nonegotiate  dot1q           trunking      99  
 3/2      nonegotiate  dot1q           trunking      99

Port      Vlans allowed on trunk  
--------  ---------------------------------------------------------------------  
 3/1      1-1005  
 3/2      1-1005

Port      Vlans allowed and active in management  domain  
--------  ---------------------------------------------------------------------  
 3/1      1-3, 99
 3/2      1-3, 99

Port      Vlans in spanning tree forwarding  state and not pruned  
--------  ---------------------------------------------------------------------  
 3/1      1-3, 99
 3/2      1-3, 99

If you have the output of a show trunk command from your Cisco device, you can use the Output Interpreter Tool (registered customers only) to display potential issues and fixes.

bang> (enable) show port channel
Port  Status     Channel               Admin Ch  
                  Mode                  Group Id  
----- ---------- -------------------- ----- -----  
 3/1  connected  on                      156   833  
 3/2  connected  on                      156   833  
----- ---------- -------------------- ----- -----

Port  Device-ID                        Port-ID                    Platform  
----- ------------------------------- -------------------------  ----------------  
 3/1  bang-rp                          GigabitEthernet3           cisco Cat4232  
 3/2   Not directly connected to switch  
----- ------------------------------- -------------------------  ----------------

If you have the output of a show port channel command from your Cisco device, you can use the Output Interpreter Tool (registered customers only) to display potential issues and fixes.

Router Module Configuration

bang-rp#show verify
Cisco Internetwork Operating System Software  
IOS (tm) L3 Switch/Router Software (CAT4232-IN-M), Version 12.0(7)W5(14.90)   INTERIM 
TEST SOFTWARE  
Copyright (c) 1986-2000 by cisco Systems, Inc.  
Compiled Fri 26-May-00 15:26 by integ  
Image text-base: 0x60010928, data-base: 0x605C8000  

ROM: System Bootstrap, Version 12.0(7)W5(15b) RELEASE SOFTWARE  

bang-rp uptime is 1 day, 22 hours, 7 minutes  
System restarted by power-on  
System image file is "bootflash:cat4232-in-mz.120-7.W5.14.90"  

cisco Cat4232 (R5000) processor with 57344K/8192K bytes of memory.  
R5000 processor, Implementation 35, Revision 2.1  
Last reset from power-on  
1 FastEthernet/IEEE 802.3 interface(s)  
4 Gigabit Ethernet/IEEE 802.3z interface(s)  
123K bytes of non-volatile configuration memory. 

16384K bytes of Flash internal SIMM (Sector size 256K).  
Configuration register is 0x1

bang-rp#show run
Building configuration...
Current Configuration:
!  
version 12.0  
no service pad  
service timestamps debug uptime  
service timestamps log uptime  
no service password-encryption  
!  
hostname bang-rp  
!  
!  
ip subnet-zero  
!  
!  
!  
interface Port-channel1                    
 no ip redirects  
 no ip directed-broadcast  
 hold-queue 300 in  
!  
interface Port-channel1.2 


!--- The configuration of this interface is for 802.1Q routing.
!--- The interface uses a VLAN 2 tag.


encapsulation dot1Q 2
ip address 2.2.2.2 255.255.255.0
no ip redirects
no ip directed-broadcast    
!  
interface Port-channel1.3

!--- The configuration of this interface is for 802.1Q routing.
!--- The interface uses a VLAN 3 tag.

encapsulation dot1Q 3  
ip address 1.1.1.2 255.255.255.0  
no ip redirects  
no ip directed-broadcast  
!
interface Port-channel1.99

!--- The configuration of this interface is for 802.1Q routing. 
!--- The interface uses VLAN 99 as the native VLAN. The native VLAN on the router 
!--- must match the one that you have configured on the switch. VLAN 99 is a dummy
!--- native VLAN. For more information, see the note
!--- in the Sample Configurations section.


encapsulation dot1Q 99 native
no ip address
no ip redirects
no ip directed-broadcast
!
interface FastEthernet1                  

!--- You can use this out-of-band interface for management.
  
 no ip address  
 no ip directed-broadcast  
 shutdown  
!  
interface GigabitEthernet1  
 ip address 3.3.3.2 255.255.255.0  
 no ip directed-broadcast  
!  
interface GigabitEthernet2  
 no ip address  
 no ip directed-broadcast  
 shutdown  
!  
interface GigabitEthernet3  
 no ip address  
 no ip directed-broadcast  
 no negotiation auto  
 channel-group 1                         

!--- Both  Gigabit Ethernet 3 and Gigabit Ethernet 4 
!--- are part of channel group 1.
  
!  
interface GigabitEthernet4  
 no ip address  
 no ip directed-broadcast  
 no negotiation auto  
 channel-group 1                          

!--- Both Gigabit Ethernet 3 and Gigabit Ethernet 4  
!--- are part of channel group 1.
  
!  
router eigrp 1  
 passive-interface FastEthernet1  
 network 1.0.0.0  
 network 2.0.0.0  
 network 3.0.0.0  
!  
ip classless  
!  
arp 127.0.0.2 0050.732a.f300 ARPA  
!  
line con 0  
 transport input none  
line aux 0  
line vty 0 4  
 login  
!  
end  
bang-rp#show cdp neighbor  
Capability Codes: R - Router, T - Trans Bridge, B - Source Route  Bridge  
                   S - Switch, H - Host, I - IGMP, r - Repeater

Device ID        Local Intrfce      Holdtme    Capability Platform  Port ID  
liki              Gig 1               
160         T S      
WS-C3508G-Gig 0/1                           

!--- Liki connects to gigabit 1 on the router. 
!--- You can only see Liki from the router; you cannot 
!--- see Liki from the Supervisor Engine.
  
JAB02380AYG(bang)Port-channel1      148          T S       WS-C4003  3/2  
JAB02380AYG(bang)Port-channel1      147          T S       WS-C4003  3/1

Troubleshoot

Session from Supervisor to 4232-L3 Module Does Not Work After It Runs for Awhile

After the switch runs for awhile, a session from the Supervisor to the 4232-L3 module fails with this error message:

4006> (enable) session 2
Trying IntlgLineCard-2...
session: Unable to tunnel to IntlgLineCard-2 (57)

The most probable cause for this is due to an incorrect adjacency formed in the Supervisor module Address Resolution Protocol (ARP) table for the 4232-L3 module inband MAC address.

This issue can be resolved with an upgrade of the system software to a CatOS version not affected by Cisco bug ID CSCdx30617 (registered customers only) .

If an upgrade of the system software is not possible, you can try these workarounds:

  • Instead of sessioning to the module, telnet to any of the IP addresses configured on it.

  • A reset of the 4232-L3 module can recover the problem temporarily.

  • A move of the sc0 interface into a different VLAN can also resolve this issue.

Periodic TFTP Requests from 4232-L3

The 4232-L3 module continually tries to load a configuration from the network and displays this error message:

%Error opening tftp://255.255.255.255/network-config (Timed out)

The L3 module can be configured to automatically download the configuration file from a TFTP server when you issue the service config command. Store the configuration files on a TFTP server and download them while booting. This is useful when the size of the configuration file is larger than the size of the NVRAM on the device.

When the L3 module is configured with the service config command, it generates TFTP requests to download its configuration from a TFTP server.

In a scenario where an IPS/IDS is used, you might observe that the router continuously sends tftp broadcast. This is confirmed by the IP address of the source and the destination is 255.255.255.255, traffic is UDP 69 (TFTP).

In order to stop the log messages from being generated, issue these commands:

Router#config terminal
Router(config)#no service config
Router(config)#exit
Router#copy running-config startup-config

Conclusion and Tips

Remember these key points when you configure the routing module on the Catalyst 4500/4000:

  • The gigabit interfaces that you see on the front panel are not the same as the gigabit interfaces that you see when you issue the show port command from the Supervisor Engine. The interfaces on the front panel are the interfaces with the names gigabit 1 and gigabit 2 on the router.

  • Make sure that the native VLAN of the trunks between the switch and the router is a dummy VLAN. The CPU routes in software all traffic that is on the native VLAN. Therefore, create one additional VLAN that you do not use elsewhere and make that VLAN the native VLAN on the links between the switch and router.

Related Information

Updated: Jan 07, 2007
Document ID: 6198