Cisco 12000 Series Internet Router Architecture: Memory Details

Introduction

This document provides an overview of the Cisco 12000 Series Internet Router memory details.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on the following hardware:

• Cisco 12000 Series Internet Router

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

For more information on document conventions, see the Cisco Technical Tips Conventions.

Memory Present on the Gigabit Route Processor (GRP)

The following types of memory exist on the GRP:

Dynamic Random Access Memory (DRAM)

Dynamic RAM is also referred to as main or processor memory. Both the GRP and the line cards (LCs) contain DRAM that enables an onboard processor to run Cisco IOS® software and store network routing tables. On the GRP, you can configure route memory ranging from the factory default of 128 MB up to the maximum configuration of 512 MB.

GRP Route Memory (located in DRAM)

The processor on the GRP uses onboard DRAM to perform a variety of important tasks including the following:

• Running the Cisco IOS software image

• Storing and maintaining the network routing tables

• Loading the Cisco IOS software image into installed line cards

• Formatting and distributing updated routing tables to installed line cards

• Monitoring temperature and voltage alarm conditions of installed cards and shutting them down when necessary

• Supporting a console port that enables you to configure the router using an attached terminal

• Participating in network routing protocols (together with other routers in the networking environment) to update the router's internal routing tables

Figure 1 shows the locations of the processor memory DRAM dual in-line memory module (DIMM) sockets and the Flash memory single in-line memory module (SIMM) socket on the GRP.

Figure 1: Location of Processor Memory DRAM and Flash Sockets in the GRP

The two route memory DIMM sockets on the GRP, labeled U39 (route memory bank 1) and U42 (route memory bank 2), respectively, enable you to configure route memory in desired increments ranging from 128 MB to 256 MB. The table below lists the available route memory configurations and associated product numbers for the Cisco 12000 series GRP. The default route memory configuration is 128 MB. If the GRP is currently equipped with one 64-MB DIMM in socket U39, you can upgrade memory by installing a second 64-MB DIMM in socket U42, or by removing the existing 64-MB DIMM and replacing it with one 128-MB DIMM.

Total Route Memory Ordered1	Cisco Product Number	DIMM Modules	DRAM DIMM Sockets
64 MB	MEM-GRP/LC-64=2	1 64-MB DIMM	U39 or U42
128 MB	MEM-GRP/LC-128=	1 128-MB DIMM	U39
256 MB	MEM-GRP/LC-256=3	2 128-MB DIMMs	U39 and U42
256 MB	MEM-GRP-256=4	1 256-MB DIMM	U39
512 MB	MEM-GRP-512=5	2 256-MB DIMMs	U39 and U42

¹Do not mix memory sizes. If installing two DIMMs, both DIMMs must be the same size.

²For GRPs equipped with the previous default of 64 MB, this option adds a second 64 MB DIMM for a total of 128 MB.

³This product is no longer available. Replace it with Cisco Product Number MEM-GRP-256=.

⁴MEM-GRP-256= is only compatible with Product Number GRP-B=. In addition, Cisco IOS Software Releases 12.0(19)S, 12.0(19)ST, or later is required. ROMMON Release 11.2(181) or later is also required.

⁵512 MB route memory configurations on the GRP are only compatible with Product Number GRP-B=. In addition, Cisco IOS Software Releases 12.0(19)S, 12.0(19)ST, or later is required. ROMMON Release 11.2(181) or later is also required.

The show diag command shows "FRU: Linecard/Module: GRP-B=" for all GRP cards, regardless whether the card is of type GRP= or GRP-B=. Since the electrically erasable programmable ROM (EEPROM) may not be correctly programmed for these cards, a workaround has been created to make it possible to distinguish between the cards. This has been fixed since Cisco IOS software version 12.0(22)S with CSCdx62997 - GRP FRU Change. If you are running a Cisco IOS software release later than 12.0(22)S, you can rely on the output of the show diag command.

However, if you are running a Cisco IOS software version earlier than 12.0(22)S, the quickest way to check the GRP is to look at the second line of the show diag output where the slot number of the GRP is located:

• MAIN: type 19, 800-2427-01 is a GRP.

• MAIN: type 19, 800-2427-03 is a GRP-B with the option to go up to 512 MB DRAM with the new rommon version 181.

Below is an example of output from the show diag command for a normal GRP that shows up as a GRP-B under any release earlier than 12.0(22)S. In this case, you should rely on the 800 number:

Router#show diag 0 
SLOT 0 (RP/LC 0 ): Route Processor 
  MAIN: type 19, 800-2427-01 rev J0 dev 16777215 
  HW config: 0xFF SW key: FF-FF-FF 
PCA: 73-2170-03 rev G0 ver 3 
  HW version 1.4 S/N CAB03515XTY 
MBUS: MBUS Agent (1) 73-2146-07 rev B0 dev 0 
  HW version 1.2 S/N CAB03505RM6 
  Test hist: 0xFF RMA#: FF-FF-FF RMA hist: 0xFF 
DIAG: Test count: 0xFFFFFFFF Test results: 0xFFFFFFFF 
FRU: Linecard/Module: GRP-B= 
!--- This is where the confusion lies; it is actually a GRP.
 
                             it is actually a GRP.
  Route Memory: MEM-GRP/LC-256=    
MBUS Agent Software version 01.46 (RAM) (ROM version is 02.02) 
Using CAN Bus A 
ROM Monitor version 180 
Primary clock is CSC 1 
Board is analyzed 
Board State is IOS Running (ACTV RP ) 
Insertion time: 00:00:03 (16w6d ago) 
DRAM size: 268435456 bytes  

Upgrading DRAM to 512 MB on the GRP

Once you have identified the type of GRP you have with its current ROMMON version, these are the different possibilities:

• GRP - This one does not support the 512 MB option. You need to replace this card by a GRP-B.

• GRP-B with ROMMON version 180 - First you need to upgrade the Cisco IOS software release to 12.0(19)S or later, and then the ROMMON version can be upgraded manually using the upgrade rom slot X command where X is the slot number where the GRP is located.

  Once these steps have been performed, you can physically upgrade the memory as described in Replacing and Upgrading Route Processor Route Memory.

• GRP-B with ROMMON version 181 or later - You need to check that you are running a Cisco IOS software release equal to or later than 12.0(19)S. Then you can physically upgrade the memory as described in Replacing and Upgrading Route Processor Route Memory.

Sizing the DRAM Memory on the GRP

There should be at least 128 MB of DRAM on the GRP. If the GRP has to handle the full Border Gateway Protocol (BGP) Internet table, 256 MB is recommended. 128 MB might be sufficient. The amount of memory needed depends on a lot of factors, such as the number of BGP peers and so on. To be on the safe side, 256 MB is recommended in today's topology. Considering the growth rate of the Internet route table, this may or may not be enough in the future.

Shared Random Access Memory (SRAM)

SRAM provides secondary CPU cache memory. The standard GRP configuration is 512 KB. Its principal function is to act as a staging area for routing table update information to and from the line cards. SRAM is not user-configurable or field-upgradable.

For more information on sizing the DRAM Memory on the GRP, see Route Processor and Line Card Memory Recommendations for the Cisco 12000 Series Internet Router.

GRP Flash Memory

Both the onboard and PCMCIA card-based Flash memory allow you to remotely load and store multiple Cisco IOS software and microcode images. You can download a new image over the network or from a local server. You can then add the new image to Flash memory or replace the existing files. You can boot the routers either manually or automatically from any of the stored images. Flash memory also functions as a Trivial File Transfer Protocol (TFTP) server to allow other servers to boot remotely from stored images or to copy them into their own Flash memory.

Onboard Flash SIMM

The onboard Flash memory (called bootflash) is located in socket U17 and contains the Cisco IOS software boot image and other user-defined files on the GRP. This is an 8 MB SIMM, which is not user configurable or field upgradable It is always recommended to synchronize the boot image with the main Cisco IOS software image.

Flash Memory Card

The Flash memory card contains the Cisco IOS software image. A Flash memory card is available as Product Number MEM-GRP-FL20=, which is a 20 MB PCMCIA Flash memory card that ships as a spare, or as a part of a Cisco 12000 series system. This card can be inserted into either of the two PCMCIA slots in the GRP, so that the Cisco IOS software can be loaded into the GRP main memory. Both type 1 and type 2 PCMCIA cards can be used.

See the PCMCIA Filesystem Compatibility Matrix for compatibility between the PCMCIA Flash cards and various platforms.

Nonvolatile RAM (NVRAM)

The information stored in the NVRAM is non-volatile, which means the information is still present in this memory after a system reload. System configuration files, software configuration register settings, and environmental monitoring logs are contained in the 512 KB NVRAM, which is backed up with built-in lithium batteries that retain the contents for a minimum of five years. NVRAM is not user configurable or field-upgradable

Erasable Programmable Read Only Memory (EPROM)

The EPROM on the GRP contains a ROM monitor that enables you to boot the default Cisco IOS software image from a Flash memory card if the Flash memory SIMM does not contain a boot helper image. If no valid image is found, the boot process ends up in ROMMON mode, which is a subset of the main Cisco IOS software, to allow basic commands. The 512 KB Flash EPROM is neither user configurable nor field upgradable

Memory Present on the Line Cards

On a line card, there are two types of user-configurable line card memory:

• Route or processor memory (located in DRAM)

• Packet memory (located in SDRAM)

Line card memory configurations and memory socket locations differ, depending on the engine type of the line card. In general, all line cards share a common set of memory configuration options for processor or route memory, but support different default and maximum configurations for packet memory based on the type of engine on which the line card is built.

If you want to find out which Layer 3 Engine type is used on one line card, refer to these tables. If you are running a Cisco IOS software later than 12.0(9)S, you can execute this command:

  Router#show diag | i (SLOT | Engine)   
      ... 
      SLOT 1  (RP/LC 1 ): 1 port ATM Over SONET OC12c/STM-4c Multi Mode 
        L3 Engine: 0 - OC12 (622 Mbps) 
      SLOT 3  (RP/LC 3 ): 3 Port Gigabit Ethernet 
        L3 Engine: 2 - Backbone OC48 (2.5 Gbps) 
      ... 

On line cards, the main memory can be configured ranging from the factory default of 128 MB (Engine 0, 1, 2) up to the maximum configuration of 256 MB which is the default for the Engine 3 and 4 LCs.

Note: If there is not enough DRAM to load Cisco Express Forwarding tables onto one line card, Cisco Express Forwarding is automatically disabled for this line card. Since this is the only switching method available on 12000 Series Internet Routers, the line card itself is disabled.

Synchronous Dynamic RAM (SDRAM) - Packet Memory

Line card packet memory temporarily stores data packets awaiting switching decisions by the line card processor. Once the line card processor makes the switching decisions, the packets are propagated into the router's switch fabric for transmission to the appropriate line card. For a line card to operate, both receive packet memory dual in-line memory module (DIMM) sockets and transmit packet memory DIMM sockets must be populated. The SDRAM DIMMs installed in a given buffer (either receive or transmit) must be the same type and size, although receive and transmit buffers can operate with different memory sizes.

Type of Engine	Default Packet Memory	Upgradable	Upgradable to...
Engine 0	MEM-LC-PKT-128=	No
Engine 1	MEM-LC1-PKT-256=	No
Engine 2	MEM-LC1-PKT-256=	Yes	MEM-PKT-512-UPG=
Engine 3	512 MB - No FRU yet	No
Engine 4	MEM-LC4-PKT-512=	No

Engine 0 and 1 line cards (see Figure 2) include four SDRAM DIMM sockets for packet buffer memory. These sockets are paired as follows:

• Receive (Rx) buffer - Two SDRAM DIMM sockets labeled RX DIMM0 and RX DIMM1

• Transmit (TX) buffer-Two SDRAM DIMM sockets labeled TX DIMM0 and TX DIMM1

Engine 2 line cards (see Figure 3) include four SDRAM DIMM sockets for buffer memory. These sockets are paired as follows:

• Transmit (TX) buffer - Two SDRAM DIMM sockets labeled TX DIMM0 and TX DIMM1

• Receive (Rx) buffer - Two SDRAM DIMM sockets labeled RX DIMM0 and RX DIMM1

The output of the show diag command displays the amount of receive and transmit packet memory:

Router#show diag   
SLOT 1  (RP/LC 1 ): 1 Port SONET based SRP OC-12c/STM-4 Single Mode 
.... 
FrFab SDRAM size: 134217728 bytes, SDRAM pagesize: 8192 bytes   !-- Transmit packet memory 
ToFab SDRAM size: 134217728 bytes, SDRAM pagesize: 8192 bytes   !-- Receive packet memory 
.... 

You can find more information about packet memory on How To Read the Output of the show controller frfab | tofab queue Commands on a Cisco 12000 Series Internet Router.

Engine 2 line cards are also equipped with one SDRAM DIMM socket for pointer lookup (PLU) and table lookup (TLU) memory (see Figure 3) and one SDRAM DIMM socket for TLU memory. PLU and TLU memory are currently not user-configurable.

Engine 0 and Engine 1 line cards are equipped with six DIMM sockets:

Figure 2: Memory Locations on an Engine 0 and Engine 1 Line Card

• Two route memory DIMM sockets

• Two pairs of packet buffer DIMM sockets (Rx and Tx pairs)

Engine 2 line cards are equipped with eight DIMM sockets:

Figure 3: Memory Locations on an Engine 2 Line Card

• Two route memory DIMM sockets

• Two pairs of packet buffer DIMM sockets (Rx and TX pairs)

• One pointer lookup (PLU) memory DIMM socket (not user-configurable)

• One table lookup (TLU) memory DIMM socket (not user-configurable)

Dynamic RAM (DRAM) - Route Memory

The table below lists the available route memory configurations and associated product numbers of DRAM DIMMs for upgrading route memory on the Cisco 12000 Series line cards.

Route Memory Configurations for Cisco 12000 Series Line Cards
Total Route Memory Ordered	Cisco Product Number	DIMM Module	Route Memory DIMM Sockets
64 MB	MEM-GRP/LC-64=1	1 64-MB DIMM	DIMM0 or DIMM1
128 MB	MEM-DFT-GRP/LC-1282	1 128-MB DIMM	DIMM0 or DIMM1
128 MB	MEM-GRP/LC-128=3	1 128-MB DIMM	DIMM0 or DIMM1
256 MB	MEM-GRP/LC-256=	2 128-MB DIMMs	DIMM0 and DIMM1

¹This option adds a second 64 MB DIMM for a total of 128 MB for line cards that were previously equipped with 64 MB.

²The standard (default) DRAM DIMM configuration for the processor on an Engine 0, 1, or 2 LC is 128 MB and on an Engine 3 or 4 LC, 256MB.

³This option allows you to order a spare module or add a second 128 MB DIMM for a total of 256 MB for LCs that are already equipped with one 128 MB DIMM.

For guidelines regarding memory replacement, see Cisco 12000 Series Gigabit Switch Router Memory Replacement Instructions.

For guidelines regarding memory recommendation, see Route Processor and Line Card Memory Recommendations for the Cisco 12000 Series Internet Router.

Related Information

• Cisco 12000 Series Internet Router Architecture - Chassis
• Cisco 12000 Series Internet Router Architecture - Switch Fabric
• Cisco 12000 Series Internet Router Architecture - Route Processor
• Cisco 12000 Series Internet Router Architecture - Line Card Design
• Cisco 12000 Series Internet Router Architecture - Maintenance Bus, Power Supplies and Blowers, and Alarm Cards
• Cisco 12000 Series Internet Router Architecture - Software Overview
• Cisco 12000 Series Internet Router Architecture - Packet Switching
• Understanding Cisco Express Forwarding
• Technical Support - Cisco Systems