This document illustrates how to achieve a high degree of optimal routing in an enterprise network connected to multiple Internet service providers (ISPs), while minimizing the memory requirements of the Border Gateway Protocol (BGP) routers. This is achieved by using AS_PATH filters for accepting only routes originated from an ISP and its directly connected autonomous systems, instead of receiving the full BGP routing table from an ISP.
There are no specific requirements for this document.
This document is not restricted to specific software and hardware versions.
For more information on document conventions, refer to the Cisco Technical Tips Conventions.
This section provides a network diagram as an example. In the example, you filter incoming BGP updates at Router 1 and Router 2 to accept the routes of the ISP and the routes of the directly connected autonomous system. Router 1 is accepting routes for ISP-A and its directly connected autonomous system C1. Similarly, Router 2 is accepting routes for ISP-B and C2. The rest of the networks, which do not belong to the ISPs and their customer autonomous system, follow the default route pointing toward ISP-A or ISP-B, depending on the enterprise routing policy.
You can observe how memory utilization varies when Router 1 accepts the complete BGP routing table of approximately 100,000 routes from its ISP, as compared to when you apply inbound AS_PATH filters on Router 1.
Note: The actual number of prefixes that make up a full feed may vary. The values in this document serve only as an example. Route-servers can provide a good idea of how many prefixes make up a full BGP table. (For more information on route servers, refer to Traceroute.org .)
This is the configuration of Router 1:
hostname R1 ! router bgp XX no synchronization neighbor 157.x.x.x remote-as 701 neighbor 157.x.x.x filter-list 80 out ! ip as-path access-list 80 permit ^$ ! end
The show ip bgp summary command output shows that 98,410 prefixes have been received from ISP-A (BGP neighbor 157.x.x.x):
R1# show ip bgp summary BGP router identifier 65.yy.yy.y, local AS number XX BGP table version is 611571, main routing table version 611571 98769 network entries and 146299 paths using 14847357 bytes of memory 23658 BGP path attribute entries using 1419480 bytes of memory 20439 BGP AS-PATH entries using 516828 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 5843 BGP filter-list cache entries using 70116 bytes of memory BGP activity 534001/1904280 prefixes, 2371419/2225120 paths, scan interval 15 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 165.yy.yy.a 4 6xx9 32962 826287 611571 0 0 01:56:13 1 165.yy.yy.b 4 6xx9 32961 855737 611571 0 0 01:56:12 1 165.yy.yy.c 4 6xx9 569699 865164 611571 1 0 01:55:39 47885 157.x.x.x 4 701 3139774 262532 611571 0 0 00:07:24 98410
The show ip route summary command output shows that 80,132 BGP routes are installed in the routing table:
R1# show ip route summary IP routing table name is Default-IP-Routing-Table(0) Route Source Networks Subnets Overhead Memory (bytes) connected 0 4 256 576 static 0 1 64 144 eigrp 6 0 5 768 720 bgp XX 80132 18622 6320256 14326656 External: 87616 Internal: 11138 Local: 0 internal 854 994056 Total 80986 18632 6321344 15322152
This command shows the amount of memory the BGP process occupies in RAM:
R1# show processes memory | begin BGP PID TTY Allocated Freed Holding Getbufs Retbufs Process 73 0 678981156 89816736 70811036 0 0 BGP Router 74 0 2968320 419750112 61388 1327064 832 BGP I/O 75 0 0 8270540 9824 0 0 BGP Scanner 70882248 Total BGP 77465892 Total all processes
The BGP process is occupying approximately 71 MB of memory.
In this example, you apply the inbound filter list to accept routes originated by ISP-A and its directly connected autonomous systems. In the example, ISP-A is advertising a default route (0.0.0.0) via external BGP (eBGP), so routes that do not pass the filter list follow the default route toward ISP-A. This is the configuration for setting up the filter list:
hostname R1 ! router bgp XX no synchronization . neighbor 157.x.x.x remote-as 701 neighbor 157.x.x.x filter-list 80 out neighbor 157.x.x.x filter-list 85 in !--- This line filters inbound BGP updates. ! ip as-path access-list 80 permit ^$ ip as-path access-list 85 permit ^701_[0-9]*$ !--- The AS_PATH filter list filters ISP and the !--- directly connected autonomous system routes. ! end
This show ip bgp summary command output shows 31,667 prefixes received from ISP-A (neighbor 157.xx.xx.x):
R1# show ip bgp summary BGP router identifier 165.yy.yy.y, local AS number XX BGP table version is 92465, main routing table version 92465 36575 network entries and 49095 paths using 5315195 bytes of memory 4015 BGP path attribute entries using 241860 bytes of memory 3259 BGP AS-PATH entries using 78360 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 4028 BGP filter-list cache entries using 48336 bytes of memory BGP activity 1735069/3741144 prefixes, 4596920/4547825 paths, scan interval 15 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 165.yy.yy.a 4 6319 226694 1787061 92465 0 0 17:31:04 1 165.yy.yy.b 4 6319 226814 1806986 92465 0 0 19:51:53 1 165.yy.yy.c 4 6319 1041069 1822703 92465 0 0 19:44:52 17424 157.xx.xx.x 4 701 14452518 456341 92465 0 0 19:51:37 31667
The show ip route summary command output shows 27,129 BGP routes in the routing table:
R1# show ip route summary IP routing table name is Default-IP-Routing-Table(0) Route Source Networks Subnets Overhead Memory (bytes) connected 0 4 256 576 static 0 1 64 144 eigrp 6319 0 6 896 864 bgp 6319 27129 9424 2339392 5299332 External: 19134 Internal: 17419 Local: 0 internal 518 602952 Total 27647 9435 2340608 5903868
The memory used by the BGP process is approximately 28 MB, as shown here:
R1# show processes memory | include BGP PID TTY Allocated Freed Holding Getbufs Retbufs Process 73 0 900742224 186644540 28115880 0 0 BGP Router 74 0 5315232 556232160 6824 2478452 832 BGP I/O 75 0 0 39041008 9824 0 0 BGP Scanner 28132528 Total BGP 34665820 Total all memory
To check the memory used by the BGP process, use the show processes memory | include bgp command. The most common issues related to an overuse of memory are listed here:
Memory allocation failure "%SYS-2-MALLOCFAIL". For more information about this error message, refer to the document Troubleshooting Memory Problems.
Refused Telnet sessions.
No output from some show commands.
"Low on memory" error messages.
"Unable to create EXEC - no memory or too many processes" console messages.
Router hanging, or no console response. For more information, refer to the document Troubleshooting High CPU Utilization on Cisco Routers.
If you run BGP-related debugs, it usually causes excessive memory consumption, which can also result in memory errors due to BGP. Debugs for BGP must be run with caution and are to be avoided if they are not required.
To store a complete global BGP routing table from one BGP peer, it is best to have a minimum of 512 MB or 1 GB of RAM in the router. If 256 MB of RAM is used, it is recommended that you use more route filters. If you use 512 MB of RAM, more Internet routes can be placed in the routing table with fewer route filters. On the Catalyst 6500/6000 receiving a full BGP table, it is recommended to have Multilayer Switch Feature Card 2 (MSFC2) with 256 MB of RAM to avoid Cisco bug ID CSCdt13244 (registered customers only) . The memory consumption by BGP routes depends on the number of attributes, such as multipath support, soft reconfiguration, the number of peers, and AS_PATH. For more details on the BGP memory requirement, refer to RFC 1774 .
Cisco Express Forwarding/distributed Cisco Express Forwarding (CEF/dCEF) switching consumes memory, depending on the routing table size. There are two main components of CEF:
The Forwarding Information Base (FIB)
The adjacency table
Both tables are stored in DRAM memory. Ensure that your Versatile Interface Processor (VIP) or line card also contains sufficient free DRAM. The "%FIB-3-FIBDISABLE : Fatal error, slot [#]: no memory" and "%FIB-3-NOMEM" error messages indicate insufficient memory in the cards.
It is highly recommended to check the VIP or line card memory before enabling dCEF. Complete these steps to confirm the memory:
Configure central CEF by issuing the ip cef command in global configuration mode.
Allow time for the FIB table to build.
Review the size of the central FIB table with the show ip cef summary command.
Determine whether the VIP or line card has sufficient available DRAM to store a similar-sized FIB table.
Issue the show controller vip [slot#] tech command, and check the output of the show memory summary command.
When running full Internet BGP routes, it is best to have at least 512 MB or 1 GB of RAM on the VIP or line card.
For more information about troubleshooting memory-related issues involving CEF/dCEF, refer to the document Troubleshooting Cisco Express Forwarding-Related Error Messages.
This chart illustrates the memory savings by implementing the filter list:
|Number of Prefixes||Memory Consumed|
|Autonomous System Filter||31,667||28,132,528|
When the BGP router receives its neighbors' full BGP routing table (98,410 routes), the router consumes approximately 71 MB. With the AS_PATH filters applied to inbound updates, the size of the BGP routing table is reduced to 31,667 routes, and the memory consumption is approximately 28 MB. This decrease in memory utilization is more than 60 percent with optimal routing.
If you review the AS Internet Graph compiled by the Cooperative Association for Internet Data Analysis (CAIDA), you can see which ISPs have the highest degree of interconnectivity (those closest to the center of the chart). With less interconnectivity, fewer routes pass through the AS_PATH filter, and the BGP memory consumption is lower. However, it is important to note that whenever AS_PATH filters are set, you need to configure a default route (0/0). Routes that do not pass the AS_PATH filter list follow the default route.
- Using Regular Expressions in BGP
- Load Sharing with BGP in Single and Multihomed Environments: Sample Configurations
- How to Use HSRP to Provide Redundancy in a Multihomed BGP Network
- Sample Configuration for BGP with Two Different Service Providers (Multihoming)
- BGP Support Page
- Technical Support - Cisco Systems
The Cisco Support Community is a forum for you to ask and answer questions, share suggestions, and collaborate with your peers.
Refer to Cisco Technical Tips Conventions for information on conventions used in this document.