Table Of Contents
Characteristics of Access and Backbone Routers
Cisco 10000 Series Router: Specialized for Aggregation
Cisco 10000 Series Router Hardware View
Technology Overview
Introduction
The increasing need for Internet connectivity is placing huge demands on Internet service providers—not just in the number of connections that users require for Internet access, but also in the services that these users require on each connection. The Internet's explosive growth is driving the requirements for higher quality, faster connectivity, and more software features for an ever-growing number of customers.
One of the largest and fastest growing components of Internet access growth is business-to-business leased line connectivity. Leased line traffic is defined as DSO, T1/E1, and T3/E3 up to leased line OC-3. Independent analysts forecast this leased line market as a $30+ billion dollar business today, growing to $45 billion in 2004, as Figure 1-1 illustrates.
Figure 1-1 Leased Line Traffic
Aggregation of leased line traffic takes place at the "edge" of the network. The edge, the point at which the customer's enterprise network intersects with the ISP network, is rapidly becoming an area of strategic significance. At the edge, network subscribers attach to the ISP's network and service providers can apply services and aggregate leased line traffic. All of this has to be done without compromising performance.
To meet these unique requirements, ISPs need products that are optimized to provide the highest levels of density and availability, along with high-touch software features. The Cisco edge services router is designed to fit this unique position in an ISP's network. The key features are:
1.
Scalability and high bandwidth to meet increased customer demand for data, voice, and video transmission.
2.
3.
4.
5.
The ISPs' demanding requirements for an edge services router are not easily met. Aggregating more and more leased lines places a tremendous strain on the system's processor performance. Similarly, enabling each new software service (such as QoS, MLPPP, or MPLS) places a new burden on the processor. To be successful, the edge services router must do the `impossible'—maintain high performance even as new network services are enabled and thousands of new leased lines are connected.
This chapter describes the following topics:
•
Structure of an ISP Network
Physically, the Internet is composed of routers interconnected by communications links. Simple networks are built from a few general-purpose routers interconnected by links owned or leased by ISPs.
As networks become more complex, with greater numbers of elements, more structure is required. Elements become specialized in their applications, management and security become more important, physical location is a consideration, and the capability to handle higher densities of customers is critical.
Structure can be imposed on a complex network by assigning specific jobs to particular routers. A common approach in ISP networks is to divide assignments among routers in the following way:
•
•
Figure 1-2 illustrates the router specialization scheme in a typical ISP network.
Figure 1-2 Typical ISP Network
Most ISPs also impose physical structure on their networks by organizing them into points of presence (POPs). A POP is a physical location where a set of access and backbone routers is located. An ISP network usually consists of several POPs.
shows how the ISP network in Figure 1-2 might be physically structured at various POPs.
Figure 1-3 ISP POPs
Characteristics of Access and Backbone Routers
Although real networks are always more complicated than theoretical architectures, it is possible to make general distinctions between the two router types. Table 1-1 summarizes the principal differences between backbone and access routers.
The differences listed in Table 1-1 are not absolute, and often a particular router can fulfill either role. However, as Internet traffic continues to grow, the demands for access routers to handle increased density and backbone routers to handle greater throughput become more important. These density and performance requirements can be met more efficiently with a platform designed for this specific purpose—an aggregation router.
Aggregation Router Definition
An aggregation router is an access router that aggregates large numbers of leased lines from ISP customers into a few trunk lines for entry onto the Internet backbone. An effective aggregation router must be designed specifically for very high throughput performance on a large number of relatively low-speed leased line interfaces with high-touch, value-added network services enabled on each connection.
POP Design
An ISP can simplify network design and maintenance by using a cookie cutter design for its POPs. In this approach, all POPs have a similar structure, with variations for the size or specific needs of each site. A typical POP design is shown in Figure 1-4. There are as many POP designs as there are ISPs.
Figure 1-4 Typical POP Design
The design shown in Figure 1-4 includes the following desirable characteristics:
•
•
•
•
An edge services router should be flexible enough to serve as an access router in a variety of POP architectures.
Driving for Density
A significant factor affecting the design of an access router is the drive for increased density—an increase in the number of subscriber ports that can be terminated on a single router. Increased density has several advantages for the ISP:
•
•
•
The appropriate degree of statistical multiplexing is not a "one size fits all" proposition. ISPs have different views on the trade-off between a network that is rich in bandwidth resources and does not rely on multiplexing, versus one that uses a lower-cost infrastructure but risks occasional congestion. For this reason, an edge services router should be configurable with different combinations of interface cards to create a wide range of multiplexing ratios.
Leased Lines—the Last Mile
The access router serves as the ISP's front line, connecting directly to routers on its customers' premises. However, there is usually a complex circuit-switched infrastructure that transports the leased line signal the "last mile" between the customer premises and the ISP POP.
One common method for constructing the last-mile network for new installations is based on metropolitan-area fiber-optic ring technology. Figure 1-5 shows a simple network that transports 1.544-Mbps DS1 and 44.736-Mbps DS3 signals from a subscriber site, across a fiber-optic ring, to an ISP POP.
Figure 1-5 SONET Metropolitan-Area Transport Network
Most new fiber-optic networks are based on Synchronous Optical Network (SONET) standards in North America or on Synchronous Digital Hierarchy (SDH) standards in much of the rest of the world. SONET/SDH technology is important in transport networks providing leased line connectivity to subscriber routers for two reasons:
•
•
POPs designed before the introduction of the standard channelized SONET/SDH interface required many racks of data service units (DSUs) to terminate T1 service over traditional copper wiring. The edge services router in today's POP can provide a high density of terminations for DS1 and DS3 connections. A single line card can terminate hundreds of DS1 circuits, all carried on a single fiber.
Cisco 10000 Series Router: Specialized for Aggregation
The increasingly stringent requirements for edge services routing—more services, more leased line connections, greater throughput—are addressed by Cisco Systems with the Cisco 10000 series router. The Cisco 10000 series router is specifically designed for use in aggregation applications by offering the following:
•
•
•
•
•
•
The remainder of this section describes the Cisco 10000 series router and features that provide industry-leading capabilities for leased-line aggregation.
Cisco 10000 Series Router Hardware View
The Cisco 10000 series router is a chassis-based product that meets all requirements for deployment in central office environments. The product has several major units:
•
•
•
–
–
Figure 1-6 shows the layout of components and key specifications of the Cisco 10008 chassis.
Figure 1-6 Cisco 10008 Chassis Layout
Figure 1-7 shows the layout and key specifications of the Cisco 10005 chassis.
Figure 1-7 Cisco 10005 Chassis Layout
Central Office Features
Several features make the Cisco 10000 series router particularly well-suited to central office installations:
•
•
•
–
–
Additional Cisco 10008 ESR Central Office Features
The Cisco 10008 chassis is well-suited for central office environments:
•
•
Interface Types
Deployment at the edge of the network requires several specialized interfaces. Theoretically, any of these interfaces could be used for connections on either the subscriber side or the Internet backbone side of the router. However, in typical installations, different interface types are used for these two applications.
For subscriber-side connections, the Cisco 10000 series router supports:
•
•
•
•
The channelized interface cards support full-rate (unchannelized) DS3 connections, as well as channelization to DS1 (1.544 Mbps), E1 (2.048 Mbps), and nxDS0 (nx64 kbps). All four interface cards also support "subrate" DS3, in which the rate of data transfer across a DS3 can be reduced to limit peak access rate. Subrate modes allow the system to interoperate with other equipment.
For subscriber-side optical connections, the Cisco 10000 series router supports:
•
•
Both of these optical interfaces provide efficient, high-performance bandwidth with STS-3/STM-1 connections for throughput of up to 155.52 Mbps over SONET/ITU-T Synchronous Digital Hierarchy (SDH) interfaces.
For connections to the backbone network, the Cisco 10000 series router supports:
•
•
•
Density Summary
In a typical application, the Cisco 10008 ESR is configured with two cards connecting to the network backbone and six cards connecting to subscriber sites. Table 1-2 compares T1 density possible for connecting to subscriber sites using channelized T3, channelized OC-12, or channelized STM-1 line cards.
In the same application, the Cisco 10005 ESR is configured with one card connecting to the network backbone and four cards connecting to subscriber sites. Table 1-3 compares T1 density possible for connecting to subscriber sites using channelized T3, channelized OC-12, or channelized STM-1 line cards.
High-Touch Feature Set
In its role as a gateway to an ISP network, the Cisco 10000 series router is uniquely positioned to protect the network and also to offer new, value-added features to ISP customers.
Underlying the Cisco 10000 series router feature set is the Cisco IOS software. This system software supports standard routing protocols and network configuration and monitoring. It also supports a variety of network interfaces.
Some areas that have received special attention in the development of the Cisco 10000 series ESR IOS software include:
•
•
•
The following sections summarize some of the situations in which advanced router features are critical to ISP networks in today's evolving Internet market.
Virtual Private Networks
Internet access is an important networking application for many ISPs, but it is not the only one. Many ISP customers also want the ability to build private networks, with paths among geographically diverse sites and limited access to the Internet. Private networks allow enhanced security by limiting access to the Internet from a few protected locations in an enterprise network.
In the past, private networks were built by enterprise information technology departments either by leasing dedicated circuits between sites or by using a virtual circuit technology such as Frame Relay or ATM. In each case, the enterprise operating the private network incurred substantial expense building and maintaining the network. ISPs often built several wide-area networks—one for Internet traffic, a second for Frame Relay, and a third for ATM.
An ISP can reduce network costs by moving all traffic onto one common network and then creating virtual private networks (VPNs) built on top of a common Internet backbone. The ISP can maintain one network instead of several, which makes it economical to offer a total private network package to businesses that do not want to manage their corporate wide-area network.
There are several techniques and technologies available for creating IP-based VPNs. Two technologies, Multiprotocol Label Switching (MPLS) and IPSec tunneling, are the leading choices for IP VPNs for service providers. The Cisco 10000 series ESR supports MPLS VPNs natively in the box.
MPLS Virtual Private Networks
MPLS VPNs, which are available in a special software release train, offer all of the value of traditional VPNs. Furthermore, because MPLS VPNs are created in Layer 3, they are more scalable and easier to configure and manage than Layer 2 VPNs.
MPLS VPNs offer:
•
•
•
•
•
MPLS VPN Application
With MPLS, the VPN is encoded in the MPLS label applied to each incoming packet by the provider's edge router (such as the Cisco 10000 series router). Labeled packets are forwarded across an ISP network according to forwarding rules specific to each label. This creates multiple virtual networks on one network infrastructure.
Forwarding rules associated with the labels on packets prevent the packets from being forwarded outside the bounds of the virtual network. These rules can also allow packets to be forwarded between the virtual network and the Internet under controlled circumstances. Figure 1-8 shows a simple application of MPLS that creates two VPNs.
Figure 1-8 MPLS VPNs
As illustrated in Figure 1-8, the following events take place:
1.
2.
3.
4.
Each Cisco 10000 series router can support 1000 or more distinct VPNs, allowing ISPs to plan for large-scale deployments.
Quality of Service Features
The traditional Internet access model is based on a one-size-fits-all approach, in which all packets in the network are treated equally. As network connectivity becomes a more critical resource for companies, ISPs respond by offering value-added services that yield different degrees of access to network services.
For example, many ISPs now offer:
•
•
Business Class Data Service
In the Internet today, all packets are treated equally, whether they belong to a consumer paying low monthly rates and downloading the latest MP3 music files, or to a large corporation transferring a business-to-business e-commerce order for a thousand custom-manufactured industrial engines. When network congestion happens, packets can be delayed or lost without regard to who sent them, how much traffic they are sending, or how much the owners of the data are willing to pay for reliable transport.
ISPs who employ quality of service (QoS) differentiation can offer business class data services, ensuring that more important traffic has an improved chance of reaching its destination, no matter how much congestion is being caused by less important traffic.
The Cisco 10000 series router offers several features that allow ISPs to offer QoS differentiation. These features include:
•
•
•
•
Voice-over-IP Services
A second important application that requires special QoS treatment is the transport of real-time packet data, particularly traffic associated with packet telephony or voice over IP (VoIP).
Successful deployment of a VoIP service requires careful attention to latency through the network, given that small variations in delay caused by network congestion can annoy listeners and can cause high error rates in fax and modem traffic.
The Cisco 10000 series router provides features that allow ISPs to offer controlled-latency services including:
•
•
Access Rate Control
As network bandwidths increase, more and more leased line users are moving from nx64 kbps circuits to DS1 (1.544-Mbps) circuits, and from DS1 to DS3 (44.736-Mbps) circuits, with some subscribers going to direct optical connections at OC-3 (155 Mbps) or faster rates.
For subscribers who are starting to outgrow a single 1.544-Mbps DS1 circuit, this poses a problem. It is a big jump from DS1 at 1.544 Mbps to DS3 at 44.736 Mbps.
The Cisco 10000 series router offers two features to help bridge this gap:
•
•
High-Performance Multilink PPP
You can progress from one DS1 to several DS1s (or from one E1 to several E1s) through the use of the high-performance Multilink PPP feature on the Cisco 10000 series ESR. This industry-standard protocol (IETF RFC 1990) uses special packet headers and procedures to distribute a single stream of packets onto several parallel links and put the stream back together at the receiving end.
The Cisco 10000 series router allows up to 10 DS1 or E1 links to be combined into a parallel path that is up to ten times faster than a single DS1 or E1. Multilink PPP is implemented with special microcode in the parallel express forwarding (PXF) network processor (see the "Forwarding Path" section on page 3-3) for high performance and scalability in the central office aggregation application.
To terminate MLP connections at the customer premises, CPE routers must be configured with MLP support. In Figure 1-9, two CPE routers are each connected to the Cisco 10000 series router by two parallel DS1 links.
Figure 1-9 Multilink PPP Application
Subrate DS3
A second way to get to an intermediate rate is to start with a DS3 and slow it down. Instead of using MLP to combine several DS1 circuits to get a rate that is between the rates for DS1 and DS3, subrating a DS3 interface limits the bandwidth on the DS3.
Although the rate at which bits are clocked across a DS3 is fixed at 44.736 Mbps, vendors of data service units (DSUs) have created various hardware mechanisms to limit the rate of user traffic that can be sent across a DS3. These rate-limiting mechanisms:
•
•
The Cisco 10000 series router line cards can provide subrate DS3 support compatible with PA-T3 and PA-2T3 port adapters on the Cisco 7200 and Cisco 7500 series, and also with products from DS3 DSU vendors. Each unchannelized DS3 attached to the Cisco 10000 series router can be configured for rates ranging from 1 Mbps up to the full rate of the DS3, 44.736 Mbps.
Subrate DS3 provides an additional benefit in flexibility. After the DS3 circuit is installed, ISPs can upgrade customer access rates with a simple software reconfiguration of the line cards at each end of the link. Figure 1-10 shows a possible subrate DS3 configuration.
Figure 1-10 Subrate DS3 Configuration
Network Security
ISPs place high emphasis on network security, because they want to defend their networks and their customers' computers from sophisticated denial of service attacks. Many standard security features are available on the Cisco 10000 series router through the use of Cisco IOS Release 12.0(S) commands. These security features include:
•
•
•
•
Some additional special measures are implemented in the Cisco 10000 series router to resist denial of service attacks. These are discussed in the following sections.
Access Lists
The Cisco 10000 series router implements high-performance access lists (standard and extended), allowing providers to specify the traffic to be forwarded through the router. A new algorithm called turbo ACL has an improved access list so that the router can process large lists with a minimum throughput penalty.
Reverse Path Forwarding Check
Many common denial of service attacks involve forged IP source addresses. The packets appear to be coming from a source that either does not exist or exists at some other point in the network. By using forged source addresses, attackers are better able to hide the attacking machines' identities, making it more challenging to find the culprits.
The Cisco 10000 series router implements a feature called reverse path forwarding (RPF) check, which can be used with both unicast and multicast traffic. The feature checks all packets forwarded through the router to ensure that each one has a plausible source address. The RPF check supplements the usual verifications performed on the destination address and other fields in the IP header.
The RPF feature does not affect the router's packet forwarding rate. Hence, network administrators are not forced to disable it to improve throughput, which is sometimes necessary if a security check impairs performance.
Fast-Path Internet Control Message Protocol
Most denial of service attacks are directed against host computers or web servers and use routers to access the target. However, attacks are also launched against the router through operations that are not normally optimized for throughput. The flooding that results can consume large amounts of router memory or processor cycles.
Internet Control Message Protocol (ICMP) is the network layer Internet protocol (documented in RFC 792) that reports errors and provides other information relevant to IP packet processing. The Cisco 10000 series router implements many of the normal ICMP response functions in the high-speed forwarding path, to ensure that floods of ping packets or other ICMP messages cannot preempt more important router functions.
In addition, messages that are sent from the high-speed forwarding path to the router's internal processor can be categorized by priority. This helps ensure that the router cannot become so busy responding to an overload of unimportant traffic that it neglects essential packets, such as keepalives and route updates, that keep the network operating.
High Availability
Availability corresponds to the probability that a network or network element will function properly at a given moment. From the outset, achieving high availability has been a critical design goal for the Cisco 10000 series router.
Cisco 10000 series router achieves high availability in the following ways:
•
•
•
The Cisco 10000 series router incorporates several mechanisms to reduce its downtime:
•
•
•
•
Route Processor Redundancy Plus
When the Cisco 10000 series router is equipped with a second Performance Routing Engine (PRE):
•
•
Route Processor Redundancy Plus (RPR+) software selects the primary PRE and the secondary PRE during system initialization. RPR+ software also boots Cisco IOS software and tracks changes in the configuration of the primary PRE.
If the primary PRE fails, RPR+ ensures that the secondary is ready to take over operation of the system with minimal disruption. RPR+ software running on each PRE monitors the health of the other PRE so that if the primary PRE fails, the secondary PRE can take over, or if the secondary PRE fails, the network operator can be notified that the system is no longer fully redundant.
Conclusion
The Cisco 10000 series router is an advanced Layer 3 aggregation router that meets the needs of today's ISPs, but also provides the flexibility to satisfy future requirements. It can aggregate thousands of leased line connections, contribute processor-intensive IP network services, and still satisfy the performance and availability requirements of today's Internet market.
The Cisco 10000 series router offers the following advantages to an ISP:
•
•
•
•
•
•
•
To ISP customers, the Cisco 10000 series router offers the following advantages:
•
•
•
To ISPs and their customers, the Cisco 10000 series router offers reliability, availability, and the capacity to handle future growth. It arises from and contributes to the synergy among developers, providers, and users that is necessary in today's telecommunications market.
