Table Of Contents

Configuring Multilink Point-to-Point Protocol Connections

Multilink Point-to-Point Protocol

Feature History for Multilink PPP

MLP Bundles

Restrictions for MLP Bundles

MLP Bundles and PPP Links

System Limits for MLP Bundles

Cisco 10000 series routers do not support VAI bundle interfaces in a PTA configuration. VAI bundles are supported only on the L2TP network server (LNS) for MLPoLNS.MLP Groups

How MLP Determines the Link a Bundle Joins

IP Addresses on MLP-Enabled Links

Valid Ranges for MLP Interfaces

MLP Overhead

Configuration Commands for MLP

interface multilink Command

ppp multilink Command

ppp multilink fragment-delay Command

ppp multilink interleave Command

ppp multilink fragment disable Command

ppp multilink group Command

MLP over Serial Interfaces

Performance and Scalability for MLP over Serial Interfaces

Restrictions and Limitations for MLP over Serial Interfaces

Single-VC MLP over ATM Virtual Circuits

Performance and Scalability for Single-VC MLP over ATM

Restrictions and Limitations for Single-VC MLP over ATM

Multi-VC MLP over ATM Virtual Circuits

Performance and Scalability for Multi-VC MLP over ATM VCs

Restrictions and Limitations for Multi-VC MLP over ATM VCs

MLP on LNS

About MLP on LNS

PPP multilink link max Command

Performance and Scalability of MLP on LNS

PXF Memory and Performance Impact for MLP on LNS

Scenario 1

Scenario 2

Restrictions and Limitations for MLP on LNS

Configuring MLP on LNS

MLPoE LAC Switching

Restrictions for MLPoE LAC Switching

MLP-Based Link Fragmentation and Interleaving

Configuring MLP Bundles and Member Links

Creating an MLP Bundle Interface

Configuration Example for Creating an MLP Bundle Interface

Enabling MLP on a Virtual Template

Configuration Example for Enabling MLP on a Virtual Template

Adding a Serial Member Link to an MLP Bundle

Adding an ATM Member Link to an MLP Bundle

Configuration Example for Adding ATM Links to an MLP Bundle

Moving a Member Link to a Different MLP Bundle

Removing a Member Link from an MLP Bundle

Changing the Default Endpoint Discriminator

Configuration Example for Changing the Endpoint Discriminator

Configuration Examples for Configuring MLP

Configuration Example for Configuring MLP over Serial Interfaces

Configuration Example for Configuring Multi-VC MLP over ATM

Configuration Example for MLP on LNS

Configuration Example for MLPoE LAC Switching

Verifying and Monitoring MLP Connections

Bundle Counters and Link Counters

Verification Examples for MLP Connections

Verification Example for the show interfaces multilink Command

Verification Example for the show ppp multilink Command

Verification Example for the show interfaces multilink stat Command

Related Documentation

Configuring Multilink Point-to-Point Protocol Connections

---

LAN-based applications and information transfer services, such as electronic mail, transmit large amounts of traffic, placing increased demand on wide-area networks (WANs). Multilink Point-to-Point Protocol (MLP) is a reliable and cost-effective solution that makes efficient use of WAN links.

This chapter describes MLP and how to configure it on serial and ATM connections on the Cisco 10000 series router. It includes the following topics:

•Multilink Point-to-Point Protocol

•MLP Bundles

•Cisco 10000 series routers do not support VAI bundle interfaces in a PTA configuration. VAI bundles are supported only on the L2TP network server (LNS) for MLPoLNS.MLP Groups

•Cisco 10000 series routers do not support VAI bundle interfaces in a PTA configuration. VAI bundles are supported only on the L2TP network server (LNS) for MLPoLNS.MLP Groups

•How MLP Determines the Link a Bundle Joins

•IP Addresses on MLP-Enabled Links

•Valid Ranges for MLP Interfaces

•MLP Overhead

•Configuration Commands for MLP

•MLP over Serial Interfaces

•Single-VC MLP over ATM Virtual Circuits

•Multi-VC MLP over ATM Virtual Circuits

•MLP-Based Link Fragmentation and Interleaving

•Configuring MLP Bundles and Member Links

•Configuration Examples for Configuring MLP

•Verifying and Monitoring MLP Connections

•Related Documentation

Multilink Point-to-Point Protocol

Multilink Point-to-Point Protocol (MLP) is used to combine multiple physical links into a single logical connection or MLP bundle (see Figure 19-1). Using MLP, you can increase bandwidth and more easily manage all of the circuits through a single interface. The MLP connection has a maximum bandwidth that is equal to the sum of the bandwidths of the component links. MLP also provides load balancing, multivendor interoperability, packet fragmentation and reassembly, and increased redundancy. The Cisco 10008 router implements the MLP specifications defined in RFC 1990.

MLP provides traffic load balancing over multiple wide-area network (WAN) links by sending packets and packet fragments over the links of bundle members. The multiple links come up in response to a defined load threshold. MLP mechanisms can calculate load on both inbound and outbound traffic, or on either direction as needed for the traffic between specific sites. MLP provides bandwidth on demand and reduces transmission latency across WAN links.

MLP allows packets to be fragmented and the fragments to be sent at the same time over multiple point-to-point links to the same remote address. Large nonreal-time packets are multilink encapsulated and fragmented into a small enough size to satisfy the delay requirements of real-time traffic. However, the smaller real-time packets are not multilink encapsulated. Instead, MLP interleaving provides a special transmit queue (priority queue) for these delay-sensitive packets to allow the packets to be sent earlier than other packet flows. Real-time packets remain intact and MLP interleaving mechanisms send the real-time packets between fragments of the larger nonreal-time packets. For more information about link fragmentation and interleaving, see the "Fragmenting and Interleaving Real-Time and Nonreal-Time Packets" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

MLP can provide increased redundancy by allowing traffic to flow over the remaining member links when a port fails. You can configure the member links on separate physical ports on the same line card or on different line cards. If a port becomes unavailable, MLP directs traffic over the remaining member links with minimal disruption to the traffic flow.

MLP mechanisms preserve packet ordering over an entire bundle, guaranteeing that network packets are processed at the receiving system in the same order that they are logically transmitted.

Valid multilink interface values for MLP over serial or multi-VC MLP over ATM are from 1 to 9999 (Release 12.2(28)SB and later), or from 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later). For example:

Router(config)# interface multilink 8

The Cisco 10008 router supports the following MLP features:

•MLP over Serial Interfaces

•Single-VC MLP over ATM Virtual Circuits

•Multi-VC MLP over ATM Virtual Circuits

•MLP on LNS

•MLPoE LAC Switching

Feature History for Multilink PPP

Cisco IOS Release	Description	Required PRE
12.0(23)SX	The MLP over Serial feature was introduced on the Cisco 10000 series router.	PRE1
12.2(28)SB	The MLP over Serial, Single-VC MLP over ATM VCs, and Multi-VC MLP over ATM VCs features were introduced on the PRE2.	PRE2
12.2(31)SB2	Support was added for the PRE3 and the valid multilink interface ranges for MLP over serial or multi-VC MLP over ATM changed from 1 to 9999 (Release 12.2(28)SB and later) to from 1 to 9999 and 65,536 to 2,147,483,647.	PRE3
12.2(33)SB	The MLPPP on LNS feature was introduced on the Cisco 10000 series router that is supported on the PRE3 and PRE4. This feature is not supported on the PRE2.	PRE3 and PRE4
12.2(33)SB2	The MLPoE LAC Switching feature is been introduced on the Cisco 10000 series router.	PRE3

MLP Bundles

MLP combines multiple physical links into a logical bundle called an MLP bundle (see Figure 19-1). An MLP bundle is a single, virtual interface that connects to the peer system. Having a single virtual interface enables fancy queuing and QoS to be applied to the traffic on the virtual interface (for example, policing and traffic shaping can be applied to the traffic flows). Each individual link to the peer system might be doing some form of fancy queuing, but none of the links knows about the traffic on the other parallel links. Fancy queuing and QoS cannot be applied uniformly to the entire aggregate traffic between the system and its peer system. A single virtual interface also simplifies the task of monitoring traffic to the peer system (for example, traffic statistics are all on one interface).

Figure 19-1 Multilink PPP Bundle

An endpoint discriminator is used to identify the member links of the MLP bundle.

Restrictions for MLP Bundles

The router supports links equal to T1/E1 or less for MLPPP bundling. You cannot bundle high-speed links (for example, E3) because the router can store only 50 ms of data based on the E1 speed.

MLP Bundles and PPP Links

MLP works with fully functional Point-to-Point Protocol (PPP) interfaces. An MLP bundle can consist of a PPP over serial link and a PPP over ATM link. As long as each link behaves like a standard serial interface, the mixed links work properly in a bundle.

Adding the ppp multilink group command to a link's configuration does not make that link part of the specified bundle. This command only places a restriction on the link. If the link negotiates to use multilink, then it must provide the proper identification to join the bundle on the multilink interface or to activate a bundle on that interface. If the link provides identification that coincides with another active bundle in the system, or the link fails to match the identity of a bundle that is already active on the multilink group interface, the connection terminates.

A link joins an MLP bundle only if it negotiates to use multilink when the connection is established and the identification information exchanged matches that of an existing bundle. If a link supplies identification information that does not match any known bundle, MLP creates a new bundle for the user.

System Limits for MLP Bundles

Table 19-1 lists the system limits for MLP bundles.

Table 19-1 System Limits for MLP Bundles

Feature	Maximum No. of Members Per Bundle	Maximum No. of Bundles Per System	Maximum No. of Member Links Per System	Multilink Interface Range	LFI Supported
MLP over Serial	10	1250	2500	1 to 9999 (Release 12.2(28)SB and later) and from 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later)	Yes Interleaving on all member links
Single-VC MLP over ATM	1	8192	8192	10,000 and higher	Yes Interleaving on 1 member link
Multi-VC MLP over ATM	10	1250	2500	1 to 9999 (Release 12.2(28)SB and later) or from 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later)	Yes Interleaving on 1 member link
MLP LAC switched MLPPPoE	1	10240	10240	—	Yes Interleaving on 1 member link

Note The multilink interface ranges described in Table 19-1 require Cisco IOS Release 12.2(28)SB or later releases. For releases earlier than Cisco IOS Release 12.2(28)SB, the valid multilink interface range is 1 to 2,147,483,647.

Cisco 10000 series routers do not support VAI bundle interfaces in a PTA configuration. VAI bundles are supported only on the L2TP network server (LNS) for MLPoLNS.MLP Groups

When you configure the ppp multilink group command on a link, the command applies a restriction to the link that indicates the link is not allowed to join any bundle other than the indicated group interface, and that the connection is to be terminated if the peer system attempts to join a different bundle.

A link actually joins a bundle when the identification keys for that link match the identification keys for an existing bundle (see the "How MLP Determines the Link a Bundle Joins" section). Configuring the ppp multilink group command on a link does not allow the link to bypass this process, unless a bundle does not already exist for this particular user. When matching links to bundles, the identification keys are always the determining factors.

Because the ppp multilink group command merely places a restriction on the link, any MLP-enabled link that is not assigned to a particular multilink group can join the dedicated bundle interface if it provides the correct identification keys for that dedicated bundle. Removing the ppp multilink group command from an active link that currently is a member of a multilink group does not make that link leave the bundle because the link is still a valid member. It is just no longer restricted to this one bundle.

How MLP Determines the Link a Bundle Joins

A link joins a bundle when the identification keys for that link match the identification keys for an existing bundle.

Two keys define the identity of a remote system: the PPP username and the MLP endpoint discriminator. The PPP authentication mechanisms (for example, PAP or CHAP) learn the PPP username. The endpoint discriminator is an option negotiated by the Link Control Protocol (LCP). Therefore, a bundle consists of all of the links that have the same PPP username and endpoint discriminator.

A link that does not provide a PPP username or endpoint discriminator is an anonymous link. MLP collects all of the anonymous links into a single bundle referred to as the anonymous bundle or default bundle. Typically, there can be only one anonymous bundle. Any anonymous links that negotiate MLP join (or create) the anonymous bundle.

When using multilink group interfaces, more than one anonymous peer is allowed. When you preassign a link to an MLP bundle by using the ppp multilink group command, and the link is anonymous, the link joins the bundle interface it is assigned to if the interface is not already active and associated with a nonanonymous user.

MLP determines the bundle a link joins in the following steps:

1. When a link connects, MLP creates a bundle name identifier for the link.

2. MLP then searches for a bundle with the same bundle name identifier.

–If a bundle with the same identifier exists, the link joins that bundle.

–If a bundle with the same identifier does not exist, MLP creates a new bundle with the same identifier as the link, and the link is the first link in the bundle.

Table 19-2 describes the commands and associated algorithm used to generate a bundle name. In the table, "username" typically means the authenticated username; however, an alternate name can be used instead. The alternate name is usually an expanded version of the username (for example, VPDN tunnels might include the network access server name) or a name derived from other sources.

Table 19-2 Bundle Name Generation 

Command	Bundle Name Generation Algorithm
multilink bundle-name authenticated	The bundle name is the peer's username, if available. If the peer does not provide a username, the algorithm uses the peer's endpoint discriminator. Note The authenticated keyword specifies that the bundle name is based on whatever notion of a username the system can derive. The endpoint discriminator is ignored entirely, unless it is the only name that can be found. The multilink bundle-name authenticated command is the default naming policy.
multilink bundle-name endpoint	The bundle name is the peer's endpoint discriminator. If there is no endpoint discriminator, the algorithm uses the peer's username.
multilink bundle-name both	The name of the bundle is a concatenation of the username and the endpoint discriminator.

IP Addresses on MLP-Enabled Links

Configuring an IP address on a link used for MLP does not always work as expected. For example, consider the following configuration:

interface Serial 1/0/0

ip address 10.2.3.4 255.255.255.0

encapsulation ppp

ppp multilink

You might expect the following behavior as a result of this configuration:

•If the interface does not negotiate to use MLP and the interface comes up as a regular PPP link, then the interface negotiates the Internet Protocol Control Protocol (IPCP) and its local address is 10.2.3.4.

•If the interface did negotiate to use MLP, then the configured IP address is meaningless because the link is not visible to IP while it is part of a bundle. The bundle is a network-level interface and can have its own IP address, depending on the configuration used for the bundle.

Instead, if a link with an IP address configured comes up and joins a bundle, IP installs a route directly to that link interface and it might try to route packets directly to that link, bypassing the MLP bundle. This behavior occurs because IP considers an interface to be up for IP traffic whenever IP is configured on the interface and the interface is up. MLP intercepts and discards these misdirected frames. This condition occurs frequently if you use a virtual template interface to configure both the PPPoX member links and the bundle interface.

Using unnumbered IP interfaces enables you to work around IP problems and configure an IP address on an MLP-enabled link. The following example shows how to configure Multi-VC MLP over ATM using an unnumbered IP interface:

!

interface Multilink1

ip unnumbered Loopback0

peer default ip address pool mlpoa_pool

ppp chap hostname m1 

ppp multilink

ppp multilink group 1

!

interface atm 2/0/0

no ip address

!

interface atm 2/0/0.1 point-to-point

pvc 0/32

ppp multilink group 1

vbr-nrt 128 64 20

encapsulation aal5mux ppp Virtual-Template1

! 

! 

interface atm 2/0/0.2 point-to-point

pvc 0/33

ppp multilink group 1

vbr-nrt 128 64 20 

encapsulation aal5mux ppp Virtual-Template1

!

interface Virtual-Template1

no ip address

keepalive 30

ppp max-configure 110

ppp max-failure 100

ppp multilink

ppp timeout retry 5

!

ip local pool mlpoa_pool 100.1.1.1 100.1.7.254

!

Valid Ranges for MLP Interfaces

Table 19-3 lists the valid ranges you can specify when creating MLP interfaces using the interface multilink command.

Table 19-3 MLP Interface Ranges

Cisco IOS Release	PRE2 MLP Interface Ranges	PRE3 MLP Interface Ranges
Release 12.2(28)SB and later	1 to 9999	—
Release 12.2(31)SB2 and later	1 to 9999 65,536 to 2,147,483,647	1 to 9999 65,536 to 2,147,483,647

MLP Overhead

MLP encapsulation adds six extra bytes (4 header, 2 checksum) to each outbound packet. These overhead bytes reduce the effective bandwidth on the connection; therefore, the throughput for an MLP bundle is slightly less than an equivalent bandwidth connection that is not using MLP. If the average packet size is large, the extra MLP overhead is not readily apparent; however, if the average packet size is small, the extra overhead becomes more noticeable.

Using MLP fragmentation adds additional overhead to a packet. Each fragment contains six bytes of MLP header plus a link encapsulation header.

Configuration Commands for MLP

This section describes the following commands used to configure MLP and MLP-based link fragmentation and interleaving:

•interface multilink Command

•ppp multilink Command

•ppp multilink fragment-delay Command

•ppp multilink interleave Command

•ppp multilink fragment disable Command

•ppp multilink group Command

For more information about MLP-based link fragmentation and interleaving, see the Cisco 10000 Series Router Quality of Service Configuration Guide.

interface multilink Command

To create and configure a multilink bundle, use the interface multilink command in global configuration mode. To remove a multilink bundle, use the no form of the command.

interface multilink multilink-bundle-number

no interface multilink multilink-bundle-number

Syntax Description

multilink-bundle-number

A nonzero number that identifies the multilink bundle.

Command History

Cisco IOS Release	Description
12.0	The interface multilink command was introduced on the Cisco 10000 series router.
12.2(16)BX	This command was introduced on the PRE2.
12.2(28)SB	This command was integrated into Cisco IOS Release 12.2(28)SB.
12.2(31)SB2	This command was introduced on the PRE3. Valid multilink interface values changed. See the Usage Guidelines or Table 19-3.

Defaults

No multilink interfaces are configured.

Usage Guidelines

For Cisco IOS Release 12.2(28)SB and later releases, the range of valid values for multilink interfaces are the following:

•MLP over Serial—1 to 9999 (Release 12.2(28)SB and later), and 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later)

•Single-VC MLP over ATM—10,000 and higher

•Multi-VC MLP over ATM—1 to 9999 (Release 12.2(28)SB and later), and 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later)

For releases earlier than Cisco IOS Release 12.2(28)SB, the valid multilink interface range is 1 to 2,147,483,647.

ppp multilink Command

To enable MLP on an interface, use the ppp multilink command in interface configuration mode. To disable MLP, use the no form of the command.

ppp multilink

no ppp multilink

Command History

Cisco IOS Release	Description
12.0(23)SX	The ppp multilink command was introduced on the Cisco 10000 series router.
12.2(16)BX	This command was introduced on the PRE2.
12.2(28)SB	This command was integrated into Cisco IOS Release 12.2(28)SB.

Defaults

The command is disabled.

Usage Guidelines

The ppp multilink command applies only to interfaces that use Point-to-Point Protocol (PPP) encapsulation.

When you use the ppp multilink command, the first channel negotiates the appropriate Network Control Protocol (NCP) layers (such as the IP Control Protocol and IPX Control Protocol), but subsequent links negotiate only the Link Control Protocol (LCP) and MLP.

ppp multilink fragment-delay Command

To specify a maximum size in units of time for packet fragments on a MLP bundle, use the ppp multilink fragment-delay command in interface configuration mode. To reset the maximum delay to the default value, use the no form of the command.

ppp multilink fragment-delay delay-max

no ppp multilink fragment-delay delay-max

Syntax Description

delay-max

Specifies the maximum amount of time, in milliseconds, that is required to transmit a fragment. Valid values are from 1 to 1000 milliseconds.

Command History

Cisco IOS Release	Description
12.0(23)SX	The ppp multilink fragment-delay command was introduced on the Cisco 10000 series router.
12.2(16)BX	This command was introduced on the PRE2.
12.2(28)SB	This command was integrated into Cisco IOS Release 12.2(28)SB.

Defaults

If fragmentation is enabled, the fragment delay is 30 milliseconds.

Usage Guidelines

The ppp multilink fragment-delay command is useful when packets are interleaved and traffic characteristics such as delay, jitter, and load balancing must be tightly controlled.

MLP chooses a fragment size on the basis of the maximum delay allowed. If real-time traffic requires a certain maximum boundary on delay, using the ppp multilink fragment-delay command to set that maximum time can ensure that a real-time packet gets interleaved within the fragments of a large packet.

By default, MLP has no fragment size constraint, but the maximum number of fragments is constrained by the number of links. If interleaving is enabled, or if a fragment delay is explicitly configured with the ppp multilink fragment-delay command, then MLP uses a different fragmentation algorithm. In this mode, the number of fragments is unconstrained, but the size of each fragment is limited to the fragment-delay value, or 30 milliseconds if the fragment delay has not been configured.

The ppp multilink fragment-delay command is configured under the multilink interface. The value assigned to the delay-max argument is scaled by the speed at which a link can convert the time value into a byte value.

ppp multilink interleave Command

To enable interleaving of real-time packets among the fragments of larger nonreal-time packets on a MLP bundle, use the ppp multilink interleave command in interface configuration mode. To disable interleaving, use the no form of the command.

ppp multilink interleave

no ppp multilink interleave

Command History

Cisco IOS Release	Description
12.0(23)SX	The ppp multilink interleave command was introduced on the Cisco 10000 series router.
12.2(16)BX	This command was introduced on the PRE2.
12.2(28)SB	This command was integrated into Cisco IOS Release 12.2(28)SB.

Defaults

Interleaving is disabled.

Usage Guidelines

The ppp multilink interleave command applies to multilink interfaces, which are used to configure a bundle.

Interleaving works only when the queuing mode on the bundle is set to fair queuing.

If interleaving is enabled when fragment delay is not configured, the default delay is 30 milliseconds. The fragment size is derived from that delay, depending on the bandwidths of the links.

ppp multilink fragment disable Command

To disable packet fragmentation, use the ppp multilink fragment disable command in interface configuration mode. To enable fragmentation, use the no form of this command.

ppp multilink fragment disable

no ppp multilink fragment disable

Command History

Cisco IOS Release	Description
11.3	This command was introduced as ppp multilink fragmentation.
12.2	The no ppp multilink fragmentation command was changed to ppp multilink fragment disable. The no ppp multilink fragmentation command was recognized and accepted through Cisco IOS Release 12.2.
12.2(28)SB	This command was integrated into Cisco IOS Release 12.2(28)SB.

Usage Guidelines

The ppp multilink fragment delay and ppp multilink interleave commands have precedence over the ppp multilink fragment disable command. Therefore, the ppp multilink fragment disable command has no effect if these commands are configured for a multilink interface and the following message displays:

Warning: 'ppp multilink fragment disable' or 'ppp multilink fragment maximum' will be 
ignored, since multilink interleaving or fragment delay has been configured and have 
higher precedence.

To completely disable fragmentation, you must do the following:

Router(config-if)# no ppp multilink fragment delay

Router(config-if)# no ppp multilink interleave

Router(config-if)# ppp multilink fragment disable

ppp multilink group Command

To restrict a physical link to joining only a designated multilink group interface, use the ppp multilink group command in interface configuration mode. To remove the restriction, use the no form of the command.

ppp multilink group group-number

no ppp multilink group group-number

Syntax Description

group-number

Identifies the multilink group. This number must be identical to the multilink-bundle-number you assigned to the multilink interface. Valid values are: •MLP over Serial—1 to 9999 •Single-VC MLP over ATM—10,000 and higher •Multi-VC MLP over ATM—1 to 9999

Command History

Cisco IOS Release	Description
12.0	The multilink-group command was introduced on the Cisco 10000 series router.
12.2	This command was changed to ppp multilink group. The multilink-group command is accepted by the command line interpreter through Cisco IOS Release 12.2.
12.2(28)SB	This command was integrated into Cisco IOS Release 12.2(28)SB.

Defaults

The command is disabled.

Usage Guidelines

By default the ppp multilink group command is disabled, which means the link can negotiate to join any bundle in the system.

When the ppp multilink group command is configured, the physical link is restricted from joining any but the designated multilink group interface. If a peer at the other end of the link tries to join a different bundle, the connection is severed. This restriction applies when MLP is negotiated between the local end and the peer system. The link can still come up as a regular PPP interface.

MLP over Serial Interfaces

The MLP over Serial interfaces feature enables you to bundle together T1 interfaces into a single logical connection called an MLP bundle (see the "MLP Bundles" section). MLP over Serial also provides the following functions:

•Load balancing—MLP provides bandwidth on demand and uses load balancing across all member links (up to 10) to transmit packets and packet fragments. MLP mechanisms calculate the load on either the inbound or outbound traffic between specific sites. Because MLP splits packets and fragments across all member links during transmission, MLP reduces transmission latency across WAN links.

•Increased redundancy—MLP allows traffic to flow over the remaining member links when a port fails. By configuring an MLP bundle that consists of T1 lines from more than one line card, if one line card stops operating, the part of the bundle on the other line cards continues to operate.

•Link fragmentation and interleaving—The MLP fragmenting mechanism fragments large nonreal-time packets and sends the fragments at the same time over multiple point-to-point links to the same remote address. Smaller real-time packets remain intact. The MLP interleaving mechanism sends the real-time packets between the fragments of the nonreal-time packets, thus reducing real-time packet delay. For more information about link fragmentation and interleaving, see the "Fragmenting and Interleaving Real-Time and Nonreal-Time Packets" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

Figure 19-2 shows an MLP bundle that consists of T1 interfaces from three T3 interfaces.

Figure 19-2 MLP Bundle for Multilink PPP over Serial Connections

Performance and Scalability for MLP over Serial Interfaces

Configure the hold-queue command in interface configuration mode for all physical interfaces. For example:

Router(config-if)# hold-queue 4096 in

For more information, see the "Scalability and Performance" chapter in this guide.

Restrictions and Limitations for MLP over Serial Interfaces

•A multilink bundle can have up to 10 member links. The router supports both full T1 interfaces and fractional T1 interfaces as member links, but fractional T1 interfaces are supported only when LFI is enabled.

---

Note You can terminate the serial links on multiple line cards in the router chassis if all of the links are the same type, such as T1 or E1.

---

•The router supports a maximum of 1250 bundles per system and a maximum of 2500 member links per system.

•The valid multilink interface ranges are from 1 to 9999 (Release 12.2(28)SB and later) and from 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later). For example:

Router(config)# interface multilink 8

•Interleaving is supported on all member links. MLP over Serial-based LFI must be enabled on an interface that has interleaving turned on.

•All member links in an MLP bundle must have the same encapsulation type and bandwidth.

•If a virtual template attached to a member link specifies a bandwidth, the router does not clone the specified bandwidth to the MLP bundle and the member links.

•You cannot manually configure the bandwidth on a bundle interface by using the bandwidth command.

•You cannot apply a virtual template with MLP configured to an MLP bundle.

•We strongly recommend that you use only strict priority queues when configuring MLP over Serial-based LFI. For more information, see the "Prioritizing Services" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

Single-VC MLP over ATM Virtual Circuits

The Single-VC MLP over ATM virtual circuits (VCs) feature enhances the MLP over Serial interfaces feature by enabling you to configure multilink Point-to-Point Protocol (MLP) on an ATM VC. By doing so, you can aggregate multiple data paths (for example, PPP over ATM encapsulated ATM VCs) into a single logical connection called an MLP bundle (see the "MLP Bundles" section). The MLP bundle can have only one member link.

MLP supports link fragmentation and interleaving (LFI). When enabled, the MLP fragmentation mechanism multilink encapsulates large nonreal-time packets and fragments them into a small enough size to satisfy the delay requirements of real-time traffic. The smaller real-time packets remain intact and MLP sends the packets to a special transmit queue, allowing the packets to be sent earlier than other packet flows. The MLP interleaving mechanism sends the real-time packets between the fragments of the nonreal-time packets. For more information about link fragmentation and interleaving, see the "Fragmenting and Interleaving Real-Time and Nonreal-Time Packets" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

Performance and Scalability for Single-VC MLP over ATM

•Configure the hold-queue command in interface configuration mode for all physical interfaces, except when configuring the OC-12 ATM line card. The 1-Port OC-12 ATM line card does not require the hold-queue command. For example:

Router(config-if)# hold-queue 4096 in

•Configure the following commands and recommended values on the virtual template interface:

–ppp max-configure 110

–ppp max-failure 100

–ppp timeout retry 5

–keepalive 30

For example:

Router(config-if)# ppp max-configure 110

Router(config-if)# ppp max-failure 100

Router(config-if)# ppp timeout retry 5

Router(config-if)# keepalive 30

For more information, see the "Scalability and Performance" chapter in this guide.

Restrictions and Limitations for Single-VC MLP over ATM

•Only one member link is supported per bundle.

•Single-VC MLP over ATM member links are restricted to nonaggregated PVCs (for example, variable bit rate-nonreal-time [VBR-nrt] and constant bit rate [CBR] ATM traffic classes only).

•The router supports a maximum of 8192 bundles per system and 8192 member links per system.

•Each member link can have a bandwidth rate up to 2048 kbps.

•The router only supports member links with the same encapsulation type.

•MLP PVCs cannot be on-demand VCs that are automatically provisioned.

•Associating MLP over ATM PVCs with ATM virtual paths (VPs) is discouraged, though not prevented.

•The valid multilink interface values are 10000 to 65534. For example:

Router(config)# interface multilink 10004

The values higher than 65534 are used for multi-member bundles

•Cisco IOS software supports a maximum of 4096 total virtual template interfaces.

•You cannot manually configure the bandwidth on a bundle interface using the bandwidth command.

•If a virtual template attached to a member link specifies a bandwidth, the router does not clone the specified bandwidth to the MLP bundle and the member links.

•You cannot apply a virtual template with MLP configured to an MLP bundle.

•If link fragmentation and interleaving (LFI) is enabled, only one link is used for interleaving. For more information, see the "Fragmenting and Interleaving Real-Time and Nonreal-Time Packets" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

•We strongly recommend that you use only strict priority queues when configuring MLP over ATM-based LFI. For more information, see the "Prioritizing Services" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

Multi-VC MLP over ATM Virtual Circuits

The Multi-VC MLP over ATM virtual circuits (VCs) feature enhances the MLP over Serial interfaces feature by enabling you to configure multilink Point-to-Point Protocol (MLP) on multiple ATM VCs. By doing so, you can aggregate multiple data paths (for example, PPP over ATM encapsulated ATM VCs) into a single logical connection called an MLP bundle (see the "MLP Bundles" section). An MLP bundle can have up to 10 member links.

Multi-VC MLP over ATM provides the following functions:

•Load balancing—MLP provides bandwidth on demand and uses load balancing across all member links (up to 10) to transmit packets and packet fragments. The multiple links come up in response to a defined load threshold. MLP mechanisms calculate load on both inbound and outbound traffic, or on either direction as needed for traffic between specific sites. Because MLP uses all member links to transmit packets and fragments, MLP reduces transmission latency across WAN links.

•Increased redundancy—MLP allows traffic to flow over the remaining member links when a port fails. You can configure the member links on separate physical ports on the same line card or on different line cards. If a port becomes unavailable, MLP directs traffic over the remaining member links with minimal disruption to the traffic flow. MLP mechanisms preserve packet ordering over an entire bundle.

•Link fragmentation and interleaving—The MLP fragmentation mechanism fragments packets and sends the fragments at the same time over multiple point-to-point links to the same remote address. MLP multilink encapsulates large nonreal-time packets and fragments them into a small enough size to satisfy the delay requirements of real-time traffic. The smaller real-time packets remain intact and MLP sends the packets to a special transmit queue, allowing the packets to be sent earlier than other packet flows. The MLP interleaving mechanism sends the real-time packets between the fragments of the nonreal-time packets.

For more information about link fragmentation and interleaving, see the "Fragmenting and Interleaving Real-Time and Nonreal-Time Packets" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

Performance and Scalability for Multi-VC MLP over ATM VCs

•Configure the hold-queue command in interface configuration mode for all physical interfaces, except when configuring the OC-12 ATM line card. The 1-Port OC-12 ATM line card does not require the hold-queue command. For example:

Router(config-if)# hold-queue 4096 in

•Configure the following commands and recommended values on the virtual template interface:

–ppp max-configure 110

–ppp max-failure 100

–ppp timeout retry 5

–keepalive 30

For example:

Router(config-if)# ppp max-configure 110

Router(config-if)# ppp max-failure 100

Router(config-if)# ppp timeout retry 5

Router(config-if)# keepalive 30

For more information, see the "Scalability and Performance" chapter in this guide.

Restrictions and Limitations for Multi-VC MLP over ATM VCs

•A maximum of 10 member links is supported per bundle.

•MLP over ATM member links are restricted to nonaggregated PVCs (for example, variable bit rate-nonreal-time [VBR-nrt] and constant bit rate [CBR] ATM traffic classes only).

•The router supports a maximum of 1250 bundles per system and 2500 member links per system.

•Each member link can have a bandwidth rate up to 2048 kbps.

•The router only supports member links with the same encapsulation type.

•The valid multilink interface ranges are from 1 to 9999 (Release 12.2(28)SB and later) and from 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later). For example:

Router(config)# interface multilink 8

•MLP PVCs cannot be on-demand VCs that are automatically provisioned.

•Associating MLP over ATM PVCs with ATM virtual paths (VPs) is discouraged, though not prevented.

•Cisco IOS software supports a maximum of 4096 total virtual template interfaces.

•You cannot manually configure the bandwidth on a bundle interface using the bandwidth command.

•You cannot apply a virtual template with MLP configured to an MLP bundle.

•If a virtual template attached to a member link specifies a bandwidth, the router does not clone the specified bandwidth to the MLP bundle and the member links.

•If link fragmentation and interleaving (LFI) is enabled, only one link is used for interleaving. For more information, see the "Fragmenting and Interleaving Real-Time and Nonreal-Time Packets" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

•We strongly recommend that you use only strict priority queues when configuring Multi-VC MLP over ATM-based LFI. For more information, see the "Prioritizing Services" chapter in the Cisco 10000 Series Router Quality of Service Configuration Guide.

MLP on LNS

Networks are migrating from the digital subscriber line (DSL) aggregation network connectivity to broadband remote access server (BRAS), with the mix of Ethernet and ATM access networks. Therefore, there is an increasing need to support MLP, and, link fragmentation and interleaving (LFI), to allow high-priority, low-latency packets to be interleaved between fragments of lower-priority, higher-latency packets. Voice over IP (VoIP) is an example of a low-latency service.

In the Cisco 12.2(33)SB release, the MLP on LNS feature is introduced for asymmetric digital subscriber line (ADSL) deployments where the upstream bandwidth (BW) is low. The MLP on LNS feature can receive fragments from the customer premises equipment (CPE), ensuring that there is less latency upstream, even if a large packet gets in between the voice packets.

The MLP on LNS feature bundles together a virtual private dial network (VPDN) session on a single logical connection, which forms an MLP bundle on the LNS. Before the Cisco IOS 12.2(33)SB release, Cisco 10000 series routers supported only multilink bundle termination on the PPP termination aggregation (PTA) router. In the Cisco IOS 12.2(33)SB release, Cisco 10000 series routers support MLP termination on the LNS also. Figure 19-3 shows an MLP on LNS application.

Figure 19-3 MLP on LNS Application

The MLP on LNS feature is described in the following sections:

•About MLP on LNS

•PPP multilink link max Command

•PXF Memory and Performance Impact for MLP on LNS

•Restrictions and Limitations for MLP on LNS

•Configuring MLP on LNS

About MLP on LNS

The multilink interface-based configuration requires one virtual template per bundle so that the multilink group # command can be configured on the virtual template. However, for the MLP on LNS feature, you can only scale up to 2000 virtual templates.

To address the virtual template scaling issue and to avoid cumbersome configuration management, in the Cisco IOS 12.2(33)SB release, virtual access bundles are supported. In virtual access bundles, the bundle interface is cloned from the virtual template when the first member link is negotiated on the LNS. The virtual access bundle support is limited to bundle termination on LNS.

Before the Cisco IOS 12.2(33)SB release, multilink interface-based configuration was used to distinguish between single and multi-member bundles. However, for the virtual access based bundle interface, you can no longer use the interface number range to distinguish between single and multi-member bundles because the bundles are generated dynamically in the Cisco IOS 12.2(33)SB release. To distinguish single and multi-member bundles, the user-specified value for the ppp multilinks max link # command is used.

The following two diagrams show two different MLP on LNS bundle configurations supported with the Cisco IOS 12.2(33)SB release. Figure 19-4 shows MLP on CPE for dial-up networks.

Figure 19-4 MLP on LNS-Multimember Bundle

Figure 19-5 shows a single-member bundle on the CPE. These are single-member bundles where the traffic received by the Cisco 10000 router is fragmented to interleave high-priority traffic in between low-priority network traffic.

Figure 19-5 MLP on LNS-Single-Member Bundle

To accommodate the scaling requirements of up to 2040 multi member and 10240 single-member bundles for the MLP on LNS feature, an additional reassembly buffer is reserved in the external column memory (XCM). The reassembly buffer that was reserved in the Cobalt space is used for multi-member bundles and the XCM reassembly buffer is used for single-member bundles.

The fixed reassembly table size for the MLP on LNS feature to buffer fragments is 256 entries. The reassembly table size restricts the maximum differential delay for all of the different paths for the member links from the CPE to the LNS. For example, if there are 10 members in a bundle, and one of the members is associated with a "slow" (high delay) path, then the other nine members must have their fragments/packets buffered while waiting for the slower link. Because the reassembly table stores descriptors, each entry represents one fragment or a whole packet if fragmentation is not in effect. The amount of time each fragment takes to get transmitted is equal to the configured fragment delay, which is independent of link bandwidth. If fragmentation is not in effect, the transmit time depends on the packet size, with smaller packets being slower. Therefore, the amount of tolerated differential delay is the reassembly table buffering limit for the other 9 links:

(256 / 9) * frag_delay = 28.4 * frag_delay

---

Note The default differential delay for MLP on LNS is 50ms.

---

Table 19-4 shows the resource usage on Cisco 10000 series router.

Table 19-4 Resource Usage

	VCCI1	HWIDB2	SWIDB3	PBLT4
Cisco 10000 series routers MAX	64000	Memory dependent	Memory dependent	16000
Bundle interface	1	1	1	1
Member link single-member bundle	1	1	1	0
Member link multi-member bundle	1	1	1	1

1 A virtual circuit connection identifier (VCCI) is a variable that identifies a virtual circuit connection between two nodes. 2 A hardware interface descriptor block (HWIDB) represents a physical interface, which includes physical ports and channelized interface definitions. 3 A software interface descriptor block (SWIDB) represents a logical sub interface such as permanent virtual circuit (PVC) or virtual LAN (VLAN), or a Layer 2 encapsulation (Point-to-Point Protocol (PPP)), or high-level data link control (HDLC). 4 An HQF resource that is used by the RP and PXF to program physical layer scheduling for an interface. It could be considered an instance of physical layer scheduling; Cisco 10000 series routers currently support 16K such instances. All bundle interfaces (single or multi-member bundles) use one instance of this resource. For single-member bundles the scheduling is done at the logical layer. All members of multi-member bundles are scheduled at the physical layer, so each member link in a multi-member bundle uses one instance.

PPP multilink link max Command

Support for the ppp multilink link max command is new for the Cisco IOS 12.2(33)SB release, to distinguish between single and multimember MLP on LNS bundles. The default maximum number of links for Cisco 10000 series routers is 10. The ppp multilink link max 1 command is required for single-member bundles. Support for this command is limited and has the following restrictions:

•No support for the Frame Relay member link bundle.

•On the MLP over Serial interface, the ppp multilink link max 1 command restricts the number of links that join the bundle. The bundle continues to use Cobalt for the reassembly space.

•For single-VC and multi-VC ATM bundles, the command overrides the MLP interface range.

---

Note The ppp multilink link max command is supported only on the PRE3.

---

Performance and Scalability of MLP on LNS

The following commands allow for better scaling when used in configuring MLP on LNS:

•Configure the hold-queue command in interface configuration mode for the trunk interfaces in which the L2TP tunnel is negotiated. For example:

Router(config-if)# hold-queue 4096 in

•Configure the following commands and recommended values on the virtual template interface:

Router(config-if)# ppp max-configure 110

Router(config-if)# ppp max-failure 100

Router(config-if)# ppp timeout retry 5

Router(config-if)# keepalive 30

•Configure the lcp renegotiation always command on the VPDN group to renegotiate between LAC and LNS. The maximum number of multilink member links that can be configured on the Cisco 10000 series routers is up to 20440. Different combinations of bundle configurations can be configured on the box at any given time based on resource availability.

For more information, see the Scalability and Performance chapter in this guide.

PXF Memory and Performance Impact for MLP on LNS

PXF performance is measured as follows:

•Packet buffer usage

The number of packet buffers available on the PRE3 is 832K small buffers (for packet sizes of 768 bytes or less) and 120K large buffers (for packet sizes greater than 768 bytes). With full scaling of 12280 bundles (2040 multilink and 10240 single link), the average number of buffers is 69.4 small buffers and 10.0 large buffers per bundle for a total of 79.4 buffers per bundle.