Table Of Contents
Configuring Load Balancing and Dynamic Channel Change (DCC) on the Cisco CMTS
Contents
Prerequisites
Prerequisites for Load Balancing on the Cisco CMTS
Prerequisites for Dynamic Channel Change for Load Balancing
Restrictions
Restrictions for Load Balancing on the Cisco CMTS
Restrictions for Dynamic Channel Change for Load Balancing
DCC Restrictions with N+1 Redundancy and Inter-card Load Balancing
Information on the Load Balancing on the Cisco CMTS Feature
Feature Overview
Types of Load-Balancing Operations
Methods to Determine When Interfaces are Balanced
Using Both Static and Dynamic Load Balancing
Load Balancing Parameters
Load-Balance Groups
Downstream Load Balancing Distribution with Upstream Load Balancing
Interaction with Spectrum Management
Benefits
How to Configure Load Balancing on the Cisco CMTS
Creating a Load-Balance Group (required)
Configuring a Load-Balance Group (optional)
Assigning Interfaces to a Load-Balance Group (required)
Restrictions
Excluding Cable Modems from a Load-Balance Group (optional)
Distributing Downstream Load Balancing with Upstream Load Balancing
Verifying Load Balancing Operations (optional)
Configuration Examples for Load Balancing on the Cisco CMTS
Load-Balance Group Example (Static Load Balancing)
Load-Balance Group Example (Passive Load Balancing)
Load-Balance Group Example (Dynamic Load Balancing)
Interface Configuration Example
Sample Configuration for Upstreams and Downstreams
How to Configure Dynamic Channel Change for Load Balancing
Configuring DCC for Load Balancing on the Cisco CMTS
Testing Dynamic Channel Change for Load Balancing
Configuration Examples of Dynamic Channel Change (DCC) for Load Balancing
System Error Messages for Load Balancing
Additional References
Related Documents
Standards
MIBs
RFCs
Technical Assistance
Command Reference
Primary Load Balancing Commands
Dynamic Channel Change Commands for Load Balancing
cable load-balance group
cable load-balance group dcc-init-technique
cable load-balance group policy
cable load-balance group threshold
show controllers cable
test cable dcc
Glossary
Configuring Load Balancing and Dynamic Channel Change (DCC) on the Cisco CMTS
OL-3787-06
November 2006
The Load Balancing feature for the Cisco Cable Modem Termination System (CMTS) allows system operators to distribute cable modems across radio frequency (RF) downstream and upstream channels on the same cable interface line card. Load balancing maximizes bandwidth and usage of the cable plant.
Effective with Cisco IOS Release 12.3(17a)BC, and later 12.3 BC releases, load balancing is further enhanced and supported with Dynamic Channel Change (DCC). This document describes all implementations of load balancing on the Cisco CMTS, dependent upon the IOS release installed.
Feature Specifications for Load Balancing on the Cisco CMTS
Feature History
|
Release
|
Modification
|
Release 12.2(15)BC1
|
This feature was introduced for Cisco uBR7246VXR and Cisco uBR10012 routers.
|
Release 12.3(9a)BC
|
This feature was introduced for the Cisco uBR7100 Series universal broadband routers.
|
Release 12.3(17a)BC
|
Dynamic Channel Change (DCC) for Load Balancing supported on the Cisco uBR10012 and Cisco uBR7246VXR routers.
|
Release 12.3(17b)BC4
|
Downstream load balancing further enhanced to equalize downstream load balancing with upstream load balancing group members. This enhancement synchronizes the pending statistic between different cable interface line cards in the load balancing group. Refer to the "Downstream Load Balancing Distribution with Upstream Load Balancing" section.
|
Supported Platforms
|
Cisco uBR7100 Series, Cisco uBR7246VXR, Cisco uBR10012 universal broadband routers.
|
Finding Support Information for Platforms and Cisco IOS Software Images
Use Cisco Feature Navigator to find information about platform support and Cisco IOS software image support. Access Cisco Feature Navigator at http://www.cisco.com/go/fn. You must have an account on Cisco.com. If you do not have an account or have forgotten your username or password, click Cancel at the login dialog box and follow the instructions that appear.
Contents
•
Prerequisites
•Restrictions
•Information on the Load Balancing on the Cisco CMTS Feature
•How to Configure Load Balancing on the Cisco CMTS
•Configuration Examples for Load Balancing on the Cisco CMTS
•How to Configure Dynamic Channel Change for Load Balancing
•Configuration Examples of Dynamic Channel Change (DCC) for Load Balancing
•System Error Messages for Load Balancing
•Additional References
•Command Reference
•Glossary
Prerequisites
Prerequisites for Load Balancing on the Cisco CMTS
The Load Balancing on the Cisco CMTS feature has the following prerequisites:
•The Cisco uBR7200 series and Cisco uBR10012 router must be running Cisco IOS Release 12.2(15)BC1 or later Cisco IOS Release 12.2 BC release.
•Load balancing can be done only on upstreams and downstreams that share physical connectivity with the same group of cable modems.
•When performing load balancing among downstreams, you must also configure the known downstream center frequency to be used on each downstream interface, using the cable downstream frequency command. (This is an informational-only configuration on cable interfaces that use an external upconverter, but it is still required for load balancing so that the Cisco CMTS knows what frequencies it should use when moving cable modems from one downstream to another.)
Prerequisites for Dynamic Channel Change for Load Balancing
•DCC can be done only to a cable modem that is physically connected to both source and target upstream or downstream channels, or both.
•Upstreams and downstream channels that share the same physical connectivity must have different center frequencies separated by channel-width.
•The difference between the physical layer parameters on the source and target DCC channels must be within the threshold required by the desired DCC initialization technique.
•DOCSIS1.1 must be enabled for a modem to behave properly for the DCC operation. Note that not all DOCSIS1.1 certified modems are DCC capable as the CableLabs DCC ATP tests need enhancement for complete coverage.
Restrictions
Restrictions for Load Balancing on the Cisco CMTS
The Load Balancing on the Cisco CMTS feature has the following restrictions:
•For additional information about support of Load Balancing on the Cisco CMTS, refer to Feature Specifications for Load Balancing on the Cisco CMTS.
•Load balancing can be done only on a per chassis basis—all interfaces in a load-balance group must be provided by the same chassis.
•A downstream or upstream can belong to only one load-balance group.
•All downstreams and upstreams in a load-balance group must share physical connectivity to the same group of cable modems. Downstreams can be in a separate load-balance group than upstreams, but all downstreams or all upstreams that have the same RF physical connectivity must be members of the same load-balance group.
•You can configure only one load-balance group per shared physical domain (upstream or interface). You cannot configure multiple load-balance groups to distribute downstreams or upstreams that share physical connectivity.
•In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis (the older limitation was 20). However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces, as long as they are in different domains. If downstream channels are not included in a load-balance group, then each downstream channel can be considered a separate domain.
•If an upstream port is operational, using the no shutdown command, and is not being used and not connected, load balancing attempts to use the port even though there are no cable modems registered on that port. When the upstream port is up, it is put into INIT state and load balancing includes this port as a potential target. However, if the load balancing sees multiple failures moving to this upstream, it is set to DISABLE state and the port is avoided later on in load balancing processes.
•The load-balancing algorithms assume a relatively even distribution of usage among modems. In the situation where one cable modem creates the bulk of the load on an interface, the load-balancing thresholds should be configured for a value above the load created by that single modem.
•Load balancing is done on cable modems in real-time, using current load-usage statistics. You cannot perform load balancing according to the time-of-day or using a schedule.
•You cannot select particular cable modems to be automatically moved for load balancing, although you can exclude cable modems from load balancing operations altogether on the basis of their MAC address or organization unique identifier (OUI). (You can use the test cable load-balance command to manually move a particular cable modem among upstreams, but this is done typically to test the configuration of the load balance groups.)
•If you have configured upstream shared spectrum groups while doing downstream load balancing, the downstream in each MAC domain must not use overlapping upstream groups. For example, the downstream in one MAC domain could use an upstream spectrum band of 10 to 30 MHz, while the downstream in a second MAC domain could use an upstream spectrum band of 30 to 42 MHz. Each MAC domain has its own upstream shared spectrum group, allowing the load-balance group to contain the downstreams for both MAC domains.
Note A MAC domain is one downstream and its associated upstreams.
•All upstream ports coming from the same splitter must be using different center frequencies that are separated by the channel width. For example, if the upstreams are using a channel width of 3.2 MHz, the center frequencies for all upstreams must be separated by at least 3.2 MHz.
•In Cisco IOS Release 12.2(15)BC1, the dynamic load balancing method uses the Downstream Frequency Override message to move cable modems between downstream channels, which results in cable modems going offline and having to reregister, resulting in a short, temporary loss of connectivity for the customer. This is because the DOCSIS 1.0 specification requires cable modems to reregister whenever the downstream is changed using the Downstream Frequency Override message. Cable modems should not go offline when they are moved between upstreams. This behavior is modified in Cisco IOS Release 12.3(17a)BC, with the introduction of four initialization techniques for Dynamic Channel Change (DCC). See the "Configuring DCC for Load Balancing on the Cisco CMTS" section.
•As required by cable interface bundling, all interfaces in a load-balance group must also be in the same HCCP interface bundle.
•If you have configured load balancing, the provisioning system must not assign specific upstream channels or downstream frequencies to individual cable modems in their DOCSIS configuration files. Any cable modems require specific upstream channels or downstream frequencies must be excluded from load-balancing operations (using the cable load-balance exclude command).
•Do not use the utilization method of load balancing on cable interfaces that have a small number of cable modems and where a single modem is responsible for the majority of the interface load. In this condition, the Cisco CMTS could end up continually moving cable modems from one interface to another in an endless attempt to load balance the interfaces. To avoid this, configure the utilization threshold to a value that is higher than what can be caused by any single cable modem.
•In Cisco IOS Release 12.2(15)BC1, you should not configure an interface for both dynamic load balancing and Hot-Standby Connection-to-Connection (HCCP) N+1 redundancy, because cable modems will go offline after a switchover. You can configure the interface for HCCP N+1 redundancy when you are using only static and passive load balancing.
Load balancing, however, does not continue after a switchover from a Working to a Protect interface. Load balancing resumes when the Cisco CMTS switches back to the Working interface. (One possible workaround is to pre-configure the Protect interface with the appropriate load-balancing commands, but you must be certain that the downstreams and upstreams in each load-balance group after the switchover have the same physical connectivity.)
Restrictions for Dynamic Channel Change for Load Balancing
•DCC initialization 0 is the default technique for load balancing DCC. Legacy line cards can only use DCC initialization technique 0.
•DCC initialization techniques 1 to 4 are strictly for downstream channel changes within the same cable interface line card, or intra-card, implementation, and cannot be used for Load Balancing between multiple cable interface line cards in inter-card implementation.
•For Load Balancing between multiple cable interface line cards (inter-card implementation), DCC initialization technique 0 is to be used in all cases, regardless of what technique is set for the load balancing group or which cable interface line card types are used.
•The source and target upstreams and downstreams must share physical connectivity with the modem desired for a DCC transaction.
•Independent downstream change is not supported, and cross-MAC domain upstream changes must occur with the associated downstream changes.
•The source and target downstream interfaces must belong to the same virtual bundle, and the same load balancing group if DCC is used for load balancing.
•For DCC initialization techniques 1 to 4, all the configuration variables of the cable modem must remain constant with the exception of the configuration variables which are explicitly changed by the DCC-REQ messages encodings.
•DCC initialization techniques 2 to 4 must not be used if the propagation delay differences between the old and new channels exceeds the ranging accuracy requirement defined in DOCSIS, i.e. ±0.25 usec plus ± ½ symbol time.
For example, for a symbol rate of 1.28Msps, the timing offset difference between the source and target upstream channel is ± floor[(0.250 us + 0.5*0.781us)/(1/10.24)] = ± 6.
•The attenuation or frequency response differences between the old and new upstream channels causes the received power at the CMTS to change by more than 6dB.
•DCC initialization technique 3 must not be used if the conditions for using technique 2 is not met.
•DCC initialization technique 4 must not be used if the conditions for using technique 2 cannot be met.
•Micro-reflections on the new upstream channel result in an unacceptable BER (greater than 1e-8) with pre-equalization coefficients set to the initial setting.
•DCC is only used for dynamic downstream load balancing on DOCSIS1.1 modems. UCC is used for dynamic upstream load balancing. DCC is not used for static load balancing.
•Prolonged interruption of the multicast traffic is expected if the cable modem moved by DCC is the first one in a dynamic multicast group on the target interface. The downstream multicast service flow cannot be reestablished until the CMTS receives an IGMP join message from the CPE as the result of the CMTS IGMP query, where the IGMP query interval is set to 1 minute. This is a IGMPv2 limitation.
DCC Restrictions with N+1 Redundancy and Inter-card Load Balancing
•Inter-card Load Balancing is not supported with cable interface line cards using N+1 Redundancy. Refer to general DCC restrictions for additional information.
•Dynamic load balancing should not be used together with N+1 redundancy. Modems with outstanding DCC transactions go offline after a switchover event.
Note When cable modems go offline during switchover event, this prompts the load balancing feature to engage. Cable modems move in relation to the switchover event. When the cable modems return online, load balancing may need to initiate again.
To facilitate load balancing during a switchover, you may increase the dynamic load balance threshold, in the case that the system has a given percentage of cable modems that reset during switchover. An alternate method is to use static load balancing with N+1 Redundancy. Refer to the "Types of Load-Balancing Operations" section for additional information.
Information on the Load Balancing on the Cisco CMTS Feature
This section describes the operation, concepts, and benefits of the Load Balancing on the Cisco CMTS feature.
•Feature Overview
•Benefits
Feature Overview
This Load Balancing on the Cisco CMTS feature allows service providers to optimally use both downstream and upstream bandwidth, enabling the deployment of new, high-speed services such as voice and video services. This feature also can help reduce network congestion due to the uneven distribution of cable modems across the cable network and due to different usage patterns of individual customers.
By default, the Cisco CMTS platforms use a form of load balancing that attempts to equally distribute the cable modems to different upstreams when the cable modems register. You can refine this form of load balancing by imposing a limit on the number of cable modems that can register on any particular upstream, using the cable upstream admission-control command.
However, this default form of load balancing affects the cable modems only when they initially register with the Cisco CMTS. It does not dynamically rebalance the cable modems at later times, such as when they might change upstream channels in response to RF noise problems, or when bandwidth conditions change rapidly because of real-time traffic such as Voice over IP (VoIP) and video services. It also does not affect how the cable modems are distributed among downstream channels.
Types of Load-Balancing Operations
The Load Balancing on the Cisco CMTS feature provides a more comprehensive load-balancing solution by adding new forms of registration-based and dynamic load balancing. In Cisco IOS Release 12.2(15)BC1, the Load Balancing on the Cisco CMTS feature supports the following configurable types of load balancing:
•Static Load Balancing—This is a form of registration-based load balancing that is done at the time a cable modem registers. When a cable modem sends its registration request (REG-REQ) and ranging request (RNG-REQ) messages, the Cisco CMTS responds with a ranging response (RNG-RSP) message that includes either a Downstream Frequency Override or an Upstream Channel ID Override field that instructs the cable modem which channels it should use.
•Passive Load Balancing—This is a form of registration-based load balancing that can be configured for individual cable modems. In this mode, the Cisco CMTS does not need to send any type of messaging to the modem. The Cisco CMTS ignores the RNG-REQ message from a cable modem that is attempting to register using a downstream or upstream that is currently overloaded. The cable modem repeats its registration request until it reaches a channel that can accept additional modems.
Note By default, the Cisco CMTS uses static load balancing, but passive load balancing can be specified for individual older cable modems (using the cable load-balance exclude command) that do not respond well to the static form. This method should be used only as needed because when used for a large number of modems, it could generate a large volume of ranging retry messages.
•Dynamic load balancing—This is a form of load balancing in which cable modems are moved among upstreams and downstreams after their initial registration and they come online, while potentially passing traffic. Cable modems that are currently online are moved when the load difference between two interfaces exceeds a user-defined percentage.
Note The dynamic form of load balancing could be considered a form of traffic-based load balancing, in that cable modems could be moved between interfaces while they are passing traffic. However, the load-balancing algorithms do not take into account the nature of traffic when considering which cable modems should be moved.
When using dynamic load balancing and an upstream channel is overloaded, the Cisco CMTS sends an Upstream Channel Change (UCC) request to a cable modem to instruct it to move to another upstream. The cable modem should move to the new upstream channel, without going offline or having to reregister with the CMTS.
When using dynamic load balancing and a downstream channel is overloaded, the Cisco CMTS sends an abort response to a cable modem's ranging request (RNG-REQ) message. When the cable modem sends new REG-REQ and RNG-REQ messages, the Cisco CMTS specifies the new downstream channel in the Downstream Frequency Override field in its RNG-RSP message. The cable modem must go offline and reregister on the new downstream channel, so as to conform to the DOCSIS 1.0 specifications.
Note In Cisco IOS Release 12.2(15)BC1, the dynamic load balancing method results in cable modems going offline and having to reregister whenever the modems are moved between downstreams. This is because the DOCSIS 1.0 specification requires cable modems to reregister whenever the downstream is changed using the Downstream Frequency Override message. Cable modems should not go offline when being moved between upstreams.
In all cases, the load balancing is done by moving cable modems from the interface with the higher load to an interface with a lower load. For dynamic load balancing, the Cisco CMTS determines which online cable modems should be moved in a round robin fashion. For static and passive load balancing, the Cisco CMTS moves cable modems only when they register or reregister.
Methods to Determine When Interfaces are Balanced
In addition to selecting how interfaces should be balancing (using the static, passive, or dynamic types of load balancing), you can also select one of the following methods that the Cisco CMTS should use to determine when interfaces are balanced:
•Modem Method—Uses the number of active cable modems on an interface.
•Service Flow Method—Uses the number of active Service Flow IDs (SFIDs) on an interface.
•Utilization Method—Uses an interface's current percentage of utilization.
See the following sections for more information about each method.
Modem Method
The modem method of load-balancing uses the number of active cable modems on an interface to determine the current load. This is a form of distribution-based load balancing, in which the absolute numbers of modems are used to determine whether interfaces are load balanced.
This method does not take into account the amount of traffic flowing through the cable modems, but the system does take into account the relative bandwidth of the channels being used, so that channels with higher bandwidths are allocated higher numbers of cable modems. This means that when interfaces are using different channel widths or modulation profiles, the system can assign different numbers of cable modems to the interfaces to achieve a balanced load. For example:
•Channel widths— If two upstreams are being load balanced, and one upstream is configured with a channel width of 1.6 MHz and the other upstream is configured for a channel width of 3.2 MHz, the Cisco CMTS allocates twice as many cable modems to the second upstream, because its channel width is twice as large as the first upstream's channel width.
•Modulation profiles— If one downstream is configured for 64-QAM and the other downstream is configured for 256-QAM, the Cisco CMTS allocates a proportionately larger number of cable modems to the second downstream so as to achieve a balanced load.
When both the channel width and different modulation profile are set differently on two interfaces, the system calculates a "weight" value to use as a guide to determine the relative bandwidths of the interfaces.
Tip In a system with balanced loads, the interfaces will contain the same number of cable modems only when the interfaces are configured with the same channel width and modulation parameters.
Service Flow Method
The service flow method of load balancing uses the number of active service flow IDs (SFIDs) on an interface to determine the current load. This is a form of distribution-based load balancing, in which the absolute numbers of service flows are used to determine whether interfaces are load balanced.
This method does not take into account the amount of traffic flowing on each SFID, but the system does take into account the relative bandwidth of the channels being used, so that channels with higher bandwidths are allocated higher numbers of SFIDs. This means that when interfaces are using different channel widths or modulation profiles, the system can assign different numbers of SFIDs to the interfaces to achieve a balanced load. For example:
•Channel widths— For example, if two upstreams are being load balanced, and one upstream is configured with a channel width of 1.6 MHz and the other upstream is configured for a channel width of 3.2 MHz, the Cisco CMTS allocates twice as many SFIDs to the second upstream, because its channel width is twice as large as the first upstream's channel width.
•Modulation profiles— For example, if one downstream is configured for 64-QAM and the other downstream is configured for 256-QAM, the Cisco CMTS allocates a proportionately larger number of SFIDs to the second downstream so as to achieve a balanced load.
When both the channel width and different modulation profile are set differently on two interfaces, the system calculates a "weight" value to use as a guide to determine the relative bandwidths of the interfaces.
Tip In a system with balanced loads, the interfaces will contain the same number of SFIDs only when the interfaces are configured with the same channel width and modulation parameters.
Utilization Method
The utilization method uses an interface's current percentage of utilization to determine the current load. This method uses the amount of traffic being sent over an interface, in the form of the percentage of total bandwidth being used. The system takes into account the relative throughput and bandwidth (as determined by the modulation profiles and channel widths) of each interface when evaluating the load on those interfaces.
For example, if two upstreams are being load-balanced using the utilization method, and the first upstream has twice the bandwidth of the second upstream, the two upstreams are considered balanced when they reach the same percentage of utilization. The first upstream is carrying more traffic than the second upstream because it has a larger capacity for traffic, but the percentage of utilization will be the same.
Note The utilization method does not go into effect until the absolute utilization of the interfaces is at least 25 percent or greater. If the utilization of an interface is less than 25 percent, the Cisco CMTS does not attempt to load balance the interfaces, regardless of the difference in usage levels between the interfaces.
Note In Cisco IOS Release 12.2(15)BC2, the average utilization figures for an upstream were reset to zero whenever the upstream configuration was changed (such as changing the modulation profile or channel width). This reset of the average utilization could skew the load balancing algorithm and cause unnecessary moving of cable modems. This is no longer the case in Cisco IOS Release 12.3(9a)BC and later releases, because the average utilization figure is reset only when the upstream is shut down, allowing load balancing operation to be more accurate.
Using Both Static and Dynamic Load Balancing
Dynamic load balancing can be used together with static load balancing. The user-configured threshold for dynamic load balancing must be equal to or larger than the user-configured threshold for static load balancing.
With this configuration, when a load imbalance occurs, the system initially uses static load balancing, moving cable modems among interfaces when the modems register. If the load imbalance continues to grow and eventually passes the dynamic threshold, the system begins moving cable modems using dynamic load balancing. Then, when enough cable modems have been moved to push the imbalance below the dynamic threshold, the system reverts to static load balancing until the load imbalance falls below the static threshold value.
Load Balancing Parameters
The Load Balancing on the Cisco CMTS feature supports static, passive, and dynamic load balancing on both upstream and downstream channels. You can configure downstreams and upstreams to use the same load-balancing parameters, or you can configure upstreams and downstreams separately.
You can determine which cable interfaces should participate in load-balancing operations. You can also choose which of the following methods should be used to determine the current load on a cable interface, and therefore determine whether cable modems should be moved:
•Number of active cable modems
•Number of active service flows
•Channel bandwidth utilization
You can also specify the threshold values that the Cisco CMTS should use to determine how to assign new cable modems to upstreams and downstreams for both types of load balancing. You can also configure whether cable modems with active Voice-over-IP (VoIP) calls should be moved, and if so, what thresholds should be used. You can also exclude certain cable modems from one or all of the different forms of load balancing.
Load-Balance Groups
To enable the Load Balancing on the Cisco CMTS feature, you first must create and configure a load-balance group, which specifies how load balancing should be performed. You then must assign cable interfaces to the load-balance group, at which point the Cisco CMTS begins performing load balancing on those cable interfaces.
You can use separate load-balance groups for upstreams or downstreams, or you can use the same load-balance group for both upstreams and downstreams. However, all cable interfaces in a load-balance group must share the same physical RF connectivity.
Note In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis (the older limitation was 20). However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces. If downstreams are not included in a load-balance group, then each downstream can be considered a separate domain.
Also, the same load-balance group must be used for all downstreams or upstreams that share RF connectivity and that are participating in load balancing. You cannot distribute downstreams or upstreams that share physical connectivity across multiple load-balance groups.
If you assign downstreams and upstreams to different load-balance groups, the Cisco CMTS performs load balancing independently on the upstreams and downstreams. If both downstreams and upstreams are assigned to the same load-balance group, the Cisco CMTS attempts to balance both the downstream and upstream load.
Figure 1 shows a simple example of how load-balance groups can be created.
Figure 1 Example of Load -Balance Groups
As shown in this figure, three load-balance groups are being used:
•All four upstreams for downstream C5/0 (U0—U3) and the first two upstreams (U0 and U1) for downstream C5/1 are used for the same node and are therefore part of the same load-balance group.
•The last two upstreams for downstream C5/1 (U1 and U2) are used for a different node and are therefore part of a separate load-balance group.
•The two downstreams, C5/0 and C5/1, are part of the same load-balance group, and this group is separate from the groups being used for the upstreams. (However, these downstreams could also be combined with one of the upstream load-balance groups.)
Note To see a sample configuration for the configuration that is shown in Figure 1, see the "Sample Configuration for Upstreams and Downstreams" section.
Downstream Load Balancing Distribution with Upstream Load Balancing
Cisco IOS Release 12.3(17b)BC4 introduces further enhancements to downstream load balancing, resulting in equalized load balancing with upstream group members. This enhancement synchronizes the pending statistic between different cable interface line cards in the load balancing group. The result is an alternative downstream load balancing scheme that makes use of per-upstream loads rather than total downstream loads.
This enhancement performs downstream load balancing that accounts upstream channel loads in the same upstream load balancing group, rather than on the basis of the entire downstream channel load. Prior Cisco IOS releases may not have distributed cable modems evenly over individual upstream channels, nor in a way that accounted for downstream and upstream together.
This enhancement applies when downstream load balancing occurs on a headend system with separate upstream load balancing segments; the upstream segments are spread over multiple downstreams segments.
Cisco IOS Release 12.3(17b)BC4 enables the configuration and operation of making downstream load balancing decisions as follows:
•The target downstream segment is in the same downstream load balancing group as the source downstream segment.
•The upstream load balancing group can be set for the corresponding channel on which a cable modem is balanced.
•The Cisco CMTS automatically locates the upstream segment for a load balancing group and processes the upstream group status on the source interface that has the lowest load.
•The target downstream segment must have an upstream channel set in the upstream load balancing group.
•The highest target upstream segment must carry less load than any other potential target —the highest upstream segment on other interfaces.
This functionality is enabled with the following command:
cable load-balance group ds-lb-group-id policy us-groups-across-ds
For example, several upstream segments can be configured across multiple downstream segments as follows:
3/0 LB10 LB11 LB12 LB13 LB1
4/0 LB10 LB11 LB12 LB13 LB1
5/0 LB10 LB11 LB12 LB13 LB1
6/0 LB10 LB11 LB12 LB13 LB1
In this example, a cable modem that comes online on the interface cable 5/0 Upstream 2 could potentially come online on the following interfaces:
•cable 3/0 upstream 2
•cable 4/0 upstream 2
•cable 6/0 upstream 2
•nowhere else, however
With downstream load balancing prior to Cisco IOS Release 12.3(17b)BC4, having 100 cable modems per segment would be possible in an extreme case that distributes cable modems as follows:
The enhancement enables the following advantages and behaviors:
•This enhancement adds support for synchronizing the pending statistic between different cable interface line cards and the network processing engine (NPE) so that a better decision can be made about where cable modems should be moved. This function can be used as a normal downstream load balancing implementation, if desired.
•This enhancement adds the us-groups-across-ds policy variable setting for configuring downstream load balancing groups.
•This enhancement adds support for the output of the show cable load-balance command.
For additional information, refer to the "Distributing Downstream Load Balancing with Upstream Load Balancing" section.
Interaction with Spectrum Management
Cisco cable interface line cards support a number of features to maximize channel bandwidth and to minimize the impact of ingress noise on cable modem traffic. These features have the following impacts upon load-balancing operations:
•Frequency hopping—Frequency hopping does not affect the load balancing algorithm, because it does not change either the bandwidth of a channel nor the number of cable modems on an interface.
•Dynamic modulation changes—The dynamic modulation feature affects the load-balancing algorithm because it typically switches an interface from a higher-bandwidth modulation profile to a lower-bandwidth modulation profile in response to noise conditions on the interface.
For example, if an upstream is configured for 16-QAM modulation, sufficient noise levels could switch the upstream to a QPSK modulation profile. Depending on the load-balancing configuration, this could then result in the movement of cable modems to other channels. Similarly, when the noise conditions improve, and the modulation is returned to the original, higher-bandwidth profile, the cable modems could be moved again to rebalance the upstream channels.
•Channel width changes—The Cisco uBR-MC16S cable interface line card supports automatic changes to the channel width in response to noise conditions. Because changing the channel width affects the throughput of a channel, this also affects the load-balancing algorithm.
For example, if noise makes the current channel width unusable, the Cisco uBR-MC16S card reduces the channel width until it finds a usable channel width. Because this reduces the available bandwidth on the channel, the load-balancing algorithm moves cable modems to rebalance the upstreams.
In addition, the Cisco uBR-MC16S card does not automatically restore the original channel width when noise conditions improve. Instead, the card changes the channel width only when it performs a subsequent frequency hop, either in response to additional noise conditions or when an operator performs a manual frequency hop. When the hop occurs, the card then searches for the largest possible channel width, and this could result in another movement of cable modems to rebalance the channels.
Benefits
The Load Balancing on the Cisco CMTS feature provides the following benefits to cable service providers and their partners and customers:
•Provides a method that service providers can use for efficient bandwidth utilization, especially when using multiple upstream channels per fiber node.
•Allows service providers to expand their networks in an efficient manner, avoiding the cost of having to install additional fiber optic equipment and further segmenting the physical plant.
•Load balancing on downstreams enables efficient bandwidth usage when using multiple downstream channels per fiber mode to enable Video over IP and other services that require high-bandwidth real-time streams.
•Load balancing of upstreams and downstreams does not require any change to the provisioning servers or to any DOCSIS configuration files.
•Load balancing of upstreams and downstreams does not require any administrator or user intervention (such as manually resetting cable interfaces or manually rebooting cable modems).
•Load balancing can be used with the virtual interfaces feature on the Cisco uBR-MC5X20S/U/T/H cable interface line cards to provide load balancing for configurable MAC domains.
•Allows service providers to equally balance their downstreams as cable modems register, so that cable modems do not all attempt to register on the same downstream, resulting in many cable modems failing to register and having to search for a new downstream.
•Cable modems can be moved among downstream and upstream channels without having to manually change any network parameters such as IP address.
•Allows service providers to stay ahead of customers' bandwidth demands by dynamically responding to current load-usage conditions.
•Allows service providers to optimize the load-balancing parameters for critical services, such as Voice over IP (VoIP).
How to Configure Load Balancing on the Cisco CMTS
The following sections describe how to create and configure load-balance groups, to enable load balancing on the Cisco CMTS. Each task is marked as required or optional, as appropriate.
•Creating a Load-Balance Group (required)
•Configuring a Load-Balance Group (optional)
•Assigning Interfaces to a Load-Balance Group (required)
•Excluding Cable Modems from a Load-Balance Group (optional)
•Distributing Downstream Load Balancing with Upstream Load Balancing
•Verifying Load Balancing Operations (optional)
Creating a Load-Balance Group (required)
This section describes how to create a load-balance group. You must create at least one load-balance group before the Cisco CMTS will begin load balancing cable modems.
SUMMARY STEPS
1. enable
2. configure terminal
3. cable load-balance group n [method modem | method service-flows | method utilization]
4. exit
DETAILED STEPS
| |
Command or Action
|
Purpose
|
Step 1
|
enable
Example:
Router> enable
Router#
|
Enables privileged EXEC mode. Enter your password if prompted.
|
Step 2
|
configure terminal
Example:
Router# configure terminal
Router(config)#
|
Enters global configuration mode.
|
Step 3
|
cable load-balance group n [method modem |
method service-flows | method utilization]
Example:
Router(config)# cable load-balance group 10
method service-flows enforce
Router(config)# cable load-balance group 11
method modem
Router(config)# cable load-balance group 12
method utilization enforce
Router(config)#
|
Creates a load-balance group with the following parameters:
•n = Number of the load-balance group. Valid range is 1 to 80, with no default.
Note In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis. The older limitation was 20. However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces, as long as they are in different domains. If downstream channels are not included in a load-balance group, then each downstream channel can be considered a separate domain.
•method modem = (Optional) Specifies that the load-balance group should use the number of active cable modems on an interface to determine the current load (default).
•method service-flows = (Optional) Specifies that the load-balance group should use the number of active service flow IDs (SFIDs) on an interface to determine the current load.
•method utilization = (Optional) Specifies that the load-balance group should use an interface's current percentage of utilization to determine the current load. (To avoid unnecessary movement of cable modems, the utilization method does not perform load balancing until the amount of utilization on an interface is at 25 percent or more.)
|
Step 4
|
exit
Example:
Router(config)# exit
Router#
|
Exits global configuration mode.
|
Configuring a Load-Balance Group (optional)
This section describes how to configure a load-balance group. All steps are optional, unless you want to change the default load-balance configuration.
SUMMARY STEPS
1. enable
2. configure terminal
3. cable load-balance group n [interval seconds]
4. cable load-balance group n threshold {load load-value [enforce threshold] | load minimum number | stability percent | ugs band-value}
5. cable load-balance group n policy ugs
6. exit
DETAILED STEPS
| |
Command or Action
|
Purpose
|
Step 1
|
enable
Example:
Router> enable
Router#
|
Enables privileged EXEC mode. Enter your password if prompted.
|
Step 2
|
configure terminal
Example:
Router# configure terminal
Router(config)#
|
Enters global configuration mode.
|
Step 3
|
cable load-balance group n [interval seconds]
Example:
Router(config)# cable load-balance group 10
interval 30
Router(config)#
|
Modify how often the Cisco CMTS checks each interface to determine the current load.
•n = Number of the load-balance group. Valid range is 1 to 80, with no default.
Note In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis. The older limitation was 20. However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces, as long as they are in different domains. If downstream channels are not included in a load-balance group, then each downstream channel can be considered a separate domain.
•seconds = Minimum time between when cable modems can be moved to load balance the interfaces. One cable modem at most is moved during each interval time period. The valid range is 0 to 3600 seconds, with a default value of 10 seconds.
|
Step 4
|
cable load-balance group n threshold {load
load-value [enforce threshold] | load minimum
number | stability percent | ugs band-value}
Example:
Router(config)# cable load-balance group 10
threshold load 20 enforce 30
Router(config)# cable load-balance group 10
threshold ugs 60
|
Specifies the thresholds to be used to determine when cable modems should be moved to achieve the desired load balancing.
•load load-value=Specifies the maximum load difference that can exist between interfaces in a group before the Cisco CMTS performs load balancing. The valid range for load-value is 1 to 100 percent, with a default of 10 percent.
Note The default of 10 percent is the minimum recommended threshold. Do not set this threshold below 10 percent unless you have been instructed to do so by Cisco TAC.
•enforce threshold = Enables dynamic load balancing, which moves online cable modems. The range for the threshold parameter starts from the current value of the load-value parameter up to 100 percent. The default equals the current value of the load-value parameter.
•load minimum number = Specifies that cable modems should be moved only if the load between the two interfaces is greater than the specified number of cable modems or service flows (valid only when the method being used is the number of modems or service flows; it is not used for the utilization method).
•stability percent = Specifies the minimum allowable percentage of good periodic ranging requests that is acceptable. When the channel has a lower percent of modems responding to the ranging requests in a one minute period, the Cisco CMTS begins moving modems. The valid range is 1 to 100 percent, with a default of 50 percent.
•ugs band-value= Specifies that the Cisco CMTS should move cable modems with active UGS service flows when the current UGS usage reaches the band-value percentage. The valid range for band-value is 0 to 100 percent, with a default of 70 percent.
|
Step 5
|
cable load-balance group n policy ugs
Example:
Router(config)# cable load-balance group 10
policy ugs
Router(config)#
|
Allows the Cisco CMTS to move cable modems that have active UGS service flows to enforce the load balancing policy.
•n = Number of the load-balance group. Valid range is 1 to 80, with no default.
Note In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis. The older limitation was 20. However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces, as long as they are in different domains. If downstream channels are not included in a load-balance group, then each downstream channel can be considered a separate domain.
|
Step 6
|
exit
Example:
Router(config)# exit
Router#
|
Exits global configuration mode.
|
Note The load-balancing algorithms assume a relatively even distribution of usage among modems. In the situation where one cable modem creates the bulk of the load on an interface, the load-balancing thresholds should be configured for a value above the load created by that single modem. You should check for this situation whenever the load-balancing algorithm is moving a large number of modems from one interface to another.
Assigning Interfaces to a Load-Balance Group (required)
This section describes how to assign cable interfaces (both downstreams and upstreams) to a load-balance group. A cable interface does not participate in load-balancing operations until it is a member of a load-balance group.
Restrictions
When assigning cable interfaces to load balance groups, be aware of the following restrictions:
•A downstream or upstream can belong to only one load-balance group.
•All downstreams and upstreams in a load-balance group must share physical connectivity to the same group of cable modems. Downstreams can be in a separate load-balance group than upstreams, but all downstreams or all upstreams that have the same RF physical connectivity must be members of the same load-balance group. You cannot distribute downstreams or upstreams that share physical connectivity across multiple load-balance groups.
•All interfaces in a load-balance group use the same load-balancing parameters. By default, all cable modems on those interfaces are included in load-balancing operations. However, you can exclude one or more particular cable modems from being moved in load-balancing operations (see the "Excluding Cable Modems from a Load-Balance Group (optional)" section).
SUMMARY STEPS
1. enable
2. configure terminal
3. interface cable x/y
4. cable load-balance group n
5. cable downstream frequency freq-hz
6. cable upstream uport load-balance group n
7. exit
8. exit
DETAILED STEPS
| |
Command or Action
|
Purpose
|
Step 1
|
enable
Example:
Router> enable
Router#
|
Enables privileged EXEC mode. Enter your password if prompted.
|
Step 2
|
configure terminal
Example:
Router# configure terminal
Router(config)#
|
Enters global configuration mode.
|
Step 3
|
interface cable x/y
Example:
Router(config)# interface cable 5/1
Router(config-if)#
|
Enters interface configuration mode for the specified cable interface.
|
Step 4
|
cable load-balance group n
Example:
Router(config-if)# cable load-balance group 10
Router(config-if)#
|
Assigns the downstream interface to the specified load-balance group.
•n = Number of the load-balance group. Valid range is 1 to 80, with no default.
Note In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis. The older limitation was 20. However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces, as long as they are in different domains. If downstream channels are not included in a load-balance group, then each downstream channel can be considered a separate domain.
|
Step 5
|
cable downstream frequency freq-hz
Example:
Router(config-if)# cable downstream frequency
453000000
Router(config-if)#
|
Specifies the known downstream center frequency to be used on this cable interface. This is an informational-only configuration on cable interfaces that use an external upconverter, but it is still required for load balancing so that the Cisco CMTS knows what frequencies it should use when moving cable modems from one downstream to another.
The freq-hz parameter specifies the frequency in Hz, with a valid range of 54,000,000 to 858,000,000. Depending on the channel width, the range of center frequency that is acceptable to a CM is 91,000,000 to 857,000,000 Hz.
|
Step 6
|
cable upstream uport load-balance group n
Example:
Router(config-if)# cable upstream 0
load-balance group 10
Router(config-if)#
|
Assigns an upstream port to the specified load-balance group.
•uport = Upstream port number. Valid range starts with 0 and ends with a value that depends on the number of upstream ports on the cable interface line card.
•n = Number of the load-balance group. Valid range is 1 to 80, with no default.
Note In later Cisco IOS Releases, such as release 12.3(17a)BC, you can create a maximum of 80 load-balance groups on each chassis. The older limitation was 20. However, in prior Cisco IOS releases, you can reuse those load-balance groups on different sets of cable interfaces, as long as they are in different domains. If downstream channels are not included in a load-balance group, then each downstream channel can be considered a separate domain.
|
| |
Note Repeat Step 3 through Step 6 as needed for each downstream cable interface and its upstream ports that should belong to this group.
|
Step 7
|
exit
Example:
Router(config-if)# exit
Router(config)#
|
Exits interface configuration mode.
|
Step 8
|
exit
Example:
Router(config)# exit
Router#
|
Exits global configuration mode.
|
Excluding Cable Modems from a Load-Balance Group (optional)
This section describes how to exclude a particular cable modem, or all cable modems from a particular vendor, from participating in static or dynamic load-balancing operations, and optionally marking the modems for passive load balancing. This task is optional, because, by default, cable modems on an interface participate in whatever load-balancing operations have been configured.
Note This step might be required for some cable modems that are not DOCSIS-compliant. Such cable modems can go offline for long periods of time when load balancing is attempted using DOCSIS MAC messages. If this is the case, use this command to exclude such cable modems from load-balancing operations until the modem can be upgraded to DOCSIS-compliant software.
Tip You must exclude cable modems that require specific upstream channels or downstream frequencies. Load balancing cannot be done when cable modems are assigned specific channels or frequencies in their DOCSIS configuration files.
SUMMARY STEPS
1. enable
2. configure terminal
3. cable load-balance exclude {modem mac-address | oui oui-value} [enforce | static | strict]
4. exit
DETAILED STEPS
| |
Command or Action
|
Purpose
|
Step 1
|
enable
Example:
Router> enable
Router#
|
Enables privileged EXEC mode. Enter your password if prompted.
|
Step 2
|
configure terminal
Example:
Router# configure terminal
Router(config)#
|
Enters global configuration mode.
|
Step 3
|
cable load-balance exclude {modem mac-address |
oui oui-value} [enforce | static | strict]
Example:
Router(config)# cable load-balance exclude mac
0102.0304.0506
Router(config)# cable load-balance exclude oui
00:00:0c
Router(config)#
|
Specifies that one or more cable modems should be excluded from load-balancing operations.
•mac-address — Hardware (MAC) address of an individual cable modem to be excluded from load balancing. (You cannot specify a multicast MAC address.)
•oui-value — Organization unique identifier (OUI) value for a vendor whose cable modems are to be excluded from load balancing. The OUI is composed of three hexadecimal bytes separated by colons or periods.
By default, the cable modems are excluded from dynamic and static load balancing, but they continue to participate in passive load balancing. Use the following options to exclude the cable modems from others combinations of load balancing:
•enforce — (Optional) Excludes the cable modems from dynamic load balancing, but they continue to participate in static load balancing.
•static — (Optional) Excludes the cable modems from static load balancing, but they continue to participate in passive load balancing and dynamic load balancing.
•strict — (Optional) Excludes the cable modems from all forms of load balancing.
•static strict — (Optional) Excludes the cable modem from static and passive load balancing, but they continue to participate in dynamic load balancing.
|
Step 4
|
exit
Example:
Router(config)# exit
Router#
|
Exits global configuration mode.
|
Distributing Downstream Load Balancing with Upstream Load Balancing
Cisco IOS Release 12.3(17b)BC4 enables the optional configuration of making downstream load balancing decisions as follows:
•The target downstream segment is in the same downstream load balancing group as the source downstream segment.
•The upstream load balancing group can be set for the corresponding channel on which a cable modem is balanced.
•The Cisco CMTS automatically locates the upstream segment for a load balancing group and processes the upstream group status on the source interface that has the lowest load.
•The target downstream segment must have an upstream channel set in the upstream load balancing group.
•The highest target upstream segment must carry less load than any other potential target —the highest upstream segment on other interfaces.
This functionality is enabled with the following command:
cable load-balance group ds-lb-group-id policy us-groups-across-ds
SUMMARY STEPS
1. enable
2. configure terminal
3. cable load-balance group ds-lb-group-id policy us-groups-across-ds
4. exit
DETAILED STEPS
| |
Command or Action
|
Purpose
|
Step 1
|
enable
Example:
Router> enable
Router#
|
Enables privileged EXEC mode. Enter your password if prompted.
|
Step 2
|
configure terminal
Example:
Router# configure terminal
Router(config)#
|
Enters global configuration mode.
|
Step 3
|
cable load-balance group ds-lb-group-id policy
us-groups-across-ds
Example:
Router(config)# cable load-balance group 1
policy 2
|
Sets the type of service flow policy for use with Load Balancing. This command synchronizes the pending statistic between different cable interface line cards in the load balancing group. The result is an alternative downstream load balancing scheme that makes use of per-upstream loads rather than total downstream loads when making load balancing decisions.
•ds-lb-group-id—Specifies the load balancing group being configured. This downstream group includes the upstream segment in load balancing decisions.
•us-groups-across-ds—Specifies the upstream group to be distributed in load balancing decisions for the downstream group specified.
|
Step 4
|
exit
Example:
Router(config)# exit
Router#
|
Exits global configuration mode.
|
Refer to the following resources for additional information:
•"Distributing Downstream Load Balancing with Upstream Load Balancing" section
•<
