Table Of Contents
Cisco Service Control for Managing Remote Cable MSO Links Solution Guide
Introduction and Solution Overview
Learning the Interface Topology
Learning the Interface Association
Managing Control and Reporting
Configuring Virtual Links Global Controllers
Applying Service Configurations to SCE Platforms
Configuring the Virtual Links Manager
Monitoring Using the p3vlink Utility
Monitoring Virtual Links Using the Reporter
Creating a New Report Instance
Verifying the Correct Policy is Enforced on a Specific Subscriber
Verifying that the Distribution of Subscribers to the Virtual Link is Correct
Inaccurate, Unavailable, or Missing Reports Information for a Specific CMTS Interface
Verifying that the VLM Updates the Collection Manager
SCE is Congested, But Connected CMTSs are Not Congested
Verifying that No Subscriber Is Associated with the Default Virtual Link
Obtaining Documentation and Submitting a Service Request
Cisco Service Control Solution Guide
Cisco Service Control for Managing Remote Cable MSO Links Solution Guide
Revised: January, 2009, OL-18380-011 About this Guide
This Cisco Service Control for Managing Remote Cable MSO Links Solution Guide describes the use of a Cisco Service Control solution in a cable environment to optimize traffic on remote links. It describes the setup of a solution that uses the Virtual Link Manager to enable traffic optimization of remote links, and the monitoring of that solution after deployment.
This guide assumes a basic familiarity with the concept of the Cisco Service Control solution, the Service Control Engine (SCE) platforms, and related components.
2 Introduction and Solution Overview
Introduction
In Cable Multiple Service Operator (MSO) networks, the SCE is inserted in a location upstream of the Cable Modem Termination System (CMTS), which is the first IP hop in the MSO network. In this location, the SCE is often used to implement fair use policies (FUP) and perform congestion mitigation. MSOs work to ensure that the existing network infrastructure is used optimally using the SCE to enforce fairness between the different subscribers when the network is in the state of congestion.
The bandwidth of the radio frequency interfaces of the CMTS ranges from several megabits per second (Mbps) to tens of Mbps. The CMTS aggregates these interfaces into higher bandwidth upstream links, typically 1 gigabit per second (Gbps), where the SCE is connected.
Cable modems which are connected to the hybrid fiber coaxial cable (HFC) downstream connection of the CMTS are associated with CMTS interfaces. The CMTS interfaces can be termed upstream or downstream upon bootup or dynamically (depending on the direction of traffic flow) when you use some load balancing algorithms.
Figure 1 shows a typical deployment topology of an SCE in an MSO network.
Figure 1 SCE Deployed in MSO Network
Solution Overview
As part of this solution, the Virtual Link Manager (VLM) makes the SCE aware of the interface association of subscribers (cable modems) and accounts for and controls aggregate traffic in the context of a physical interface (CMTS upstream or downstream). This awareness allows the SCE to perform congestion mitigation at the level of the CMTS physical interface.
The solution is required to manage a large number of subscribers, each of which is connected to the CMTS and through it to the SCE. Within the CMTS, subscribers are connected to shared radio frequency interfaces, where they use the Data Over Cable Service Interface Specification (DOCSIS) MAC layer to transport their traffic. The shared radio frequency interfaces are termed upstream and downstream.
CMTS upstream and downstream interfaces are aggregated through the CMTS interface toward the core of the network, and eventually end in one or several physical interfaces that are connected to the SCE, which monitors and controls traffic.
MSOs use the SCE to prioritize traffic within each of the CMTS radio frequency interfaces when traffic is congested. Cisco Service Control provides a virtual link concept that allows MSOs to monitor and control traffic for each interface.
Although it is possible to prioritize traffic by allocating packages to pairs of interfaces, this is time-consuming. The virtual links approach simplifies the model. Each virtual link is monitored and controlled separately within the service control solution, while virtual link provisioning is performed through the SCE CLI. The policy remains simple, and reflects only per subscriber tiering.
Figure 2 shows traffic traveling through the SCE that is mapped into SCE virtual links which reflect the CMTS physical interfaces. Monitoring and control is performed in the context of the virtual links.
Figure 2 Traffic Traversing SCE that is Mapped to SCE Virtual Links
The Cisco Service Control solution provides the VLM to automate many of the configuration actions that the network administrator ordinarily performs, including:
•Provisioning virtual-link maps for each SCE based on the interface maps of the CMTSs that provide network access.
•Configuring bandwidth values for the virtual links based on the CMTS interface speed values. These values tend to change regularly as part of cable plant maintenance, or automatically as part of spectrum management. These changes must be traced and acted upon by the network administrator.
•Building the subscriber manager virtual-link mapping configuration
The virtual-link mapping configuration defines the mapping between:
–The DHCP options that define the subscribers' interface associations that are extracted from the DHCP acknowledge message.
–The corresponding virtual links ID.
Learning the Interface Topology
To control and report traffic in the context of a remote interface, you must map the topology in terms of the available CMTS interfaces and their associated bandwidth. This map must include keys that are used by the SCE to associate subscriber traffic with specific interfaces.
The SCE learns the interface topology by retrieving the CMTS configuration using the Simple Network Management Protocol (SNMP) and converting the configuration to a virtual links map. Virtual links are provisioned to the relevant SCEs.
Learning the Interface Association
Interface association awareness is achieved through DHCP integration. The CMTS IP (specifically, the Relay-Agent IP, or giaddr) is part of the DHCP dialog and upstream and downstream interface IDs are included in the Relay-Agent option (for example, option 82 [encoded in suboption 1, the circuit ID]). This information allows the SCE to uniquely identify upstream and downstream interfaces to which a subscriber is mapped, even in cases in which more than one CMTS is connected to an SCE.
The SCE DHCP sniffer login event generator (LEG) extracts the CMTS IP and reports it to the subscriber manager, which performs the appropriate virtual-link association, allowing the SCE to manage the traffic correctly.
Dynamic giaddr Learning
When the VLM queries the CMTS device, it reads all of the IP addresses from the CMTS device IP table and creates the mapping table that is used to map IPs to the CMTS device to which they are related. Many of the IP addresses that are read from the CMTS device are not used by subscribers which can cause the mapping table to become too big and unmanageable. To prevent this, the VLM dynamically learns and removes the giaddr values.
•When a subscriber logs in, the CMTS device appends the giaddr to the DHCP transaction.
–If the giaddr already exists in the mapping table, the subscriber is logged in with a specific vlink policy.
–If the giaddr does not exist in the mapping table, the subscriber is logged in with the default vlink policy and the giaddr is added to the mapping table. If the giaddr is related to a CMTS device (this information is written in the mapping table that was created when the VLM queries the CMTS device):
— The VLM adds this giaddr to be one of the CMTS device giaddr attributes.
— The VLM updates the DHCP LEG with the mapping table related to the new giaddr.
When the next DHCP transaction occurs, the subscriber is moved from the default vlink policy to a specific vlink policy.
•The VLM defines a lease time for each dynamic giaddr. If no further login operations occur during the lease time period:
–The VLM removes the giaddr from its list of giaddr values.
–The IP value is no longer a giaddr in the CMTS device: when performing p3vlink --show-device -d <device>, the giaddr attribute does not contain the removed IP.
–The LEG removes the entries from the mapping table that are related to the giaddr.
–For each subscriber, the VLM checks if the subscriber giaddr custom property is the same as the removed giaddr and if so, changes the property to be the IP address of the CMTS device.
The following example shows the current details of a subscriber:
p3subs --show -s lynn_jonesName: lynn_jonesDomain: subscribersMappings:IP: 1.1.1.13/32Properties:downVlinkId=7 Name=device1_1_Cmts8/1-downstream1upVlinkId=4 Name=device1_1_Cmts8/1-upstream1Custom Properties:giaddr=1.1.1.1Command terminated successfullyIf the IP address 1.1.1.1 is the removed giaddr and 2.2.2.2 is the CMTS device IP address, the result of the lease time operation is as follows:
p3subs --show -s lynn_jonesName: lynn_jonesDomain: subscribersMappings:IP: 1.1.1.13/32Properties:downVlinkId=7 Name=device1_1_Cmts8/1-downstream1upVlinkId=4 Name=device1_1_Cmts8/1-upstream1Custom Properties:giaddr=2.2.2.2Command terminated successfullyManaging Control and Reporting
SCE virtual links emulate the physical interfaces of the CMTSs and the VLM provisions the links with the bandwidth required to control the traffic.
1. For each CMTS physical interface (either upstream or downstream), the VLM creates a virtual link on the SCE.
2. The VLM maps traffic that travels from a subscriber that is associated to this interface to the virtual link.
3. To create the proper association of subscribers to virtual links, the VLM creates a mapping between the DHCP information (CMTS-ID, upstream-ID, downstream-ID) and the virtual link IDs.
–The VLM creates an upstream virtual link for every upstream-ID on a CMTS.
–The VLM creates a downstream virtual link for every downstream-ID on a CMTS.
Subscriber management logic is required to associate subscribers with their upstream and downstream virtual links based on the attributes that the DHCP LEG extracts from the DHCP traffic.
In addition to the virtual links association, the subscriber is also assigned a package. In terms of bandwidth management, you can only use schemes that use one virtual-link-controller per direction; therefore, you should design the bandwidth controller architecture (committed information rate, peak information rate, and assurance level) accordingly.
3 Configuring the Solution
This chapter describes:
•Basic topology for managing remote cable MSO links and the high-level steps to configure the solution.
•Prerequisites for configuring a solution that uses traffic optimization on remote links with the Virtual Links Manager (VLM)
•Configuring the VLM using the configuration files contained in the subscriber manager installation.
Solution Topology
Figure 3 shows a system that can be configured for managing remote cable MSO links.
Figure 3 Traffic Optimization on Remote Links Topology
Note Using a collection manager is optional and is not required for the solution to work. However, if you do not use a collection manager, the reports provided by the Service Control Application Reporter (SCA Reporter) can be selected only by the virtual link index and not by the virtual link name.
To work with the collection manager, you must associate the collection manager with the SCE and also with the subscriber manager. However, it is not mandatory to associate the collection manager with the subscriber manager. In this case, the collection manager receives the RDRs but does not automatically receive the index to the vlink name mappings.
The operator configures the IP addresses of the CMTS devices, the SCEs, the CM, and their interrelations on the SM (1). The SM queries each CMTS device through SNMP to determine its interfaces and their corresponding interface speeds (2). The SM provisions the SCE virtual links (3).
CMTS Device Compatibility
This traffic optimization on remote links solution currently supports the following Cisco CMTS Universal Broadband Router (uBR) devices:
•uBR10K
•uBR7246
To use other CMTS devices, you must ensure the following conditions are met:
•The upstream and downstream interface IDs are encoded as part of option 82, sub option 1 (the circuit ID) and appears in the DHCPACK message.
•The CM-MAC, which is used as the Subscriber-ID, is encoded as option 82, suboption 2 (the remote ID) and appears in the DHCPACK message.
•The DHCP traffic flows through the SCE.
The traffic optimization on remote links solution uses the following MIBs and RFCs:
•RFC 1213 MIB-2 interface—The following nodes are required: ipAddrTable, ipAddrEntry, ipAdEntAddr, ifTable, ifEntry, ifIndex, ifDescr, ifType, ifSpeed, ifAdminStatus.
•IANAifType MIB
Prerequisites
Before you set up the managing remote cable MSO links solution, you must complete the following:
•Install Release 3.5.0 or later onto the Subscriber Manager, Collection Manager (optional), and Service Control Engines (SCE).
•Install the Cisco Service Control Application for Broadband (SCA BB) (the Engage pqi file) on the SCEs used in the solution. See the Cisco Service Control Application for Broadband User Guide, the "Using the Network Navigator" chapter, the "How to Install PQI Files on SCE Devices" section.
Configuring the Solution
The procedures in this section describe how to configure the solution.
Configuring Virtual Links Global Controllers
Step 1 Start the SCA BB console by choosing Start > All Programs > Cisco SCA > SCA BB Console 3.5.0 > SCA BB Console 3.5.0.
The Cisco Service Control SCA BB Console splash screen appears. After the Console loads, the main window of the Console appears. The first time that you launch the Console, the Welcome view is open in the main window.
Step 2 To close the Welcome view, click Go to the console. The Welcome view closes. The Network Navigator tool is open in the Console.
Step 3 From the Console main menu, choose Tools > Service Configuration Editor.
If no service configurations are open when you open the Service Configuration Editor tool, a No Service Configuration Is Open dialog box appears.
a. To create a new service configuration, click Yes. A New Service Configuration Settings dialog box appears.
b. Select an operational mode for the service configuration.
c. Click OK.
The new service configuration is added to the Console window, open on the Policies tab, and becomes the active service configuration.
Step 4 In the Global Policy Preferences area, click Edit Preferences.
The Global Controllers mode dialog box appears.
Step 5 Ensure that the Enable Virtual Links Mode check box is checked.
Step 6 Click Finish.
Applying Service Configurations to SCE Platforms
Step 1 Using the Network Navigator tool, in the Site Manager tree, right-click an SCE device. A popup menu appears.
Step 2 From the menu, choose Apply Service Configuration.
The Choose Policy dialog box appears, listing all service configurations that are open in the Service Configuration Editor.
Note If only one service configuration is open in the Service Configuration Editor, a Password Management dialog box appears. Go to Step 5.
Note If the open policy is a virtual links policy, the Apply Template Virtual Links Values dialog box prompts you to apply the template virtual links value to the existing virtual links. Go to Step 4 a.
Step 3 Select a service configuration from the list.
Step 4 Click OK.
If the policy is a virtual links policy, the Apply Template Virtual Links Values dialog box prompts you to apply the template virtual links value to the existing virtual links.
a. To apply the template virtual links value to the existing virtual links, click Yes.
A Password Management dialog box appears.
Step 5 Enter the appropriate password.
Step 6 Click Apply.
The Password Management dialog box closes. An Applying service configuration to SCE progress bar appears. The service configuration is applied to the selected SCE platform.
Configuring the Virtual Links Manager
Step 1 On the subscriber manager machine, open the p3sm.cfg configuration file, which is located in the ~pcube/sm/server/root/config/ directory.
For details about making configuration changes in the p3sm.cfg file see the Cisco Service Control Management Suite Subscriber Manager User Guide, the "Configuration Files Options" chapter.
a. Create a section for any SCE devices and define the IP address and optionally the port values of the SCE devices. The following example shows the SCE sections created for two SCE devices named SCE1 and SCE2.
[SCE.SCE1] ip=209.165.201.2 [SCE.SCE2] ip=209.165.201.3
b. (Optional) Create a section for the collection manager and define the IP address and the connected SCE devices. Defining the CM port value is optional; if a value is not defined, the default value of 14375 is used.
The following example shows the CM section created that receives the Raw Data Records (RDRs) for two SCE devices named SCE1 and SCE2.
[CM.CM1] ip=209.165.202.129 port=14375 sce_list=SCE1,SCE2
Note Using a collection manager is optional and is not required for the solution to work. However, if you do not use a collection manager, all of the virtual link reports can be selected only by the virtual link index and not by the virtual link name.
Step 2 Save and close the p3sm.cfg configuration file.
Step 3 On the subscriber manager machine, open the dhcpsnif.cfg configuration file which is located in the ~pcube/sm/server/root/config/ directory.
For details about making configuration changes in the dhcpsnif.cfg file see the Cisco SCMS SM LEGs User Guide, the "SCE-Sniffer DHCP LEG" section, "Configuring the SCE-Sniffer DHCP LEG" part.
a. In the [SCE-Sniffer DHCP LEG] section, set the start value to yes.
[SCE-Sniffer DHCP LEG] start=yes
All other values can be left at default values.
b. In the [Subscriber ID] section, set the dhcp_option to the DHCP option that contains the subscriber ID and set the dhcp_option_type to binary. For example:
[Subscirber ID] dhcp_option=82:2 dhcp_option_type=binary
All other values can be left at default values.
Step 4 Save and close the dhcpsnif.cfg configuration file.
Step 5 On the subscriber manager machine, open the dhcp_pkg.cfg configuration file which is located in the ~pcube/sm/server/root/config/ directory.
For details about making configuration changes in the dhcp_pkg.cfg file, see the Cisco SCMS SM LEGs User Guide, the "SCE-Sniffer DHCP LEG" section, the "Configuring the SCE-Sniffer DHCP LEG" part.
a. Define a downstream virtual link policy by setting the parameters:
[DHCP.Policy.VirtualLinkDownstream] policy_property_name=downVlinkId options_order_for_policy_name=giaddr,82:1 options_type=integer,binary allow_login_with_no_policy=true use_default=false default_policy=0
Note Do not define the mapping_table parameter.
b. Define an upstream virtual link policy by setting the parameters:
[DHCP.Policy.VirtualLinkUpstream] policy_property_name=upVlinkId options_order_for_policy_name=giaddr,82:1 options_type=integer,binary allow_login_with_no_policy=true use_default=false default_policy=0
Note Do not define the mapping_table parameter.
Step 6 Save and close the dhcp_pkg.cfg configuration file.
Note When the VLM is activated, the DHCP sniffer adds a custom property to each subscriber with the value of the giaddr option, which is one of the CMTS IP addresses.
Step 7 On the subscriber manager machine, open the vlink.cfg configuration file which is located in the ~pcube/sm/server/root/config/ directory.
a. In the [General] section, configure the following parameters:
–start—Setting the start parameter to yes instructs the SM to start the Virtual Link Manager (VLM) when the SM starts. Possible values for this parameter are yes and no.
Note When the start parameter is set to no, the data stored in the database for all CMTS devices is deleted.
–monitoring_period—Determines the interval in minutes at which the VLM queries the CMTS devices for any interface changes. Setting this parameter to 0 stops the VLM querying the CMTS devices, but does not stop the VLM. The default value is 60.
–upstream_vlink_factor—Determines the percentage of the interface bandwidth that the SCE allows to be transmitted from the CMTS device to the Internet. The default value is 95.
–downstream_vlink_factor—Determines the percentage of the interface bandwidth that the SCE allows to be transmitted from the Internet to the CMTS device. The default value is 95.
Note When defining the upstream_vlink_factor and downstream_vlink_factor parameters, take precaution. Setting a value too low causes bandwidth capacity to be wasted. Setting a value too high causes data to be lost due to dropped packets.
–log_all—Setting the log_all parameter to true causes the system to dump log messages to the user log.
Note Setting log_all to true can cause performance degradation.
The following example shows the [General] section of the vlink.cfg configuration file:
[General] start=true monitoring_period=60 upstream_vlink_factor=95 downstream_vlink_factor=95 log_all=false
b. For each CMTS device, configure a [Device.<device name>] section with the following parameters:
Note The <device name> of the section is used as part of the virtual link name for all virtual links associated with this CMTS device. The name of the CMTS device also appears in the reporter.
–ip—Specifies the IP address of the CMTS device.
–sce_name—Specifies the name of the SCE to which the CMTS device is connected. The sce_name must match an SCE section defined in the p3sm.cfg configuration file.
–upstream_vlink_factor—(Optional) Determines the percentage of the interface bandwidth that the SCE allows to be transmitted from this CMTS device to the Internet. Setting this parameter overrides the setting in the [General] section. This parameter is optional. If it is not configured, the value defined in the [General] section is used. The default value is 95.
–downstream_vlink_factor—(Optional) Determines the percentage of the interface bandwidth that the SCE allows to be transmitted from the Internet to this CMTS device. Setting this parameter overrides the setting in the [General] section. This parameter is optional. If it is not configured, the value defined in the [General] section is used. The default value is 95.
–log_all—Setting the log_all parameter to true causes the system to dump log messages to the user log for this CMTS device. If the log_all parameter in the [General] section is true, setting this parameter to false has no effect; if the log_all parameter in the [General] section is false, setting this parameter to true enables logging for operations related only to this CMTS device; such as, CMTS device creation and deletion.
–snmp_community—Specifies the SNMP community value for the VLM to communicate with the CMTS device. The default value is public.
–giaddr_external—For each giaddr value, the VLM creates a policy mapping table in the DHCP LEG without waiting for a login from the giaddr values. Use this parameter to insert giaddr values to the mapping table that was created by the VLM after querying the CMTS device. Use a comma `,' delimiter between IP addresses. The default value for this parameter is empty.
–giaddr_replace—Specifies whether or not the VLM uses the IP addresses defined in the giaddr_external parameter as the only giaddr list of the CMTS device. When set giaddr_replace=no, the VLM uses the IP addresses defined in giaddr_external and the giaddr list that is found when querying the CMTS device as the CMTS giaddr list. Possible values for this parameter are yes and no. The default value is no.
–giaddr_remove—For each giaddr value, the VLM creates a policy mapping table in the DHCP LEG without waiting for a login from the giaddr values. Use this parameter to remove a list of IPs from the mapping table that was created by the VLM after querying the CMTS device. Use a comma `,' delimiter between IP addresses.
The following example shows a [Device.<device name>] section of the vlink.cfg configuration file:
[Device.CMTS1] ip=192.0.2.10 sce_name=SCE1 log_all=false
Step 8 Save and close the vlink.cfg configuration file.
Step 9 Load the configuration to the SM by running the following command on the SM machine from the bin directory. (Run the command as user pcube).
>p3sm --load-config
4 Monitoring the Solution
This chapter describes the three monitoring mechanisms that you can use to monitor the traffic optimization on remote links solution:
•p3vlink command line utility (CLU)
•SCE command line interface (CLI)
•Reporter tool virtual links report template group
Virtual Links Names
Each virtual link represents a single interface on the CMTS device and the virtual link name comprises the CMTS device name and the interface name. The virtual links are named according to the following naming convention:
<Device name>_<Interface name>
•<Device name>—This portion of the name is set when configuring the CMTS in the vlink.cfg configuration file.
•<Interface name>—This portion of the name identifies the specific CMTS interface including the direction and the interface index. This information is how the CMTS describes the interface internally and is retrieved from the CMTS device using SNMP.
–The direction portion of the virtual link name indicates the virtual link direction. This can be "upstream" or "downstream."
–The interface index indicates the specific interface on the CMTS of the virtual link.
The following is an example virtual link name for a CMTS device named Device-1:
Device-1_CMTS1/0-upstream1
Monitoring Using the p3vlink Utility
The p3vlink utility provides the ability to show virtual link configurations and metrics related to the virtual links. The command format is:
p3vlink OPERATION [OPTIONS]The following tables list the p3vlink operations and options.
p3vlink Utility Examples
To show the CMTS device general configuration, CMTS device list, and CMTS device information:
p3vlink --show General data:-------------Start: YesMonitor Every: 60 minutesBW Up Factor 95BW Down Factor 95Next query operation: Wed Nov 05 08:40:33 IST 2008Next ip removal operation: Wed Nov 12 10:40:11 IST 2008Device list-----------1) Name: device, Query state: Completed, Last successful query: Wed Nov 05 08:39:35 IST 2008Command terminated successfully>To show the general configuration of a specified CMTS device:
p3vlink --show-device -d CMTS1 --detail Name: CMTS1IP: 192.0.2.10SCE Related: sce0Upstream factor: 95Downstream factor: 95Last success Query: Thu Jun 19 17:54:48 IDT 2008Last Query Attempt: Thu Jun 19 17:54:48 IDT 2008Last Query Status: CompletedSync state with SCE: doneSync state with CM: doneGiaddr List: 127.0.0.1Num of up interfaces: 6Num of down interfaces: 2VLink Information:1) Name: CMTS1_Cmts0/0-upstream2, VLink Id: 1, Direction: UP, If Speed: 5000.2) Name: CMTS1_Cmts0/0-downstream1, VLink Id: 1, Direction: DOWN, If Speed: 10000.3) Name: CMTS1_Cmts0/0-upstream3, VLink Id: 2, Direction: UP, If Speed: 10000.4) Name: CMTS1_Cmts1/0-downstream1, VLink Id: 2, Direction: DOWN, If Speed: 20000.5) Name: CMTS1_Cmts0/0-upstream1, VLink Id: 3, Direction: UP, If Speed: 10000.6) Name: CMTS1_Cmts1/0-upstream2, VLink Id: 4, Direction: UP, If Speed: 20000.7) Name: CMTS1_Cmts1/0-upstream3, VLink Id: 5, Direction: UP, If Speed: 20000.8) Name: CMTS1_Cmts1/0-upstream1, VLink Id: 6, Direction: UP, If Speed: 20000.Command terminated successfully>The output of this command includes the following four information elements:
•Num of up interfaces—Summarizes the total number of upstream virtual links related to this CMTS device. "Unknown" indicates that the VLM was not able to communicate with the CMTS device.
•Num of down interfaces—Summarizes the total number of downstream virtual links related to this CMTS device. "Unknown" indicates that the VLM was not able to communicate with the CMTS device.
•Sync state with SCE:
–Done—The SCE is fully synchronized with CMTS device information. When working in cascade mode, both the active and standby SCEs are synchronized with CMTS device data.
–Not-done—The SCE (or one of the SCEs in cascade mode) is not synchronized with all CMTS device data. Use the command p3vlink --resync -n <sce which manages the device> to perform the synchronization operation.
•Sync state with CM
–Done—The CM is fully synchronized with CMTS device information.
–Not-done—The CM is not synchronized with all CMTS device data. Use the command p3vlink --resync -n <sce which manages the device> to perform the synchronization operation.
–N/A—The SCE to which the CMTS device belongs, is not connected to any CM.
To show all of the virtual links for a specific network element (SCE):
p3vlink --show-vlinks -n sc0device0_0_Cmts0/1-downstream1, vlink id=15, direction=DOWNdevice0_0_Cmts0/1-upstream1, vlink id=8, direction=UPdevice0_0_Cmts0/1-upstream2, vlink id=16, direction=DOWNdevice0_1_Cmts1/1-downstream1, vlink id=11, direction=DOWNdevice0_1_Cmts1/1-upstream1, vlink id=6, direction=UPdevice0_1_Cmts1/1-upstream2, vlink id=12, direction=DOWNdevice0_2_Cmts2/1-downstream1, vlink id=25, direction=DOWNdevice0_2_Cmts2/1-upstream1, vlink id=13, direction=UPdevice0_2_Cmts2/1-upstream2, vlink id=26, direction=DOWNdevice0_3_Cmts3/1-downstream1, vlink id=13, direction=DOWNdevice0_3_Cmts3/1-upstream1, vlink id=7, direction=UPdevice0_3_Cmts3/1-upstream2, vlink id=14, direction=DOWNdevice0_4_Cmts4/1-downstream1, vlink id=21, direction=DOWNdevice0_4_Cmts4/1-upstream1, vlink id=11, direction=UPdevice0_4_Cmts4/1-upstream2, vlink id=22, direction=DOWNdevice0_5_Cmts5/1-downstream1, vlink id=1, direction=DOWNdevice0_5_Cmts5/1-upstream1, vlink id=1, direction=UPdevice0_5_Cmts5/1-upstream2, vlink id=2, direction=DOWNdevice0_6_Cmts6/1-downstream1, vlink id=9, direction=DOWNdevice0_6_Cmts6/1-upstream1, vlink id=5, direction=UPdevice0_6_Cmts6/1-upstream2, vlink id=10, direction=DOWNdevice1_0_Cmts7/1-downstream1, vlink id=3, direction=DOWNdevice1_0_Cmts7/1-upstream1, vlink id=2, direction=UPdevice1_0_Cmts7/1-upstream2, vlink id=4, direction=DOWNdevice1_1_Cmts8/1-downstream1, vlink id=7, direction=DOWNdevice1_1_Cmts8/1-upstream1, vlink id=4, direction=UPdevice1_1_Cmts8/1-upstream2, vlink id=8, direction=DOWNdevice1_2_Cmts9/1-downstream1, vlink id=27, direction=DOWNdevice1_2_Cmts9/1-upstream1, vlink id=14, direction=UPdevice1_2_Cmts9/1-upstream2, vlink id=28, direction=DOWNdevice1_3_Cmts10/1-downstream1, vlink id=23, direction=DOWNdevice1_3_Cmts10/1-upstream1, vlink id=12, direction=UPdevice1_3_Cmts10/1-upstream2, vlink id=24, direction=DOWNdevice1_4_Cmts11/1-downstream1, vlink id=19, direction=DOWNdevice1_4_Cmts11/1-upstream1, vlink id=10, direction=UPdevice1_4_Cmts11/1-upstream2, vlink id=20, direction=DOWNdevice1_5_Cmts12/1-downstream1, vlink id=5, direction=DOWNdevice1_5_Cmts12/1-upstream1, vlink id=3, direction=UPdevice1_5_Cmts12/1-upstream2, vlink id=6, direction=DOWNdevice1_6_Cmts13/1-downstream1, vlink id=17, direction=DOWNdevice1_6_Cmts13/1-upstream1, vlink id=9, direction=UPdevice1_6_Cmts13/1-upstream2, vlink id=18, direction=DOWNCommand terminated successfully>To show the vlink data of a specific link:
p3vlink --show-vlink-data --vlink-name=device_Cmts0/0-downstream1VLink Id: 1Direction: downStreamSCE Name: sce0Device Name: deviceIf Speed: 30000> Command terminated successfully
Note If more than one vlink has the same name, this command displays the information for all of the vlinks.
To show the subscribers using virtual links:
Use the p3subsdb command to list all the subscribers:
p3subsdb --show-alllynn_jonesCommand terminated successfully>Use the p3subs command to show the virtual links of a particular subscriber:
p3subs --show -s lynn_jonesName: lynn_jonesDomain: subscribersMappings:IP: 1.1.1.13/32Properties:downVlinkId=7 Name=device1_1_Cmts8/1-downstream1upVlinkId=4 Name=device1_1_Cmts8/1-upstream1Custom Properties:giaddr=1.1.1.1Command terminated successfully>Use the p3vlink command to show the subscribers that are associated with a particular CMTS device:
p3vlink --show-subs -d device1_1Subscribers related to device: device1_1 vlink-id: 4, giaddr: 1.1.1.1, direction UPlynn_jonesSubscribers related to device: device1_1 vlink-id: 7, giaddr: 1.1.1.1, direction DOWNlynn_jonesCommand terminated successfully>Monitoring Using the SCE CLI
The SCE provides two CLI commands to monitor the virtual links in the solution:
•show interface LineCard virtual-links all—Displays the configured virtual links id and direction.
•show interface LineCard virtual-links changed—Displays only the virtual links whose global controller (GC) values differ from their directional template virtual link.
The following examples show the output from the CLI virtual links monitoring commands.
SCE2000#> show interface LineCard 0 virtual-links allVirtual Link enabledVirtual link index 1 direction upstreamVirtual link index 2 direction upstreamVirtual link index 3 direction upstreamVirtual link index 4 direction upstreamVirtual link index 12 direction upstreamVirtual link index 13 direction upstreamVirtual link index 14 direction upstreamVirtual link index 15 direction upstreamSCE2000#> show interface LineCard 0 virtual-links changedVirtual Link enabledVirtual link index 3 direction upstreamGlobal Controller index 0 timebased values = 300,300,300,300Global Controller index 1 timebased values = 500,500,500,500Virtual link index 12 direction upstreamGlobal Controller index 0 timebased values = 700,700,700,700Virtual link index 14 direction upstreamGlobal Controller index 0 timebased values = 5500,5500,5500,5500Global Controller index 1 timebased values = 1500,1500,1500,1500Monitoring Virtual Links Using the Reporter
The Service Control Application for Broadband (SCA BB) includes a Reporter tool that allows you to produce reports based on the traffic analysis performed by the SCE platform. The information is sent from the SCE platform and is stored in a database. The Reporter can query and retrieve information from the database and present the results in a comprehensive range of reports.
The Reporter includes the Virtual Links Monitoring group of report templates that allow you to view statistics of bandwidth or volume of traffic used by a virtual link. The reports are provided per service usage counter for the total volume used by the virtual link. The volume consumption can be displayed per service for the virtual link.
Each report can be filtered to focus on a virtual link ID, a virtual link name, a virtual link direction, or a combination of the virtual link identifiers.
The Virtual Links Monitoring group includes the following report templates:
•VLink Bandwidth per Service—Shows the distribution of bandwidth among the different service usage counters defined in the system for all subscribers
•VLink Aggregated Usage Volume per Service—Shows the total volume of traffic (upstream and downstream) for each service usage counter
•VLink Hourly Usage Volume per Service—Shows the distribution of volume among the different service usage counters defined in the system, grouped by hour
•VLink Daily Usage Volume per Service—Shows the distribution of volume among the different service usage counters defined in the system, grouped by day
•Daily Peak BW for all VLinks—Shows the daily value of the maximum bandwidth (1-hour or 2-hour average) for all virtual links
Creating a New Report Instance
Step 1 Go to the SCA BB console. From the Tools menu, choose Reporter.
The Reporter opens and the Templates tab appears.
Step 2 In Templates view, expand the Virtual Links Monitoring group.
Step 3 Right-click a report instance (for example, VLink Bandwidth per Service).
A popup menu appears.
Step 4 From the menu, choose New.
The New Report Wizard dialog box appears, allowing you to configure the new report.
Step 5 In the Report name field, enter the name of the report instance.
The default report name is VLink Bandwidth per Service #1. (If you create another report instance from this report template, it is named VLink Bandwidth per Service #2, and so on. You can rename report instances.)
Step 6 To create a report that focuses on a particular CMTS device:
a. In the Select VLink names row, click the right column and click the Browse button that appears.
b. Click the Search button.
c. In the search box, enter the name of the CMTS device with a * before and after; for example *CMTS_1*
All of the virtual links that contain the string `CMTS_1' appear.
d. Select all of the results and click OK.
The report is produced that focuses on the specified CMTS device.
5 Troubleshooting the Solution
This section describes a number of problem scenarios that you may encounter when using the solution.
Subscriber Complaints
To troubleshoot the cause of bad service to subscribers, consider the following:
•Congestion in the backbone network may cause congestion in subscriber traffic
•A problem with the solution, such as assigning the incorrect policy to the subscriber, or an incorrect interface association, may exist.
To make sure the problem is not with the SCE, perform the following procedures:
Verifying the Correct Policy is Enforced on a Specific Subscriber
Step 1 Verify that the subscriber manager-SCE subscriber data is synchronized.
In the SCE, use the show interface linecard 0 subscriber name <sub MAC> command to verify that the upvlinkId and downVlinkId values are the same as in the p3subs --show -s <sub MAC> subscriber manager CLU output and that the subscriber package ID is as expected.
The following example shows the output of the SCE CLI:
SCE2000#>show interface LineCard 0 subscriber name lynn_jonesSubscriber 'lynn_jones' manager: SMSubscriber 'lynn_jones' properties:downVlinkId=10monitor=0new_classification_policy=0packageId=0QpLimit[0..17]=0*17,8QpSet[0..17]=0*17,1upVlinkId=10...The following example shows the output of the subscriber manager CLU:
>p3subs --show -s lynn_jonesName: lynn_jonesDomain: subscribersMappings:IP: 5.101.5.129/32Properties:downVlinkId=10 Name=dev0_9_if19-downupVlinkId=10 Name=dev0_9_if19-upstream0Custom Properties:giaddr=5.101.254.105Command terminated successfullyStep 2 Verify that the subscriber manager-SCE virtual links data is synchronized.
Compare the output of the SCE show interface linecard 0 virtual-link changed command with the subscriber manager CLU p3vlink --show-vlink-data --vlink-name <UpVlinkId/DownVlinkId Name> to make sure that the PIR configuration is correct.
Note The SCE PIR value is in kilobits. You can calculate the VLM value by performing a multiplication with the relevant CMTS device factor as defined in the vlink.cfg configuration file. The CMTS device factor value appears in the output of the subscriber manager CLU p3vlink --show-device -d <device-name>.
The following example shows the output of the SCE CLI:
SCE2000#>show interface LineCard 0 virtual-links changedVirtual Link enabledVirtual link index 1 direction upstreamGlobal Controller index 0 timebased values = 1200,1200,1200,1200...Virtual link index 10 direction upstreamGlobal Controller index 0 timebased values = 4000,4000,4000,4000...Virtual link index 1 direction downstreamGlobal Controller index 0 timebased values = 1100,1100,1100,1100...Virtual link index 10 direction downstreamGlobal Controller index 0 timebased values = 3900,3900,3900,3900...The following example shows the output of the p3vlink --show-vlink-data subscriber manager CLU:
>p3vlink --show-vlink-data --vlink-name dev0_9_if19-upstream0VLink Id: 10Direction: UpstreamSCE Name: sce0Device Name: dev0_9If Speed: 4000000Command terminated successfullyThe following example shows the output of the p3vlink --show-device subscriber manager CLU:
>p3vlink --show-device -d dev0_9Name: dev0_9IP: 10.56.196.26SCE Related: sce0Upstream factor: 100Downstream factor: 100Last success Query: Wed Dec 17 09:42:05 IST 2008Last Query Attempt: Wed Dec 17 09:42:05 IST 2008Last Query Status: CompletedSync state with SCE: doneSync state with CM: done...Command terminated successfullyTo fix this problem:
a. Monitor all CMTS device interfaces using the subscriber manager CLU p3vlink --show-device -d NAME [--detail].
b. Verify that in the output of the CLU "Sync state with SCE" is done.
If Sync state with SCE is not done:
–View the connection state between the subscriber manager and the SCE using the subscriber manager CLU p3net --show -n SCE_NAME.
–Synchronize the SCE with the CMTS device configuration using the subscriber manager CLU p3vlink --resync -n SCE_NAME.
Step 3 Verify that the VLM-CMTS data is synchronized.
Compare the virtual link names from the subscriber manager CLU p3subs --show -s <sub MAC> output with the CMTS configuration to verify that the subscriber is assigned to the correct CMTS and CMTS interface.
Note The virtual link name structure is <CTMS_NAME>_<INTERFACE_DESCRIPTION> and is obtained from the configuration file and the SNMP ifTable data.
Make sure that the PIR value is the same as the CMTS interface speed (as taken from the SNMP ifTable data) by comparing the virtual link PIR values from the subscriber manager CLU p3vlink --show-vlink-data --vlink-name <UpVlinkId/DownVlinkId Name> to the CMTS configuration.
To fix this problem:
a. Use p3vlink --show-device -d <DEVICE_NAME> to verify that:
–The monitor period is not 0
–The last query completed successfully. Check the user-log for reasons for failure.
b. Use p3vlink --start-query -d <DEVICE_NAME> to force a start query on a specific CMTS device.
Verifying that the Distribution of Subscribers to the Virtual Link is Correct
Step 1 Obtain the VLM name for a specific interface using the p3subs --show -s <sub MAC> command, or build the interface name as <CTMS_NAME>_<INTERFACE_DESCRIPTION>.
Step 2 Retrieve the subscribers list related to a specific interface and compare it to the CMTS data using the p3vlink --show-subs --vlink-name=<UpVlinkId/DownVlinkId Name> command:
>p3vlink --show-subs --vlink-name=dev0_9_if19-upstream0Subscribers related to device: dev0_9 vlink-id: 10, giaddr: 5.101.254.100, direction UP...Subscribers related to device: dev0_9 vlink-id: 10, giaddr: 5.101.254.105, direction UP000004650424000004650425000004650429lynn_jones00000465043E...
Inaccurate, Unavailable, or Missing Reports Information for a Specific CMTS Interface
The SCA Reporter can generate per interface consumption reports that can be used to monitor the solution. The VLM updates the reporter with the interface ID to name data and the SCE sends the raw data with the interface ID (virtual link).
A problem can stem from the VLM update or from the SCE RDRs.
Two possible solutions include:
1. Verify that the SCE sends virtual link update RDRs (VLUR) to the collection manager. For information, see the Cisco SCE8000 Software Configuration Guide.
2. Verify that the VLM updates the collection manager.
Verifying that the VLM Updates the Collection Manager
Step 1 Verify the connectivity between the subscriber manager and the collection manager.
View the configured network elements and verify that the collection manager exists using the subscriber manager CLU p3net --show-all --detail command.
View the subscriber manager-collection manager connection properties and state using the p3net --show -n CM_NAME command, and then verify that the SCE list property value contains the SCE to which the CMTS is connected.
To fix this problem:
a. Check that the subscriber manager configuration file contains the relevant collection manager information:
–Check that a CM Section exists and points to the SCE section
–Validate the collection manager IP and port
–Check that the SCE is related to the collection manager.
b. See the Cisco Service Control Management Suite Subscriber Manager User Guide, the "Subscriber Manager Overview" chapter, the "Information About Communication Failures" section.
Step 2 Verify that the VLM and collection manager data is synchronized.
Verify that the synchronization state with collection manager is set to done using the CLU p3vlink --show-device -d <CMTS_NAME> command. To view a list of configured virtual links, use the collection manager CLU update_vlinks.sh --sce=<SCE_IP> --show command.
To fix this problem:
a. Force the VLM and the collection manager to synchronize using the p3vlink --resync -n <CM_NAME> command.
b. For manual configuration, see the Cisco Service Control Management Suite Collection Manager User Guide to add virtual links using the CLU update_vlinks.sh --sce=<SCE_IP> --file=vlinks.csv.
The following example shows the output of the p3net --show -n command:
>p3net --show -n cm0Network Element Information:============================Name: cm0Host: 10.56.197.231Ip: 10.56.197.231Port: 14375Status: Connection readyType: Collection ManagerSCE List: sce0Command terminated successfullyThe following example shows the output of the p3vlink --show-device -d command:
>p3vlink --show-device -d dev0_9Name: dev0_9IP: 10.56.196.26SCE Related: sce0...Sync state with SCE: doneSync state with CM: done...Command terminated successfullyThe following example shows the output of the update_vlinks.sh script:
./update_vlinks.sh --sce=10.56.197.232 --show...TIME_STAMP| SE_IP| VLINK_ID| VLINK_DIRECTION| VLINK_NAME|----------+------------------+------------------+------------------+------------------------+...2008-12-17| 10.56.197.232| 10| 0| dev0_9_if19-upstream0|2008-12-17| 10.56.197.232| 10| 1| dev0_9_if19-down|...
SCE is Congested, But Connected CMTSs are Not Congested
If the virtual link assignment is incorrect or false subscriber login operations are not handled by the correct virtual link controller, virtual links can become congested. To resolve this issue, you must confirm that there are no subscribers associated with the virtual link and also verify that the distribution of subscribers to the virtual link is as expected.
To verify that the distribution of subscribers to the virtual link is as expected, see the "Verifying that the Distribution of Subscribers to the Virtual Link is Correct" section.
Verifying that No Subscriber Is Associated with the Default Virtual Link
Step 1 Use the SCE CLI show interface LineCard 0 subscriber property <upVlinkId|downVlinkId> equals 0 to make sure that there are no subscribers associated with the default virtual link.
Subscribers in the SCE can have a virtual link ID set to 0 in several cases:
•If the giaddr list is learned automatically, a few subscribers may have this policy if they performed the first login from this giaddr. In this case, compare the results with the subscriber manager database.
•If a specific interface or CMTS is removed from the VLM, at some point all of the subscribers associated with this interface or CMTS are set to use the default virtual link.
•If no match exists in the DHCP Sniffer LEG mapping table and the LEG is configured to log in a subscriber with the default virtual link.
The following example shows the output of the show interface LineCard 0 subscriber property upVlinkId equals 0 CLI:
SCE2000#>show interface LineCard 0 subscriber property upVlinkId equals 0N/A00000465000100000465000200000465000300000465000400000465000500000465000600000465000700000465000800000465000900000465000A00000465006F0000046500700000046500710000046500720000046500730000046500740000046500750000046500760000046500770000046500780000046500DD0000046500DE0000046500DFThe following example shows the output of the p3dhcpsniff --show-policy --policy=upVlinkId --detail CLU:
>p3dhcpsniff --show-policy --policy=upVlinkId --detailPolicy Name: upVlinkId=======================separator:_use default: falsedefault value: 0allow no package: trueconcat option:giaddr,concat option:82:1,concat option type: integer,binarylog success: truelog default success: falseNumber of mappings: 4005.101.254.100_00010000=95.101.254.100_00010003=105.101.254.100_80010000=95.101.254.100_80010003=105.101.254.101_00010000=95.101.254.101_00010003=105.101.254.101_80010000=95.101.254.101_80010003=105.101.254.102_00010000=95.101.254.102_00010003=105.101.254.102_80010000=95.101.254.102_80010003=105.101.254.103_00010000=95.101.254.103_00010003=105.101.254.103_80010000=95.101.254.103_80010003=105.101.254.104_00010000=95.101.254.104_00010003=105.101.254.104_80010000=95.101.254.104_80010003=105.101.254.105_00010000=95.101.254.105_00010003=105.101.254.105_80010000=95.101.254.105_80010003=105.101.254.106_00010000=95.101.254.106_00010003=105.101.254.106_80010000=9
Userlog Messages
Table 3 lists the messages that can be written to the userlog.
6 Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS Version 2.0.
CCDE, CCENT, Cisco Eos, Cisco HealthPresence, the Cisco logo, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0812R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
© 2009 Cisco Systems, Inc. All rights reserved.