Pseudowires(PW) are used to provide end-to-end services across an MPLS network. They are the basic building blocks that can provide a point-to-point service as well as a multipoint service such as VPLS, which is practically a mesh of PWs used to create the bridge domain across which the packets flow.
Edited by: Kumar Sridhar
Readers of this document should be knowledgeable of the following:
MPLS Tunneling concepts
The information in this document is based on The Cisco® Carrier Packet Transport (CPT) Product Family and in particular CPT50.
Pseudowires conceptually look as follows:
The end-to-end service is composed of 2 parts. The Attachment Circuit (AC) part and the Pseudowire part. The whole circuit end-to-end is still referred to as Pseudowire in Cisco Trasnport Controller(CTC), but bear in mind the two part distinction exhibited here for the troubleshooting that follows.
Also remember that a tunnel must have been created to house the Pseudowire service that is configured above. The tunnel may be protected (as depicted here) or unprotected.
Pseudowire part practically starts and stops at the tunnel end points (if you exclude the MPLS encapsulation block shown here).
The AC part starts from the tunnel end point all the way toward the client facing interface, where the Ethernet Flow Point (EFP) is defined, to identify the specific client traffic that is being transported through this Pseudowire. There are 2 ACs; one on each end.
The AC carries the customer traffic in its native form, i.e. Ethernet frames with or without VLAN tagging depending on whether we are creating a VLAN based Pseudowire or Ethernet Based Pseudowire (AC Type box in the PW creation wizard). The MPLS labels for the specific PW service as well as for the tunnel over which it is riding are then added. Packets are then sent across the Pseudowire part of the circuit into the MPLS cloud. This process is called Label Imposition in MPLS terminology. On the far end, the reverse process occurs, i.e. the labels are removed or the Label Disposition occurs, and the packets, which are now back to native Ethernet frames, are then delivered to the other end through the far end AC part of the Pseudowire circuit.
Troubleshooting a Pseudowire
For the Pseudowire service to work end to end, the Pseudowire part and the 2 AC parts have to work together. Troubleshooting the circuit involves each part, where each of the AC-PW-AC parts are debugged separately to identify where the problem is.
In the following troubleshooting discussion, it is assumed that the PW has been configured correctly, and all Layer 1 or physical layer issues have already been debugged and ruled out.
First, debugging the PW part is easy. Start by identifying the circuit through the command “show mpls l2 vc” run in IOS window on an end node. Note the Virtual Circuit Identifier(VCID) as well as the Destination node address of the connection.
10.88.130.201#show mpls l2 vc
Local intf Local circuit Dest address VC ID Status
If the PW is down, then ensure the tunnel (here tunnel 102) is in good shape, and if not, then troubleshoot the tunnel issue. Troubleshooting the tunnel is beyond the scope of this article.
Ensure the labels in the stack are defined as shown above, i.e. they are not blank. Make sure the PW is programmed in the hardware by executing the command show platform mpls pseudowire pwid using the appropriate PWID number.
Entry for VlanId 200 not found in VLAN_XLATE table
DA Mac: 1 80.C20 .0 0
mpls_label(VC Label): 19
SA Mac: 4055.3958.E0E1
-- DISPOSITION SIDE --
Error: Entry not found in EGR_VLAN_XLATE table!
soc_mem_read: invalid index -1 for memory EGR_VLAN_XLATE
The logs indicate that the PW is bound and setup in the hardware, with the correct VLAN and labels, in agreement with what was seen before.
If any data point does not match or is missing, then the issue is in the driver, which did not setup and bind the PW in hardware. This points to a software or hardware defect.
If so far all is well, then you can try to ping the PW part internally by using the IOS command “ping mpls pseudowire 22.214.171.124 12 reply mode control-channel”. Note again that this pings the PW part only from one tunnel end point to the other and does not touch to the AC part of the circuit.
Note that the ping was successful and that the 5 ping echo packets are recorded as received. Also, note that the ping request packets are not recorded as sent. It seems the echo request/reply packets are sent by the CPU into the stream post the counter, and thus are not recorded.
If the pings do not work, then we should step back and debug the tunnel to ensure it is operational.
If the PW part still looks good, then focus on the AC part on each end. This is the difficult part since there is not much debug support for it, and the AC path may include several cards and interfaces as in the case with Cisco CPT50.
But there are few things that can be checked.
You can send a pattern from a tester or do a ping from the client side equipment and watch for the packets being received by the client facing interface on the CPT box. This would be easy to do for a port based PW, but not for a VLAN based PW since the interface does not display packets per VLAN. In any case the command “show int …” for the client facing interface should show packet count incrementing at least as a sign that packets are ingressing properly and if no other VLAN based circuits are active.
Bear in mind that these packets ingressing through the AC, are supposed to be MPLS labeled, and then sent across the PW to the other side. Thus, they should show in the statistics of the PW part as packets sent. So look for them in the command” show mpls l2 vc 12 detail | beg statistics”
10.88.130.201#show mpls l2 vc 12 detail | beg statistic
And they should show as packets "receive" in the same command on the far end. So the send PW packets on this end and the receive PW packets on the far end should match the number of packets sent from the client equipment. Using the same command ” show mpls l2 vc 12 detail | beg statistics” on the far end shows:
10.88.130.202#show mpls l2 vc 12 detail | beg statis
You can see the match in the packets between the send on one end and receive on the other.
In case you need to clear the MPLS counters, use the command “clear mpls counters”.
Another way to check the statistics is to use the SPAN feature to replicate the incoming EFP traffic to a spare port on the CPT node and then look for the statistics on this port to monitor the packets received from the customer interface.
And finally you can run BCM shell commands on the different fabric and line cards to track the packets internally, but that is beyond the scope of this article.