CUPS UP - Option 34 (CSS Data) Output not Visible in Monitor Protocol/Subscriber

Introduction

This document describes the behavior of monitor protocol/subscriber on Control and User Plane Separation (CUPS) User Plane (UP) about option 34 Call Search Spaces (CSS) Data, where the behavior is changed from the legacy non-CUPS system.

Problem

In legacy non-CUPS, Cisco Packet Data Network Gateway (PGW), when option 34 (CSS Data) is enabled for monitor protocol/subscriber, packets that are sent to and from the Enhanced Charging Service (ECS) module are printed.

MONITOR GLOBAL PROTOCOLS:

11 - SNMP                                 21 - L2TP (Admin only)
12 - RADIUS Authentication (Admin only)   22 - L2TPMGR (Admin only)
13 - RADIUS Accounting (Admin only)       23 - L2TP Data (Admin only)
14 - A11 (R-P Interface) (Admin only)     24 - GTPC (Admin only)
15 - Mobile IPv4 (Admin only)             25 - TACACS (Admin only)
16 - A11MGR (Admin only)                  26 - GTPU (Admin only)
17 - PPP (Admin only)                     27 - GTPP (Admin only)
18 - A10 (Admin only)                     28 - DHCP (Admin only)
19 - User L3 (Admin only)                 29 - CDR (Admin only)
20 - USERTCP STACK (Admin only)           30 - DHCPV6 (Admin only)
31 - RADIUS COA (Admin only)              51 - SCTP (Admin only)
32 - MIP Tunnel (Admin only)              52 - M3UA (Admin only)
33 - L3 Tunnel (Admin only)               53 - SCCP (Admin only)
> 34 - CSS Data (Admin only)              54 - TCAP (Admin only)
35 - CSS Signaling (Admin only)           55 - MAP (Admin only)

The sample output from legacy non-CUPS PGW is here:

Sunday May 02 2021
<<<<OUTBOUND 11:22:22:165 Eventid:77000(9)
CSS Uplink Output PDU to ACS- slot:2 cpu:17 inst:4369
192.168.0.177 > 192.168.3.1: icmp: echo request (DF) (ttl 64, id 45402, len 84)

Sunday May 02 2021
***CONTROL*** 11:22:22:165 Eventid:77202
Rule matched : DEFAULT for uplink packet of subscriber MSID : 593440538564309

Sunday May 02 2021
INBOUND>>>>> 11:22:22:165 Eventid:77001(9)
CSS Uplink Input PDU from ACS- slot:3 cpu:34 inst:8738
192.168.0.177 > 192.168.3.1: icmp: echo request (DF) (ttl 64, id 45402, len 84)

But with CUPS UP, option 34 does not print packets anymore, just the rule matching messages.

Wednesday May 12 2021
***CONTROL*** 03:46:03:656 Eventid:77203
Rule matched : DEFAULT for uplink packet of subscriber CallID : 0x3e02061

Solution

The monitor functionality is re-written for CUPS UP because of the new architecture (Control and User Plane are separated, Vector Packet Processor (VPP) is involved, and so on). As a part of this change, option 34 does not print packets anymore. Instead, option 19 (User L3) will print the packet after any packet modification in ECS, as close to SGi as possible.

In summary, option 26 (GTPU) and option 19 (User L3) can be used to monitor user traffic.

Picture for how uplink/downlink packet appear in monitor subscriber options

In the uplink direction, to have a copy of a packet before packet modification, we need option 26 for GTPU. Option 19 should be the packet as seen on SGi, with header enrichment, TCP MSS modification and so on.

In the downlink direction, 2 copies of the packet were generated by option 19 and NO GTPU. This is intentional due to the fact that, in the outbound direction, the sessmgr does not see the GTPU encapsulated packet. The GTPU header is added by VPP after sessmgr processed the packet. Instead, the implementation is changed to print the post-ECS packet again by User L3 hence 2 copies of the packet are printed by option 19.

Sample output from both non-CUPS and CUPS are shown here for comparison.

CUPS:

Wednesday May 12 2021
INBOUND>>>>> 03:46:03:652 Eventid:142004(3)
GTPU Rx PDU, from fe80:192:168:1::3:2152 to fe80:10:200:1::100:2152 (92) TOS:0
TEID: 0x0000200E, Message type: GTP_TPDU_MSG (0xFF)
Sequence Number:: NA
GTP HEADER FOLLOWS:
Version number: 1
Protocol type: 1 (GTP C/U)
Message Type: 0xFF (GTP_TPDU_MSG)
Message Length: 0x0054 (84)
Tunnel ID: 0x0000200E
GTP HEADER ENDS.
Payload protocol: IPv4
PROTOCOL PAYLOAD FOLLOWS:
192.168.1.1 > 192.168.205.1: icmp: echo request (DF) (ttl 64, id 1521, len 84)
PROTOCOL PAYLOAD ENDS.

Wednesday May 12 2021
INBOUND>>>>> 03:46:03:652 Eventid:51000(0)
IPv4 Rx PDU
192.168.1.1 > 192.168.205.1: icmp: echo request (DF) (ttl 64, id 1521, len 84)

Wednesday May 12 2021
***CONTROL*** 03:46:03:656 Eventid:77203
Rule matched : DEFAULT for uplink packet of subscriber CallID : 0x3e02061

Wednesday May 12 2021
<<<<OUTBOUND 03:46:03:658 Eventid:51004(0)
IPv4 Tx PDU
192.168.205.1 > 192.168.1.1: icmp: echo reply (DF) (ttl 64, id 11245, len 84)

Wednesday May 12 2021
***CONTROL*** 03:46:03:658 Eventid:77203
Rule matched : DEFAULT for downlink packet of subscriber CallID : 0x3e02061

Wednesday May 12 2021
<<<<OUTBOUND 03:46:03:658 Eventid:51004(0)
IPv4 Tx PDU
192.168.205.1 > 192.168.1.1: icmp: echo reply (DF) (ttl 64, id 11245, len 84)

Legacy non-CUPS:

Sunday May 02 2021
INBOUND>>>>> 11:22:22:165 Eventid:142004(3)
GTPU Rx PDU, from 192.168.1.4:2152 to 192.168.10.1:2152 (92) TOS:0
TEID: 0x800CA001, Message type: GTP_TPDU_MSG (0xFF)
Sequence Number:: NA
GTP HEADER FOLLOWS:
Version number: 1
Protocol type: 1 (GTP C/U)
Message Type: 0xFF (GTP_TPDU_MSG)
Message Length: 0x0054 (84)
Tunnel ID: 0x800CA001
GTP HEADER ENDS.
Payload protocol: IPv4
PROTOCOL PAYLOAD FOLLOWS:
192.168.0.177 > 192.168.3.1: icmp: echo request (DF) (ttl 64, id 45402, len 84)
PROTOCOL PAYLOAD ENDS.

Sunday May 02 2021
INBOUND>>>>> 11:22:22:165 Eventid:51000(0)
IPv4 Rx PDU
192.168.0.177 > 192.168.3.1: icmp: echo request (DF) (ttl 64, id 45402, len 84)

Sunday May 02 2021
<<<<OUTBOUND 11:22:22:165 Eventid:77000(9)
CSS Uplink Output PDU to ACS- slot:2 cpu:17 inst:4369
192.168.0.177 > 192.168.3.1: icmp: echo request (DF) (ttl 64, id 45402, len 84)

Sunday May 02 2021
***CONTROL*** 11:22:22:165 Eventid:77202
Rule matched : DEFAULT for uplink packet of subscriber MSID : 593440538564309

Sunday May 02 2021
INBOUND>>>>> 11:22:22:165 Eventid:77001(9)
CSS Uplink Input PDU from ACS- slot:3 cpu:34 inst:8738
192.168.0.177 > 192.168.3.1: icmp: echo request (DF) (ttl 64, id 45402, len 84)

Sunday May 02 2021
<<<<OUTBOUND 11:22:22:166 Eventid:51004(0)
IPv4 Tx PDU
192.168.3.1 > 192.168.0.177: icmp: echo reply (ttl 64, id 56260, len 84)

Sunday May 02 2021
<<<<OUTBOUND 11:22:22:166 Eventid:77002(9)
CSS Downlink Output PDU to ACS- slot:2 cpu:17 inst:4369
192.168.3.1 > 192.168.0.177: icmp: echo reply (ttl 64, id 56260, len 84)

Sunday May 02 2021
***CONTROL*** 11:22:22:166 Eventid:77202
Rule matched : DEFAULT for downlink packet of subscriber MSID : 593440538564309

Sunday May 02 2021
INBOUND>>>>> 11:22:22:166 Eventid:77003(9)
CSS Downlink Input PDU from ACS- slot:3 cpu:34 inst:8738
192.168.3.1 > 192.168.0.177: icmp: echo reply (ttl 64, id 56260, len 84)

Sunday May 02 2021
<<<<OUTBOUND 11:22:22:166 Eventid:142005(3)
GTPU Tx PDU, from 192.168.10.1:2152 to 192.168.1.4:2152 (92) TOS:0
TEID: 0x60010896, Message type: GTP_TPDU_MSG (0xFF)
Sequence Number:: NA
GTP HEADER FOLLOWS:
Version number: 1
Protocol type: 1 (GTP C/U)
Message Type: 0xFF (GTP_TPDU_MSG)
Message Length: 0x0054 (84)
Tunnel ID: 0x60010896
GTP HEADER ENDS.
Payload protocol: IPv4
PROTOCOL PAYLOAD FOLLOWS:
192.168.3.1 > 192.168.0.177: icmp: echo reply (ttl 63, id 56260, len 84)
PROTOCOL PAYLOAD ENDS.