-
null
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter describes the Flow Recovery feature and provides detailed information on the following topics:
The Flow Recovery feature is introduced to support flow checkpoint for ECS rules. Flows of a rule can be recovered if the rule is made eligible. To make any rule eligible for checkpointing or recovery, the rule must be configured in the service scheme framework and identified based on rulename. Any type of rule can be eligible for checkpointing irrespective of the rule definition or protocol type. The rule type can be static/predefined, group-of-ruledef, dynamic, or ADC rule.
Flow Recovery is a licensed Cisco feature requiring a separate feature license. Contact your Cisco account representative for more information.
With the previous implementation, Cisco PCEF supports ICSR/SR check pointing at session level. Session and bearers were recovered as part of session recovery or ICSR switchover but flows were not recovered. Due to this when the packets were received on already established flows after the recovery, the packets were matched to default rules based on L3/L4 definition. This causes billing impact as flows were incorrectly charged.
If the flow recovery/checkpointing feature is enabled, then ICSR control outage might have a significant impact.
The enhancements with the Flow Recovery feature are listed below.
List of eligible rules for flow recovery: A configurable trigger action CLI flow recovery is added to configure flow-recovery under service scheme. The flows matching the configured rules in service scheme will be recovered. The flows will be checkpointed to AAAMgr when the delay timer for the flow matching these rules expires post the flow establishment.
Limit configuration for the number of flows recovered: The maximum number of flows to be checkpointed or recovered will be limited per subscriber using the system-limit flow-chkpt-per-call CLI at active-charging service level. This limit controls the number of flows that can be recovered across rules and bearers for a single call.
The number of flows checkpointed/recovered will be limited at per SessMgr instance level. This will be a non-configurable value.
The rule from recovered list is uninstalled for the session
The flow idle time-out happens
The TCP-based application flow is gracefully terminated when a RST packet is received at the gateway
The TCP-based application flow is not terminated when FIN packet is received at the gateway and cleared after the flow idle timeout happens
Handoff/ICSR scenario: After a rule is recovered for control events such that the bearer/rule information is not available (due to delay in EGTP message), the packets for these flows will match the default rule based on L3/L4 definition until transactions are complete. Once the control events are no longer pending, the packets will start matching the last matched rule from flow recovery list.
Similarly for ICSR, the packets for recovered flows will match the default rules based on L3/L4 definition. Once the information is received from new chassis, the packets will start matching the last matched rule from flow recovery list.
KPI and Bulk Statistics: KPI and Bulk Statistics are added on a per rulename basis for the rules that are eligible for checkpointing. When a rule is configured as eligible for recovery, KPI and bulk statistics for the rule will be displayed. For a rule that is deleted from the service-scheme framework, KPI and bulk statistics will not be displayed. SRP level statistics are also added to maintain the SRP bandwidth required for flow checkpoint.
This sections describes the relationship of the Flow Recovery feature with other supported features.
The following table describes the impact to inline services with the flow recovery implementation.
The behavior applies to recovered flows unless explicitly mentioned.
|
Inline Services |
Flow Recovery Impact |
Description |
|---|---|---|
|
CF Static and Dynamic |
No |
CF redirect and CF blacklist post recovery will not work and packets will be allowed. |
|
ICAP |
No |
Post recovery GET messages for the recovered flow will not be forwarded to ICAP server. |
|
Tethering Detection |
No |
For IP-TTL based tethering detection, the post recovery flow will be considered as tethered if the first packet post recovery is uplink packet. For OS-DB signature based tethering detection, the post recovery flow will be considered as tethered if the SYN packet received post recovery has matching signature in the database. |
|
P2P |
No |
P2P rules are not supported for recovery. |
|
Post processing rules |
No |
URL Redirect for recovered flows will not be supported based on post processing rules. The default action "Discard" will be applied. |
|
X-Header insertion |
No |
X-header insertion will not work for recovered rules. |
|
DSCP/IP-TOS Marking |
Yes |
Packets will be marked post recovery with correct DSCP/IP-TOS values. |
|
Idle-timeout handling |
Yes |
The flow is terminated after idle timeout and checkpoint limits are decremented. |
|
TCP based application Protocol |
No |
Analyzer will not be engaged for recovered flows. |
|
UDP Based application Protocol |
Yes |
Analyzer will be engaged for recovered flows if configured in rulebase. |
|
ICSR |
Yes |
Flows will be recovered to match the last match rule pre ICSR and for which checkpoint was successful. |
|
Session Recovery |
Yes |
Flows will be recovered to match the last match rule pre SR and for which checkpoint was successful. |
|
IPv6 |
Yes |
IPv6 flows will be recovered. |
The following table describes the behavior of flow actions with the flow recovery implementation.
This applies to recovered flows unless explicitly mentioned.
|
Flow Action |
Description |
|---|---|
|
Rate Limiting |
Per bearer rate limiting, per rule rate limiting, ITC rate limiting, or QG related rate limiting is applied. |
|
Allow/Deny packet |
Flow-action discard OR dynamic-rule flow-status enforcement is applied. |
|
URL redirect |
Run time URL redirection is not applied for a recovered flow. If the packets of a flow are redirected before recovery and the matching rule is eligible and checkpointed, packets are redirected post recovery as well. |
|
Flow Readdress |
Since the flow readdress is identified at SYN packet and SYN packets are not analyzed post recovery, flow readdress will not work post recovery. |
|
Flow Terminate |
Flow terminate action is applied for dynamic changes to charging action configuration. |
|
Next Hop |
Packets are forwarded to the next hop address. |
The following table describes the impact to ECS charging methods with the flow recovery implementation.
The behavior applies to recovered flows unless explicitly mentioned.
|
Charging Method |
Flow Recovery Impact |
|---|---|
|
eGCDR |
No |
|
PGW-CDR |
No |
|
VoGx |
No |
|
Gy |
No |
|
RF |
No |
|
EDR |
In case of recovery, generated EDR will show "Unknown" for sn-direction because the direction of first packet is not known. |
|
FDR |
No |
|
UDR |
No |
This section lists the limitations associated with the Flow Recovery/Checkpoint feature.
Few calls may be lost if session recovery occurs when the system is experiencing very high flow setup rates.
Checkpoint failures (failure counters being incremented) might be seen when large number of flows are being check pointed (>12K per seconds). The failed checkpoints are retried again. If a session recovery, unplanned card migration, or unplanned switchover occurs while the checkpoints are being retried, call loss may be seen.
Disabling of flow recovery feature (via CLI) while active calls (using flow recovery) are present might result in session manager restart.
Rule Matching – Packets are not matched to L7 rule after bearer movement from dedicated bearer to default bearer during LTE to WiFi HO. The same behavior will be seen post recovery.
Static rule match on dedicated bearer – After recovery, flow packets will always match the last matched rule. If the TFT are installed on dedicated bearer post recovery, such that the last match checkpointed rule is a static rule but the packets are received on dedicated bearer due to matching TFT, the packets are still matched to static rule but on dedicated bearer. Charging and QoS parameters are applied as per dedicated bearer.
EDR loss for SR/Unplanned ICSR – EDR is not generated in case of session recovery or unplanned ICSR.
Parent-child relation loss post recovery – The parent-child relationship post recovery is not maintained for protocols such as RTP and RTSP.
Lifetime of recovered flows – Lifetime will not be incremented for recovered flows for which no packet is received post recovery and is deleted due to flow idle timeout.
Service scheme framework – Flow checkpointing is dependent on the service scheme framework. Since this framework is introduced only in release 20.2, flow checkpointing for the existing calls/flows is supported only for the 20.2 build during upgrade scenarios. In all other upgrade scenarios, flow checkpointing will not be supported for the existing calls/flows.
This section describes the Flow Recovery configuration. The Service Scheme framework configuration is required to configure and enable this feature for a subscriber.
Use the sample configuration to enable the Flow Recovery feature.
configure active-charging service s1 trigger-action ta1 flow-recovery #exit trigger-condition tc1 rule-name = rule01 delay = 600 #exit service-scheme ss1 trigger flow-create priority 1 trigger-condition tc1 trigger-action ta1 #exit #exit end
Any TCP-based application protocol sends out mid-flow acknowledgements that are TCP packets. These packets are generally matched to a L3/L4 based rule on that bearer. In such cases a delete checkpoint will be sent whenever the L3/L4 rule is processed for the flow and again a checkpoint to AAAMgr is sent when the L7 rule is processed for the flow. This could result in high number of checkpoints sent/received and delete checkpoints sent/received. To avoid this scenario, delay charging must be configured for all packets or mid-flow packets in conjunction with flow-recovery for the rule under rulebase using the flow control-handshaking charge-to-application { all-packets | mid-session-packets } CLI configuration.
The rules eligible for checkpointing must be configured in the service-scheme framework and identified based on rulename. The rule type can be static/predefined, group-of-ruledef, dynamic or ADC rule. A group-of-ruledef can be independently made eligible for checkpointing even if the constituent ruledefs are not eligible for checkpoint. If both group-of-ruledef and constituent ruledefs are eligible for checkpoint, either one can be checkpointed based on the higher action priority configured in rulebase.
A delay timer to delay the checkpoint of a flow can be configured under the service-scheme trigger condition. This configuration controls the checkpoint sent/received (both checkpointing and deleting checkpoint) for flows which may last for a shorter duration. A new trigger action type "flow recovery" is also added to the service-scheme configuration.
Use the following configuration in the ACS Trigger Action Configuration mode to enable flow recovery for a trigger-action.
configure active-charging service <service_name> trigger-action <trigger_action_name> [ no ] flow-recovery exit
Notes:
Use the following configuration in the ACS Trigger Condition Configuration mode to define the list of rules along with the delay after which the flows can be checkpointed. When configured in conjunction with flow-recovery trigger, the flows for the rule(s) will be checkpointed as per session level and call level limit after the delay timer is expired.
configure active-charging service <service_name> trigger-condition <trigger_condn_name> delay = <delay_time> rule-name operator <rule_name> exit
Notes:
The range of the delay timer to be configured is 1 through 600 seconds. If the "delay" CLI command is not configured under trigger-condition, any flow for the rule will be checkpointed immediately on flow creation.
The operators "contains", "starts-with", and "ends-with" cannot be used with dynamic rule names. For dynamic rules, the entire rulename must be mentioned with the "=" operator.
rule_name must be an alphanumeric string of 1 through 63 characters.
In any defined trigger-condition, a user can configure upto a maximum of 15 entries.
To have more rules eligible for flow checkpoint, a user can configure multiple trigger condition(s) associated with the same trigger-action.
Use the no rule-name command to remove the particular rule from the list of eligible rules for flow checkpoint. For wildcard-based rule definition, this command must contain rulename in the same format.
Use the no delay command to checkpoint all eligible rules immediately without any delay.
Use the following configuration in the ACS Service Scheme Configuration mode to enable trigger for every new flow.
configure active-charging service <service_name> service-scheme <serv_scheme_name> [ no ] trigger flow-create exit
Use the following configuration in the ACS Configuration mode to control the number of flows that can be checkpointed per call.
configure active-charging service <service_name> system-limit flow-chkpt-per-call <num_flows> exit
Notes:
A new bulk statistic schema flow-kpi is added to maintain the KPI for flows. These counters will be the sum of the statistics for the corresponding rule(s) across session manager instance.
The Flow Recovery feature license is required to display KPI and configure bulk statistics.
Use the following configuration to configure the Flow KPI bulk statistic schema.
configure bulkstat mode file file_number flow-kpi schema schema_name format schema_format exit
Use the following configuration to view the Flow KPI bulk statistics in the Flow KPI schema.
show bulkstats variables flow-kpi
show active-charging flow-kpi [ all | instance instance_id ]
The sections listed below describe the KPI, Bulk statistics, and SRP level checkpointing statistics that can be used to identify if the checkpoints are sent and successful.
This section provides information regarding show commands and/or their outputs in support of the Flow Recovery feature.
The output of this command is modified to include a new flag to identify if the flow is a recovered flow or not. The value will display Y for a recovered flow and N for a non-recovered flow. The flag will be displayed if flow recovery license is present.
In support of the Flow Recovery feature, the show active-charging flow-kpi all CLI command is added to display the cumulative KPI for each rule across session managers and can also filter the statistics for all rules based on sessmgr instance. KPI are added on a per rule basis.
Rule Name
Active Flows
SR Flow Checkpoint Sent
SR Flow Checkpoint Received
GR Flow Checkpoint Sent
GR Flow Checkpoint Received
SR Flow Checkpoint Delete Sent
SR Flow Checkpoint Delete Received
GR Flow Checkpoint Delete Sent
GR Flow Checkpoint Delete Received
Flows of lifetime bucket1
Flows of lifetime bucket2
Flows of lifetime bucket3
The following field indicates whether flow recovery is enabled or disabled.
The following fields are added to the output of this command in support of this feature:
The SRP checkpoint statistics are added as part of checkpoint and recovery for ECS and ADC flows. Statistics for any flow checkpoint sent and received as well as delete checkpoint sent and received will be maintained.
ACS_FLOW_INFO and DEL_ACS_FLOW_INFO fields are displayed only after the creation/deletion of flow checkpoints respectively. When the flow checkpoints are sent from active chassis to standby chassis, the corresponding ACS_FLOW_INFO micro checkpoint statistics will be incremented. Similarly, after the expiry of idle timeout configured in the active-charging service, the flows will be deleted and the corresponding DEL_ACS_FLOW_INFO micro checkpoint statistics will be incremented.
A sample output is shown below. The rate limited MC checkpoints are displayed under "Checkpoints Rate limited" statistics and will be pegged only for ACS_FLOW_INFO and DEL_ACS_FLOW_INFO micro checkpoints.
show srp checkpoint statistics [OR] show srp checkpoint statistics sessmgr all SRP Session Checkpoint Sent Statistics ====================================== Checkpoints Rate limited-------------------------------------------------------+ Bytes sent------------------------------------------------------------------+ | External_audit_in_progress-----------------------------------------+ | | Recovery in progress--------------------------------------------+ | | | Write list congested-----------------------------------------+ | | | | Invalid GR state------------------------------------------+ | | | | | Session not ready for recovery-------------------------+ | | | | | | Encoding failure ignored----------------------------+ | | | | | | | Encoding failure---------------------------------+ | | | | | | | | Ignored as Invalidate CRR sent----------------+ | | | | | | | | | Ignored as first FC never sent-------------+ | | | | | | | | | | Checkpoint rate(last 30 min)------------+ | | | | | | | | | | | Checkpoint rate(now)-----------------+ | | | | | | | | | | | | Encoding attempted---------------+ | | | | | | | | | | | | | Messages sent--------------+ | | | | | | | | | | | | | | Checkpoint-type | | | | | | | | | | | | | | | UPDATE_CLPSTATS 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 ACS_SESS_INFO 2060 2060 0 0 0 0 0 0 0 0 0 0 0 298130 0 ACS_CALL_INFO 2060 2060 0 0 0 0 0 0 0 0 0 0 0 306240 0 PGW_UBR_MBR_INFO 1000 1000 0 0 0 0 0 0 0 0 0 0 0 457000 0 ACS_FLOW_INFO 30 50 0 0 0 0 0 0 0 0 0 0 0 8490 20 DEL_ACS_FLOW_INFO 30 50 0 0 0 0 0 0 0 0 0 0 0 6060 20 Full Checkpoint 1000 1000 0 0 0 0 0 0 0 0 0 0 0 4754470 0 ==================================================================================================
A new Flow KPI schema is added in support of the Flow Recovery feature.
The following bulk statistics are added in the Flow KPI Schema:
ecs-flow-rule-name – The name of rules eligible for flow checkpointing.
ecs-num-active-flow – Total number of active flows of the rule.
ecs-flow-chkpt-sent-sr – Total number of SR flow checkpoint sent for the rule.
ecs-flow-chkpt-recvd-sr – Total number of SR flow checkpoint received for the rule.
ecs-flow-chkpt-sent-gr – Total number of GR flow checkpoint sent for the rule.
ecs-flow-chkpt-recvd-gr – Total number of GR flow checkpoint received for the rule.
ecs-del-flow-chkpt-sent-sr – Total number of SR delete flow checkpoint sent for the rule.
ecs-del-flow-chkpt-recvd-sr – Total number of SR delete flow checkpoint received for the rule.
ecs-del-flow-chkpt-sent-gr – Total number of GR delete flow checkpoint sent for the rule.
ecs-del-flow-chkpt-recvd-gr – Total number of GR delete flow checkpoint received for the rule.
ecs-flow-lifetime-bucket1 – Total number of flows of lifetime_bucket1. Lifetime value of bucket1 is configurable.
ecs-flow-lifetime-bucket2 – Total number of flows of lifetime_bucket2. Lifetime value of bucket2 is configurable.
ecs-flow-lifetime-bucket3 – Total number of flows of lifetime_bucket3. Lifetime value of bucket3 is configurable.
Feedback