Interchassis Session Recovery

This chapter describes how to configure Interchassis Session Recovery (ICSR). The product Administration Guides provide examples and procedures for configuration of basic services on the system. You should select the configuration example that best meets your service model, and configure the required elements for that model as described in the respective product Administration Guide, before using the procedures described below.

IMPORTANT:

ICSR is a licensed Cisco feature that requires a separate license. Contact your Cisco account representative for detailed information on specific licensing requirements. For information on installing and verifying licenses, refer to the Managing License Keys section of Software Management Operations.

This chapter discusses the following:

Overview

The ICSR feature provides the highest possible availability for continuous call processing without interrupting subscriber services. ICSR allows the operator to configure geographically distant gateways for redundancy purposes. In the event of a node or gateway failure, ICSR allows sessions to be transparently routed around the failure, thus maintaining the user experience. ICSR also preserves session information and state.

ICSR is implemented through the use of redundant chassis. The chassis are configured as primary and backup, with one being active and one standby. Both chassis are connected to the same AAA server. A checkpoint duration timer controls when subscriber data is sent from the active chassis to the standby chassis. If the active chassis handling the call traffic goes out of service, the standby chassis transitions to the active state and continues processing the call traffic without interrupting the subscriber session.

The chassis determine which is active through a proprietary TCP-based connection known as the Service Redundancy Protocol (SRP) link. The SRP link is used to exchange Hello messages between the primary and backup chassis and must be maintained for proper system operation.

ICSR licenses are currently supported for the following services:
  • eHRPD – Evolved High Rate Packet Data
  • ePDG – Evolved Packet Data Gateway
  • GGSN – Gateway GPRS Support Node
  • HA – Home Agent
  • IPSG – IP Services Gateway
  • MME – Mobility Management Entity
  • P-GW – Packet Data Network Gateway
  • PDSN – Packet Data Serving Node
  • S-GW – Serving Gateway
  • SAEGW – System Architecture Evolution Gateway

IMPORTANT:

For releases prior to 17.0, ICSR should not be configured on chassis supporting L2TP calls.

IMPORTANT:

Contact your Cisco account representative to verify whether a specific service supports ICSR as an option.

Interchassis Communication

Chassis configured to support ICSR communicate using periodic Hello messages. These messages are sent by each chassis to notify the peer of its current state. The Hello message contains information about the chassis such as its configuration and priority. A dead interval is used to set a time limit for a Hello message to be received from the chassis’ peer. If the standby chassis does not receive an Hello message from the active chassis within the dead interval, the standby chassis transitions to the active state. In situations where the SRP link goes out of service, a priority scheme is used to determine which chassis processes the session. The following priority scheme is used:

  • route modifier
  • chassis priority
  • MIO/UMIO MAC address

Checkpoint Messages

Checkpoint messages are sent from the active chassis to the standby chassis. These messages are sent at specific intervals and contain all the information needed to recreate the sessions on the standby chassis, if that chassis were to become active. Once a session exceeds the checkpoint duration, checkpoint data is collected on the session.

AAA Monitor

AAA servers are monitored using the authentication probe mechanism. AAA servers are considered Up if the authentication-probe receives a valid response. AAA servers are considered Down when the max-retries count specified in the configuration of the AAA server has been reached. SRP initiates a switchover when none of the configured AAA servers responds to an authentication probe. AAA probing is only performed on the active chassis.

IMPORTANT:

A switchover event caused by an AAA monitoring failure is non-revertible.

If the newly active chassis fails to monitor the configured AAA servers, it remains as the active chassis until one of the following occurs:
  • a manual switchover
  • another non-AAA failure event causes the system to switchover
  • a CLI command is used to clear the AAA failure flag and allow the chassis to switch to standby

BGP Interaction

The Service Redundancy Protocol implements revertible switchover behavior via a mechanism that adjusts the route modifier value for the advertised loopback/IP Pool routes. The initial value of the route modifier value is determined by the chassis’ configured role and is initialized to a value that is higher than a normal operational value. This ensures that in the event of an SRP link failure and an SRP task failure, the correct chassis is still preferred in the routing domain.

The Active and Standby chassis share current route modifier values. When BGP advertises the loopback and IP pool routes, it converts the route modifier into an autonomous systems (AS) path prepend count. The Active chassis always has a lower route modifier, and thus prepends less to the AS-path attribute. This causes the route to be preferred in the routing domain.

If communication on the SRP link is lost, and both chassis in the redundant pair are claiming to be Active, the previously Active chassis is still preferred since it is advertising a smaller AS-path into the BGP routing domain. The route modifier is incremented as switchover events occur. A threshold determines when the route modifier should be reset to its initial value to avoid rollover.

Requirements

ICSR configurations require the following:

  • Two chassis configured for the same service types. The services must be bound on an SRP-activated loopback interface.
  • Both chassis must have identical hardware.
  • Three contexts:
    • Redundancy – to configure the primary and backup chassis redundancy.
    • Source – AAA configuration of the specified nas-ip-address must be the IP address of an interface bound to an HA, or any core network service configured within the same context.
    • Destination – to configure monitoring and routing to the PDN.
  • AAA RADIUS server
  • Border Gateway Protocol (BGP) – ICSR uses the route modifier to determine the chassis priority.

IMPORTANT:

ICSR is a licensed Cisco feature. Verify that each chassis has the appropriate license before using these procedures. To do this, log in to both chassis and execute a show license information command. Look for “Inter-Chassis Session Recovery”. If the chassis is not licensed, please contact your Cisco account representative.

The following figure shows an ICSR network.


Figure 1. ASR 5500 ICSR Network

ICSR Operation

This section shows operational flows for ICSR.

The following figure shows an ICSR process flow due to a primary failure.


Figure 2. ICSR Process Flow (Primary Failure)

The following figure shows an ICSR process flow due to a manual switchover.


Figure 3. ICSR Process Flow (Manual Switchover)

Chassis Initialization

When the chassis are simultaneously initialized, they send Hello messages to their configured peer. The peer sends a response, establishes communication between the chassis, and messages are sent that contain configuration information.

During initialization, if both chassis are misconfigured in the same mode - both active (primary) or both standby (backup), the chassis with the highest priority (lowest number set with the ICSR priority command) becomes active and the other chassis becomes the standby.

If the chassis priorities are the same, the system compares the two MAC addresses and the chassis with the higher MIO/UMIO MAC address becomes active. For example, if the chassis have MAC addresses of 00-02-43-03-1C-2B and 00-02-43-03-01-3B, the last 3 sets of octets (the first 3 sets are the vendor code) are compared. In this example, the 03-1C-2B and 03-01-3B are compared from left to right. The first pair of octets in both MAC addresses are the same, so the next pairs are compared. Since the 01 is lower than the 1C, the chassis with the MIO/UMIO MAC address of 00-02-43-03-1C-2B becomes active and the other chassis the standby.

Chassis Operation

This section describes how the chassis communicate, maintain subscriber sessions, and perform chassis switchover.

Chassis Communication

If one chassis in the active state and one in the standby state, they both send Hello messages at each hello interval. Subscriber sessions that exceed the checkpoint session duration are included in checkpoint messages that are sent to the standby chassis. The checkpoint message contains subscriber session information so if the active chassis goes out of service, the backup chassis becomes active and is able to continue processing the subscriber sessions. Additional checkpoint messages occur at various intervals whenever subscriber session information is updated on the standby chassis.

For additional information see the Service Redundancy Protocol Configuration Mode Commands chapter in the Command Line Interface Reference.

Chassis Switchover

If the active chassis goes out of service, the standby chassis continues to send Hello messages. If the standby chassis does not receive a response to the Hello messages within the dead interval, the standby chassis initiates a switchover. During the switchover, the standby chassis begins advertising its srp-activated loopback and pool routes into the routing domain. Once the chassis becomes active, it continues to process existing AAA services and subscriber sessions that had checkpoint information, and is also able to establish new subscriber sessions.

When the primary chassis is back in service, it sends Hello messages to the configured peer. The peer sends a response, establishes communication between the chassis, and sends Hello messages that contain configuration information. The primary chassis receives an Hello message that shows the backup chassis state as active and then transitions to standby. The Hello messages continue to be sent to each peer, and checkpoint information is now sent from the active chassis to the standby chassis at regular intervals.

When chassis switchover occurs, the session timers are recovered. The access gateway session recovery is recreated with the full lifetime to avoid potential loss of the session and the possibility that a renewal update was lost in the transitional checkpoint update process.

Configuring Interchassis Session Recovery (ICSR)

IMPORTANT:

The ICSR configuration must be the same on the primary and backup chassis. If each chassis has a different Service Redundancy Protocol (SRP) configuration, the session recovery feature does not function and sessions cannot be recovered when the active chassis goes out of service.

This section describes how to configure basic ICSR on each chassis. For information on commands that configure additional parameters and options, refer to the Command Line Interface Reference.

CAUTION:

ICSR should not be configured for chassis supporting L2TP calls.

The procedures described below assume the following:

  • The chassis have been installed and configured with core network services.
    For more configuration information and instructions on configuring services, refer to the respective product Administration Guide.
  • In addition, the IP address pools must be srp activated.
  • AAA server is installed, configured and accessible by both chassis.
    For more information on configuring the AAA server, refer to the AAA Interface Administration and Reference.
  • BGP router installed and configured. See Routing for more information on configuring BGP services.

To configure the ICSR on a primary and/or backup chassis:

  1. Configure the SRP context by applying the example configuration in Configuring the Service Redundancy Protocol (SRP) Context.
  2. Modify the source context of the core network service by applying the example configuration in Modifying the Source Context for ICSR.
  3. Modify the destination context of core network service by applying the example configuration in Modifying the Destination Context for ICSR.
  4. Optional: Disable bulk statistics collection on the standby system by applying the example configuration in Disabling Bulk Statistics Collection on a Standby System.
  5. Verify your primary and backup chassis configuration as described in Verifying the Primary and Backup Chassis Configuration.
  6. Save your configuration as described in Verifying and Saving Your Configuration.

Configuring the Service Redundancy Protocol (SRP) Context

To configure the system to work with ICSR:

  1. Create the chassis redundancy context and bind it to the IP address of the primary chassis by applying the example configuration in Creating and Binding the SRP Context.
  2. Configure the chassis redundancy context with priority, chassis mode, hello interval, dead-interval and peer IP address by applying the example configuration in Configuring SRP Context Parameters
  3. Configure the SRP context with interface parameters (including interface name, IP address and port number) for interchassis communication by applying the example configuration in Configuring the SRP Context Interface Parameters.
  4. Verify your SRP context configuration as described in Verifying SRP Configuration.
  5. Save your configuration as described in Verifying and Saving Your Configuration.

Creating and Binding the SRP Context

Use the example below to create the SRP context and bind it to primary chassis IP address:

IMPORTANT:

ICSR is configured using two systems. Be sure to create the redundancy context on both systems. CLI commands must be executed on both systems. Log onto both chassis before continuing. Always make configuration changes on the primary chassis first. Before starting this configuration, identify which chassis to configure as the primary and use that login session.

configure
  context srp_ctxt_name [-noconfirm]
    service-redundancy-protocol
      bind address ip_address
      end

Notes:

  • ICSR should be configured and maintained in a separate context.
  • Be sure to bind the local IP address to the primary chassis. When configuring the backup chassis, be sure to bind the local IP address to the backup chassis.

Configuring SRP Context Parameters

IMPORTANT:

CLI commands must be executed on both chassis. Log onto both chassis before continuing. Always make configuration changes on the primary chassis first.

Basic Parameters

This configuration assigns a chassis mode and priority, and also configures the redundancy link between the primary and backup chassis:

configure
  context srp_ctxt_name
    service-redundancy-protocol
      chassis-mode { primary | backup }
      priority priority
      peer-ip-address ip_address
      hello-interval dur_sec
      dead-interval dead_dur_sec
      end

Notes:

  • ICSR should be configured and maintained in a separate context.
  • When assigning the chassis mode on the backup chassis be sure to enter the backup keyword.
  • The priority determines which chassis becomes active in the event that both chassis are misconfigured with the same chassis mode; see Chassis Initialization. The higher priority chassis has the lower number. Be sure to assign different priorities to each chassis.
  • Enter the IP chassis of the backup chassis as the peer-ip-address to the primary chassis. Assign the IP address of the primary chassis as the peer-ip-address to the backup chassis.
  • The dead-interval must be at least three times greater than the hello-interval. For example, if the hello interval is 10, the dead interval should be at least 30. System performance is severely impacted if the hello interval and dead interval are not set properly. An optional delay-interval command allows you to delay the start dead-interval for an interval following the loading of configuration files.

Configuring the SRP Context Interface Parameters

This procedure configures the communication interface with the IP address and port number within the SRP context. This interface supports interchassis communication.

IMPORTANT:

CLI commands must be executed on both chassis. Log onto both chassis before continuing. Always make configuration changes on the primary chassis first.

configure
  context vpn_ctxt_name [-noconfirm]
      interface srp_if_name
      ip-address { ip_address | ip_address/mask }
      exit
    exit
  port ethernet slot_num/port_num
    description des_string
    medium { auto | speed { 10 | 100 | 1000 } duplex { full | half } }
    no shutdown
    bind interface srp_if_name srp_ctxt_name 
    end

Verifying SRP Configuration

Verify that your SRP contexts were created and configured properly by running the show srp info command (Exec Mode) on each chassis.

Notes:

  • The interval is specified as an integer divisible by 15 in the range from 30 through 1440 (Default = 45 minutes). The interval range for sending full checkpoints is 30 minutes to 24 hours (1140 minutes).

Modifying the Source Context for ICSR

To modify the source context of core service:

  1. Add the Border Gateway Protocol (BGP) router AS-path and configure the gateway IP address, neighbor IP address, remote IP address in the source context where the core network service is configured, by applying the example configuration in Configuring BGP Router and Gateway Address.
  2. Configure the service redundancy context with the BGP neighbor context and IP address to monitor the BGP link activity by applying the example configuration in Configuring the SRP Context for BGP.
  3. Verify your BGP context configuration by following the steps in Verifying BGP Configuration.
  4. Save your configuration as described in Verifying and Saving Your Configuration.

Configuring BGP Router and Gateway Address

Use the following example to create the BGP context and network addresses.

configure
  context source_ctxt_name
    router bgp AS_num
      network gw_ip_address
      neighbor neighbor_ip_address remote-as AS_num
      end

Notes:

  • source_ctxt_name is the context where the core network service is configured.

Configuring the SRP Context for BGP

Use the following example to configure the BGP context and IP addresses in the SRP context.

configure
  context srp_ctxt_name
    service-redundancy-protocol
      monitor bgp context source_ctxt_name neighbor_ip_address
      end

neighbor_ip_address can be entered in IPv4 dotted-decimal or IPv6 colon-separated-hexadecimal notation. Multiple IP addresses can be added per context as IPv4 and/or IPv6 IP addresses.

An ICSR failover is triggered when all BGP peers within a context are down.

Optionally, you can configure SRP peer groups within a context. ICSR failover would then occur if all peers within a group fail. This option is useful in deployments in which a combination of IPv4 and IPv6 peers are spread across multiple paired VLANs, and IPv4 or IPv6 connectivity is lost by all members of a peer group.

A sample configuration for SRP peer groups within a context (“PGWin”) appears below.

monitor bgp context
PGWin 10.1.1.16 group 1
monitor bgp context
PGWin 10.1.1.17 group 1
monitor bgp context
PGWin 69.2.215.0 group 2
monitor bgp context
PGWin 69.2.215.1 group 2
monitor bgp context
PGWin 2001:4333:201:1102:103:2a1:: group
3
monitor bgp context
PGWin 2001:4333:201:1102:103:2a1:0:1 group
3

In the above sample configuration, ICSR failover would occur if both addresses in group 1, 2 or 3 lost connectivity.

For additional information refer to the description of the monitor bgp, monitor diameter and monitor authentication-probe commands in the Service Redundancy Protocol Configuration Mode Commands chapter of the Command Line Interface Reference.

Verifying BGP Configuration

Verify your BGP configuration by entering the show srp monitor bgp command (Exec Mode).

Modifying the Destination Context for ICSR

To modify the destination context of core service:

  1. Add the BGP router and configure the gateway IP address, neighbor IP address, remote IP address in the destination context where the core network service is configured, by applying the example configuration in Configuring BGP Router and Gateway Address in Destination Context.
  2. Configure the service redundancy context with BGP neighbor context and IP address to monitor the BGP link activity by applying the example configuration in Configuring SRP Context for BGP for Destination Context.
  3. Set the subscriber mode to default by following the steps in Setting Subscriber to Default Mode.
  4. Verify your BGP context configuration by following the steps in Verifying BGP Configuration in Destination Context.
  5. Save your configuration as described in Verifying and Saving Your Configuration.

Configuring BGP Router and Gateway Address in Destination Context

Use the following example to create the BGP context and network addresses.

configure
  context dest_ctxt_name
    router bgp AS_num
      network gw_ip_address
      neighbor neighbor_ip_address remote-as AS_num
      end

Notes:

  • AS_num is the autonomous systems path number for this BGP router.

Configuring SRP Context for BGP for Destination Context

Use the following example to configure the BGP context and IP addresses in the SRP context.

configure
  context srp_ctxt_name
    service-redundancy-protocol
      monitor bgp context dest_ctxt_name neighbor_ip_address
      end

Setting Subscriber to Default Mode

Use the following example to set the subscriber mode to default.

configure
  context dest_ctxt_name
    subscriber default
      end

Verifying BGP Configuration in Destination Context

Verify your BGP configuration by entering the show srp monitor bgp command (Exec Mode).

Disabling Bulk Statistics Collection on a Standby System

You can disable the collection of bulk statistics from a system when it is in the standby mode of operation.

IMPORTANT:

When this feature is enabled and a system transitions to standby state, any pending accumulated statistical data is transferred at the first opportunity. After that no additional statistics gathering takes place until the system comes out of standby state.

Use the following example to disable the bulk statistics collection on a standby system.
configure
  bulkstat mode
    no gather-on-standby
    end

Repeat this procedure for both systems.

Verifying the Primary and Backup Chassis Configuration

This section describes how to compare the ICSR configuration on both chassis.

  1. Enter the show configuration srp command on both chassis (Exec mode). Verify that both chassis have the same SRP configuration information. The output looks similar to following:
    config
    
      context source
    
        interface haservice
    loopback
    
          ip address 172.17.1.1
    255.255.255.255 srp-activate
    
        #exit
    
        radius attribute
    nas-ip-address address 172.17.1.1
    
        radius server
    192.168.83.2 encrypted key 01abd002c82b4a2c
    port 1812
    
        radius accounting
    server 192.168.83.2 encrypted key 01abd002c82b4a2c
    port 1813
    
        ha-service
    ha-pdsn
    
          mn-ha-spi
    spi-number 256  encrypted secret 6c93f7960b726b6f6c93f7960b726b6f
    hash-algorithm md5
    
          fa-ha-spi
    remote-address 192.168.82.0/24
    spi-number 256 encrypted secret 1088bdd6817f64df
    
          bind address
    172.17.1.1
    
        #exit
    
      #exit
    
      context destination
    
        ip pool dynamic
    172.18.0.0 255.255.0.0
    public 0 srp-activate
    
        ip pool static
    172.19.0.0 255.255.240.0
    static srp-activate
    
      #exit
    
      context srp
    
        service-redundancy-protocol
    
        #exit
    
      #exit
    

Configuring Subscriber State Management Audit Process

This audit is to ensures that two ICSR peers are in synch and identifies any discrepancies prior to any scheduled or unscheduled switchover events.

  1. Enter the SRP Context mode and enter the service-redundancy-protocol command.
  2. Enter the audit daily-start-time command. Specify the daily start time as an hour and minute. For example, a start time of 06 00 indicates that the audit will begin at 6:00 AM.
  3. Enter the audit periodicity command. Specify the interval in minutes for generating SRP audit statistics as an integer from 60 through 1440. For example, a periodicity of 90 indicates that SRP audit statistics will be generated every 90 minutes beginning at the specified start time. Default = 60. A sample configuration sequence appears below.
    config
    
      context srp
    
        service-redundancy-protocol
    
          audit daily-start-time
    06 00
    
          audit periodicity 90
    
          end
    

Updating the Operating System

Updating the operating system (StarOS™) on ICSR chassis requires performing an Off-line update of each chassis while it is standby mode. Traffic disruption is minimal since an active chassis will be handling call sessions while the standby chassis is being updated.

The general upgrade sequence is as follows:
  • Download the StarOS software image and copy/transfer it to both chassis.
  • Save the currently running configurations on both chassis.
  • Update the standby backup chassis first.
  • Initiate an SRP switchover from the active primary chassis to make the standby backup chassis active.
  • Update the standby primary chassis.
  • Initiate an SRP switchover from the active backup chassis to make the standby primary chassis active.

The four-part flowchart below shows a more complete view of all the procedures required to complete the StarOS upgrade process.

CAUTION:

Enabling the Demux on MIO/UMIO feature changes resource allocations within the system. This directly impacts an upgrade or downgrade between StarOS versions in ICSR configurations. Contact Cisco TAC for procedural assistance prior to upgrading or downgrading your ICSR deployment.


Figure 4. ICSR Software Upgrade – Part 1

Figure 5. ICSR Software Upgrade – Part 2

Figure 6. ICSR Software Upgrade – Part 3

Figure 7. ICSR Software Upgrade – Part 4

Both ICSR Chassis

Log into the CLI of the primary and backup and perform the tasks described below.

Downloading and Transferring the StarOS Build

  1. Verify that there is enough free space on the /flash device to accommodate the new operating system image file by entering the following Exec mode command:
    [local]host_name# directory /flash
    
  2. Access to the Cisco support site and download facility is username and password controlled. Download the software image to a network location or physical device (USB stick) from which it can be uploaded to the /flash device.
  3. Transfer the new operating system image file to the /flash device on the MIO/UMIO using one of the following methods:
    1. Copy the file from a network location or local device plugged into the MIO/UMIO using the copy command
      [local]host_name# copy from_url to_url [-noconfirm]
      
    2. Transfer the file to the /flash device using an FTP client with access to the system. The FTP client must be configured to transfer the file using binary mode.
    3. Transfer the file to the /flash device using an SFTP client with access to the system.
  4. Verify that the image file was successfully transferred to the /flash device by running the Exec mode the following command
    [local]host_name# directory /flash
    
  5. Run the show version /flash/image_filename command to verify the build information. For example:
    [local]host_name# show version /flash/image_filename.bin
    

    Note:

    Any CRC errors will be displayed in the output of the above command. If any errors appear, check the build and re-transfer it onto the chassis. Confirm that the correct image version and build description is displayed

Standby Backup Chassis

Log into the backup standby chassis and perform the tasks described below.

Performing Health Checks

Health checks are a series of Exec mode show commands to determine the readiness of the system to handle a software update.

  1. Run show card table all |grep unknown. No output should be displayed.
  2. Run show card table |grep offline. No output should be displayed.
  3. Run show resources |grep Status. The output should display “Within acceptable limits”.
  4. Run show alarm outstanding. Review the output for any issues that may preclude performing the software update.

Performing SRP Checks

Service Redundancy Protocol (SRP) checks verify that the mechanism for monitoring ICSR system status is operational.

  1. Run show srp monitor all.
  2. Review the output for any issues that may preclude performing the software update.

Performing BGP Checks

Border Gateway Protocol (BGP) checks are only required when BGP is used to support redundant interchassis communication. These checks are run per context and per service type.

  1. For each BGP-enabled context, run show ip bgp summary. Verify that the BGP peers are connected and IPv4 and IPv6 peers are up. Repeat for all BGP-enable contexts.
  2. Run show service_name all |grep "Service Status:". The service should be “Started”. Repeat for all services running on the chassis.

Updating the Boot Record

You must add a new boot stack entry for the recently downloaded software image (.bin) file.

  1. Run the Exec mode show boot command to verify that there are less than 10 entries in the boot.sys file and that a higher priority entry is available (minimally there is no priority 1 entry in the boot stack).
  2. Create a new boot stack entry for the new file group, consisting of the new operating system image file and the currently used CLI configuration file by entering the following Global Configuration command:
    [local]host_name(config)# boot system priority number  image image_url /flash/filename config cfg_url /flash/filename
    
  3. Assign the next highest priority to this entry, by using the <N-1> method, wherein you assign a priority number that is one number less than your current highest priority. If priority 1 is in use, you must renumber the existing entries to ensure that at least that priority is available. The maximum number of boot stack entries that can be contained in the boot.sys file is 10. If there are already 10 entries in the boot stack, you must delete at least one of these entries (typically, the lowest priority) and, if necessary, renumber some or all of the other entries before proceeding. Use the no boot system priority command to delete a book stack entry. For information on using the boot system priority command, refer to the Adding a New Boot Stack Entry section in this guide

Synchronizing File Systems

Synchronize the local file systems by entering the following Exec mode command:
[local]host_name# filesystem synchronize
all 

Reloading the Chassis

Reboot the chassis by entering the following command:
[local]host_name# reload [-noconfirm]

As the system reboots, it loads the new operating system software image and its corresponding CLI configuration file using the new boot stack entry configured earlier.

After the system reboots, establish a CLI session and enter the show version command to verify that the active software version is correct.

Optional for PDSN: If you are using the IP Pool Sharing Protocol during your upgrade, refer to Configuring IPSP Before the Software Upgrade in the PDSN Administration Guide.

Updating the Configuration File

Features in the new operating system may require changes tot he configuration file. These changes can be done manually or facilitated by custom scripts prepared by Cisco TAC. Make whatever changes are necessary prior to saving the updated configuration file.

Verifying the Software Version

After the system has successfully booted, verify that the new StarOS version is running by executing the Exec mode show version command.

Saving the Configuration File

Use the Exec mode save configuration command to save the currently running configuration to the /flash device and to an off-chassis location (external memory device or network URL). The off-chassis copy assures that you will have a fallback, loadable configuration file should a problem be encountered.

Completing the Update Process

Repeat the following tasks to complete the upgrade process on the standby secondary chassis:

Waiting for Session Synchronization

Allow time for session synchronization to occur between the ICSR chassis before preceding to the next steps.

  1. Run the show session recovery status verbose command on both chassis. Proceed to the next steps only when no errors are seen in the output of this command.
  2. On the standby chassis, run show srp checkpoint statistics |more.
  3. On active chassis, run show subs summary |grep Total.
  4. Compare the number of subscribers on the active chassis and the number of Current pre-allocated calls: on the standby chassis. They should be similar (within 5%). Allow a few minutes for systems to complete synchronization.

Primary Chassis

Log into the active primary chassis and complete the tasks described below.

Initiating an SRP Switchover

An SRP switchover places the primary chassis in standby mode and makes the backup chassis active. The secondary chassis is now processing sessions with the upgraded software.

  1. On the primary chassis run the srp initiate-switchover command. All existing sessions will be migrated to the backup chassis and it begins servicing new session requests. Allow the switchover process to complete.
  2. On the primary chassis, run the show srp info command. Chassis State should indicate Standby when switchover is complete.
  3. On the backup chassis, confirm the switchover is complete by running the show srp info command. Chassis State should indicate Active when switchover is complete.

Checking AAA Monitor Status on the Newly Active Chassis

If your network deployment requires communication with AAA servers, log into the newly active chassis and perform an AAA monitor check. You will be checking for the existence of any SNMP traps that indicate the chassis cannot communicate with AAA servers (starSRPAAAUnreachable).

  1. Run the Exec mode command show snmp trap history |grep starSRPAAAUnreachable.
  2. There should be no output for this command, or no very recent SNMP trap notifications (based on the event timestamp).
  3. If the active chassis cannot communicate with one or more AAA servers, refer to Checking AAA Monitor Status for additional information on how to proceed.

Completing the Software Update

Initiating an SRP Switchover

An SRP switchover places the primary chassis in active mode and makes the backup chassis active. The primary chassis is now processing sessions with the upgraded software.

  1. On the backup chassis run the srp initiate-switchover command. All existing sessions will be migrated to the primary chassis and it begins servicing new session requests. Allow the switchover process to complete.
  2. On the backup chassis, run the show srp info command. Chassis State should indicate Standby when switchover is complete.
  3. On the primary chassis, confirm the switchover is complete by running the show srp info command. Chassis State should indicate Active when switchover is complete.

Checking AAA Monitor Status

If your network deployment requires communication with AAA servers, check the status of communication with AAA servers as described in Checking AAA Monitor Status on the Newly Active Chassis.

Making Test Calls

Once the chassis state is verified and subscribers are migrated, perform new call testing to make sure calls are successful.

Fallback Procedure

To revert to the previous configuration and software build, perform the following steps as a user with administrative privileges.

  1. Run the Exec mode show boot command. The topmost lowest numbered entry of the displayed output should be the new configuration with the new software build. The second topmost entry should be the backup configuration.
  2. Remove the topmost boot entry n, and synchronize the configuration across the management cards.
    [local]host_name# config
    
    [local]host_name(config)# no boot system priority n
    
    [local]host_name(config)# end
    
    [local]host_name# filesystem synchronize all
    
  3. Reboot the system to load its previous configuration.
    [local]host_name# reload
    
  4. Perform health checks as described in Performing Health Checks.