Cisco IOS Firewall Stateful Failover

It is recommended that you implement stateful failover in one of the following recommended deployment scenarios:

• Dual LAN Interface
• LAN WAN Interface

In a dual LAN interface scenario, the active and standby routers running the firewall are connected to each other via LAN interface on both the inside and the outside (see the figure below). HSRP is configured on both the inside and outside interface. The next hop routers in this scenario talk to the HA pair via the virtual IP address. In this scenario there are two virtual IP address, one on the inside and the other on the outside. Virtual IP addresses cannot be advertised using routing protocols. You need to create static routes on the next hops to get to the virtual IP address.

You need to configue HSRP tracking in order to track multiple pairs of interfaces. If you run HSRP on only one pair of interfaces, or run on both without mutual tracking of the pairs, each pair functions independently of each other and are unaware of each other's state changes. For example, if HSRP is run on only the two outside interfaces (as shown in the figure below), this could cause HSRP to failover on the outside interfaces, whilst the inside interfaces are unchanged. This causes the black holing of traffic, which continues to be directed to the primary from the inside. This introduces the possibility of problems arising from one interface on a primary router failing and triggering a move to the secondary, while the other interface on the ex-primary remains active. Mutual tracking means that if the outside interface does fail, the inside interface on the same router will also be deemed down allowing for complete router failover to the secondary.

Figure 1

Dual Interface Network Topology

In a LAN WAN scenario, the inside interface of the Active Standby pair running the firewall are connected via LAN interface on the inside and WAN interface on the outside (see the figure below). HSRP is configured on the inside interface. The inside network communicates with the HA pair using the inside virtual IP address.

HSRP tracking should be configured on the inside LAN interfaces to track the state of the outside WAN interface. If the outside WAN interface goes down on the active the LAN interface that is tracking it reduces the HSRP priority and initiates a failover to the standby. Traffic from the outside flowing into the HSRP pair should now be directed to the new active device.

In the scenario where the LAN interfaces track the WAN interfaces, the failover to the standby happens immediately. However, for traffic to start flowing on the new active router, routing convergence needs to happen. The net failover time is dictated by the routing protocol.

In this scenario the traffic flows from the inside to the outside through the Active due to the HSRP configuration on the inside LAN interfaces. The traffic from the outside to the inside should also flow through the active device. The configuration of the network so that the traffic always flows through the active is beyond the scope of this document. In this scenario, the network administrator is responsible to ensure that the traffic always flows through the active device.

Firewall stateful failover is a client of Cisco IOS SSO. SSO is a method of providing redundancy and synchronization for Cisco IOS applications and features.

• State Synchronization
  
• Bulk Synchronization
  

HSRP provides two services--IP redundancy and a Virtual IP (VIP) address. Each HSRP group may provide either or both of these services. Cisco IOS firewall stateful failover uses the IP redundancy services from only one HSRP standby group. It can use the VIP address from one or more HSRP groups. Use the following task to configure HSRP on the outside and inside interfaces of the router.

Note	Perform this task on both routers (active and standby) and on both interfaces of each router.

Before You Begin

If a switch connects the active and standby routers, you must perform one of the following steps to ensure that the correct settings are configured on that switch:

• Enable the spanning-tree portfast command on every switch port that connects to a HSRP-enabled router interface.
• Disable the Spanning Tree Protocol (STP) on the switch only if your switch does not connect to other switches. Disabling spanning tree in a multi-switch environment may cause network instability.
• Enable the standby delay minimum [min-delay] reload [reload-delay] command if you do not have access to the switch. The reload-delay argument should be set to a value of at least 120 seconds. This command must be applied to all HSRP interfaces on both routers.

For more information on HSRP instability, see the document Avoiding HSRP Instability in a Switching Environment with Various Router Platforms .

Note	You must perform at least one of these steps for correct HSRP operation.

Note

Both the inside (private) interface and the outside (public) interface must belong to separate HSRP groups, but the HSRP group number can be the same. The state of the inside interface and the outside interface must be the same--both interfaces must be in the active state or standby state; otherwise, the packets will not have a route out of the private network. Standby priorities should be equal on both active and standby routers. If the priorities are not equal, the higher priority router will unnecessarily take over as the active router, negatively affecting uptime. The IP addresses on the HSRP-tracked interfaces of the standby and active routers should both be either lower or higher on one router than the other. In the case of equal priorities (an HA requirement), HSRP will assign the active state on the basis of the IP address. If an addressing scheme exists so that the public IP address of Router A is lower than the public IP address of Router B, but the opposite is true for their private interfaces, an active/standby-standby/active split condition could exist which will break connectivity. Interface ACL should allow HSRP traffic to flow through. Each time an active device relinquishes control to become the standby device, the active device will reload. This functionality ensures that the state of the new standby device synchronizes correctly with the new active device.

1.    enable

2.    configure terminal

3.    interface type number

4.    standby standby-group-number name standby-group-name

5.    standby standby-group-number ip ip-address

6.    standby standby-group-number track interface-name

7.    standby [group-number] preempt

8.    standby [group-number] timers [msec] hellotime [msec] holdtime

9.    standby delay minimum [min-delay] reload [reload-delay]

10.    Repeat.

interface Ethernet0/0
 ip address 209.165.201.1 255.255.255.224
 standby 1 ip 209.165.201.3
 standby 1 preempt
 standby 1 name HA-out
 standby 1 track Ethernet1/0
 standby delay minimum 120 reload 120

• Troubleshooting Tips
  
• What to Do Next
  

1.    enable

2.    configure terminal

3.    redundancy inter-device

4.    scheme standby standby-group-name

5.    exit

6.    ipc zone default

7.    association 1

8.    protocol sctp

9.    local-port local-port-number

10.    local-ip device-real-ip-address [device-real-ip-address2

11.    retransmit-timeout retran-min [msec] retran-max [msec]

12.    path-retransmit max-path-retries

13.    assoc- retransmit retries

14.    exit

15.    remote-port remote-port-number

16.    remote-ip peer-real-ip-address [peer-real-ip-address2

!
redundancy inter-device
 scheme standby HA-in
!
!
ipc zone default
 association 1
  no shutdown
  protocol sctp
   local-port 5000
    local-ip 10.0.0.1
    retransmit-timeout 300 10000
    path-retransmit 10
    assoc-retransmit 10
   remote-port 5000
    remote-ip 10.0.0.2
!

• Troubleshooting Tips
  
• What to Do Next
  

Before You Begin

Before performing this task, the Cisco IOS firewall inspect rule must be configured. Also, HSRP and SSO must be configured to enable box-to-box redundancy.

Note	The inspect rules should not have ICMP or protocols for which Cisco IOS firewall supports Layer 7 inspection. >

1.    enable

2.    configure terminal

3.    interface [interface-name]

4.    ip inspect [rule] in| out redundancy stateful [hsrp-group-name]

5.    exit

1.    enable

2.    configure terminal

3.    ip inspect redundancy update seconds [10-60]

4.    exit

1.    enable

2.    debug ip inspect ha [manager | update]

1.    enable

2.    clear ip inspect ha [sessions-all | statistics]

1.    enable

2.    show ip inspect ha {session [detail] | statistics} [vrf vrf-name]}

The following tables provide examples of Stateful Failover error messages and alert message.

The table below contains the stateful failover HA error messages.

*Apr 13 17:09:20.539: %FW_HA-3-SUBSYS_INIT_FAILED: Firewall High availability subsystem initialization failed

*Apr 13 16:50:30.007: %FW_HA-3-TW_INIT_FAILED: Firewall High availability update timer initialization failed

*Apr 13 16:50:30.007: %FW_HA-3-RF_REG_FAILED: Firewall High availability registration to RF failed
*Apr 13 16:50:30.007: %FW_HA-3-CF_REG_FAILED: Firewall High availability registration to CF failed

May 20 21:57:10.475: %FW_HA-6-NO_INSPECT_RULE_ON_STDBY: Firewall High availability - inspect rule is not configured on standby for interface e0/0 dir in/out

*May 20 21:57:10.475: %FW_HA-6-PROT_MISMATCH: Firewall High availability - L4/L7 protocol mismatch between active and standby

May 20 21:57:10.475: %FW_HA-6-NO_HSRP_GNAME_ON_STDBY: Firewall High availability - Inpsect redundancy group is not configured on standby for interface e0/0 dir in/out

*May 20 21:57:10.475: %FW_HA-6-CONFIG_MISMATCH: Firewall High availability - Inspect HA config mismatch between active and standby. act:inspect rule a_test, hsrp_grp a_hsrp_group; stdby:inspect rule s_test hsrp_grp s_hsrp_group 

If audit trail is configured on the standby HA device the standard alerts that are shown when a session is added or deleted will be changed to reflect that the session is a standby session. The table below contains the stateful failover alert messages.

*Apr 14 23:53:44.641: %FW-HA-6-SESS_AUDIT_TRAIL_STDBY_START: Start tcp standby session: initiator (10.0.0.10:22955) -- responder (11.0.0.10:23) 

*Apr 14 23:57:52.891: %FW-HA-6-SESS_AUDIT_TRAIL_STDBY_STOP: Stop tcp standby session: initiator (10.0.0.10:35148) -- responder (11.0.0.10:23) 

*Apr 14 23:57:52.891: %FW-HA-6-SESS_AUDIT_TRAIL_STDBY_TO_ACT: Firewall HA transitioning from Standby to Active HA state