Application Networking Solutions

Replication Acceleration Deployment Guide

  • Viewing Options

  • PDF (305.6 KB)
  • Feedback


Data replication applications such as NetApp's SnapMirror ® are dependant on the IP network infrastructure for their performance. Wide Area Networks (WAN) introduce latency, packet loss, congestion and bandwidth limitations that impact data replication performance. Additionally, WAN links are inefficiently utilized, having to transmit repeated data patterns due to the similarity of data in an organization. With limited replication/backup time windows, IT organization struggle to process the ever increasing amounts of data that need to be replicated across their WAN links
Data replication traffic has unique characteristics that require dedicated and specialized resources to expedite processing and overcome the challenges and limitations imposed by the WAN. Data replication traffic is by nature high in volume with a low number of TCP connections. These connections are long lived per session and they are persistent over a long time. This is unlike branch office to data center traffic which is low volume with a high number of TCP connections that are short lived per session. A solution to overcome these limitations and allow efficient backup within the allocated time is needed.

Solution to Data Replication

Cisco Wide Area Application Services (WAAS) version 4.0.19 introduces a new optimization mode called `Replication Accelerator' mode which is designed to accelerate the storage systems' protocol like NetApp's SnapMirror replication activities between Data Centers (also called as DC-to-DC). The optimization between branch offices and data centers (BO-to-DC) currently supported with WAAS is also known as `Application Accelerator' mode.
When a Cisco Wide Area Engine (WAE-7371 or WAE-7341) is enabled for `Replication Accelerator' mode the default policy will be automatically changed such that the WAE will accelerate only traffic related to data replication operations. As soon as `replication-accelerator' mode is enabled the WAEs will be able to detect each other without any additional configuration. In addition to auto discovery and appropriate policy changes the TCP buffers will also be tuned automatically to support typical data replication traffic. Thus, the installation and configuration of WAEs for replication acceleration have become extremely easy with Cisco WAAS.
In this mode, the disk I/O performance has been fine tuned to support the need of typical data center applications. Similarly, the TCP and DRE optimization parameters are also adjusted to suit the data center applications.


• `Replication-Accelerator' mode is supported only on WAE-7371 and WAE-7341 running 4.0.19

• Device mode `Replication Accelerator' can only be configured via CLI

• As the auto-discovery mechanism distinguishes between WAE running `Replication Accelerator' mode versus the WAE running `Application Accelerator' mode the connections between two WAEs running incompatible device mode will not be optimized. Instead, these connections will be established as `pass-through' connections.

• Maximum supported DRE fan-out is 9 peers using WAE-7371 and 4 DRE peers using WAE-7341.

• On WAE-7371 maximum supported concurrent TCP connections are 5000. Similarly, on WAE-7341 maximum supported concurrent TCP connections are 2500.

• When a WAE's device mode change from `Application Accelerator' to `Replication Accelerator' the default policy will be changed as well. So, any changes to existing policy while WAE was in Application Accelerator mode will be lost after the new device mode becomes effect. The same is applicable if a WAE's device mode changes from `Replication Accelerator' to `Application Accelerator'.

Deployment Scenarios

As in a Branch Office-to-Data Center (Application Acceleration) scenario, traffic can be intercepted and/or redirected to a WAE that is enabled for Replication Acceleration. This can be done using one of the following methods:

• Inline (Preferred for Replication Acceleration)

• WCCP (N+1 cluster for High Availability and Load Balancing)

• ACE, CSM module, CSS, or similar device

Configuring WAE for Replication Acceleration-Inline mode

In the below scenario WAE was installed in `Inline` and enabled for `Replication-Accelerator' mode. To obtain an optimal performance for data replication, Cisco WAAS recommends using `Inline' mode and installing the WAE that will be performing data replication optimization as close as possible to the storage systems. Thus the WAE that is configured for `Replication Accelerator' will not have to receive non-Data Replication (DR) traffic. The `Inline' mode also supports `Port-Channel', which enables load-balancing and `high-availability' should one of the physical link fails. Refer to the `Related Documents' section for more information about configuring `Port-Channel' on WAE. The following shows the configuration details:

Figure 1. Cisco WAE in Inline Mode optimizing Data Replication over Native IP

The following configuration example shows how to enable `Replication Accelerator' mode on WAE. In the above scenario `Replication Accelerator' mode must be enabled on WAEs at both sides.
device mode replication-accelerator <<<< Enables Replication Acceleration on WAE
interface InlineGroup 1/0
ip address
inline vlan all

Note: After the WAE is configured for replication acceleration it must be reloaded for the changes to take effect. As soon as it is reloaded the default policy will be automatically changed such that the WAE engine will accelerate only traffic related to data replication operations.

Configuring WAE and Router for WCCP Redirection

In the below deployment scenario WAE is receiving packets through WCCP redirection. As mentioned in the `Inline' mode configuration, it is recommended to install the WAE as close as possible to the storage systems. In the network topology depicted below, the WAE that is enabled for `Replication Acceleration' is connected to a Cisco Catalyst 6500 which is close to the storage system. Thus, it improves the performance by not having to compete with non-Data Replication traffic while traversing through the network infrastructure.

Figure 2. Cisco WAE in WCCP Mode Optimizing Data Replication over Native IP

The following configuration example shows how to enable WCCP on a router and WAE. In Cisco Catalyst 6500 switches it is recommended to configure using `L2-redirect' with mask assignment since L2-redirect processed on the hardware.

Router/MSFC Configuration

hostname Router
ip wccp 61<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Enables WCCP 61
ip wccp 62<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Enables WCCP 62
ip cef
ip domain name
interface GigabitEthernet1/0
description `To Local Area Network'
ip address
ip wccp 61 redirect in <<<<<<<<<<<<<<< Source based redirection to WAE for LAN traffic
duplex full
speed 100
interface GigabitEthernet1/1
description `To Cisco WAE Appliance'
ip address
interface Serial0
description `To Wide Area Network'
ip address
ip wccp 62 redirect in<<<<<<<<<<<< Destination based redirection to WAE for WAN traffic

WAE Configuration

The following is a sample configuration on WAE specific to WCCP.
device mode replication-accelerator <<<<<< Enables `Replication Acceleration'
primary-interface GigabitEthernet 2/0
interface GigabitEthernet 2/0<<<<<<<<<<<<< connected to 6500's GigE1/1
ip address
ip default-gateway
wccp version 2
wccp router-list 1 <<<<<<<<<<< Registers with router
wccp tcp-promiscuous router-list-num 1
l2-redirect mask-assign <<<<<<<<<<<< Enables L2 based redirect for better performance on Cat6500

Installing New WAEs to do Replication Acceleration in an Existing WAE Environment

The following deployment scenarios describe additional ways to deploy new WAE's in Replication Accelerator mode in a data center where Cisco WAAS has already been deployed to optimize/accelerate application traffic (`Application Accelerator' mode) between this data center and remote branch offices.
As mentioned above, to achieve optimal performance for data replication traffic, the `Inline mode' is recommended.
In both deployment scenarios, the WAEs deployed to provide Application Acceleration between the data center and remote branch offices are deployed as a cluster using WCCP. This provides N+1 scalability, high-availability, and load-balancing.
In the first scenario the new WAE is installed in `Inline' mode to achieve `Replication Acceleration'. As soon as the `replication-accelerator' mode is enabled on these newly installed WAEs they can detect each other via `auto-discovery' mechanism.
In the below scenario since the RA-WAE is installed in `Inline' mode and closer to storage systems it is recommended to use WCCP redirect-list so that already optimized replication traffic will not enter the AA-WAE (Application Acceleration-WAE). The following shows a sample configuration on the router:
host Router
ip wccp 61 redirect-list 135
ip wccp 62 redirect-list 135
access-list 135 permit ip any
access-list 135 permit ip any
access-list 135 deny ip any any
In the above configuration example the assumption is that non-replication traffic is located in / 16. Please note that if there are permit statements for traffic to a specific destination (host/subnet), there must also be a permit statement for traffic coming from that same destination. The same applies to `deny' statements.
Also, it is recommended to apply the same ACL to both service groups 61 and 62. This way the router will treat the traffic the same way regardless of the direction from which it is coming.
While the `deny' statement at the end of the ACL is not necessary (deny is implicit), it is recommended to add an explicit `deny' statement for clarity.

Figure 3. Newly Installed RA Mode WAE in an Existing Cisco WAAS Environment

In the following scenario, WAEs that are dedicated for `Replication Acceleration' and `Application Acceleration' are installed in off-path using N+1 WCCP cluster which provides N+1 scalability, high-availability, and load-balancing. As shown in the topology, the WAEs that are configured to do `Replication Acceleration' are connected to a switch that is very close to data storage system at both data centers.

Figure 4. Newly Installed RA Mode WAE in an Existing Cisco WAAS (Application Accelerator) Environment -Both are in WCCP Cluster


• In order to achieve optimal performance in `Replication-Accelerator' mode we recommend installing WAEs in `Inline' mode. However, WCCP is also supported.

• Install the WAEs that are configured for `Replication Acceleration' closer to the storage systems on all the Data Centers. This is highly recommended.

• In a scenario where one can not install the WAE that is enabled for RA mode closer to storage systems there are two options available.

a) Use the WCCP redirect-list with ACL such that only replication / backup traffic enters the WAE that is enabled for RA mode.

b) Set the `Other' classifier in default policy to `Pass Through' and create fewer numbers of customized classifiers that classifies appropriate storage protocol / port number being used to do replication.

• `Replication Accelerator' is recommended between Data Centers that have their WAN link being `Low to Medium Latency' in the range of 20-80ms and `Medium to High Bandwidth' in the range of 20Mbps to 622 Mbps (OC-12).

• To get best possible performance we recommend installing the same type of WAEs (WAE-7371 or WAE-7341) on both end of replication. For example, if you have a `Data-Center-1' that is replicating with `Data-Center-2' then install either WAE-7371 or WAE-7341 at both Data Centers.

• For easier management and better reporting in CM it is recommended to configure separate a device group for WAEs that participate in `Replication Acceleration'.

• When deploying WAEs in `inline' mode, for management purposes it is recommended to connect one of the GigE ports to the network (preferably on management VLAN).

• For ease of management and for better reporting it is recommended to group RA-WAEs as a `Device Group'.

Default Policy

The table below shows the default policy details that are used in Replication Accelerator mode. The first column shows the type of application being used to achieve the data replication; the second column shows the name of the classifier being used in the WAE; and the third column shows the destination port numbers for each.

































(ADAPTER-dynamic ports)UUID:e3514235-4b06-11d1-ab04-00c04fc2dcd2



(ADAPTER-dynamic ports)UUID:f5cc59b4-4264-101a-8c59-08002b2f8426

























Note: You can enable `replication-accelerator' mode only using CLI. There will be no support for configuration of the device mode from the Central Manager. However, the WAAS Central Manager running 4.0.19 or above has the ability to manage `Replication-Accelerator' mode if WAE is configured for `Replication-Accelerator'.

A Quick View of Central Manager

The following example shows that a WAE is enabled for Replication Accelerator that can be seen via the Central Manager.
The Device home page will also show this information for each WAE. The `Device Info' panel currently has information at `Type' of the device. Prior to 4.0.19 it can show whether a given WAE is configured to be Central Manager or WAE. This is updated to show the device mode as well.
For example: WAE (Replication Accelerator) or WAE (Application Accelerator)

Note: When this mode is enabled or disabled, DRE cached will be cleared.

Useful Show Commands

WAE-7371-DC1#show device-mode configured
Configured device mode: replication-accelerator
WAE-7371-DC1#show device-mode current
Current device mode: replication-accelerator
The following command will be helpful to find out the status of `inline group'. Many of the counters listed along with this output can be helpful to verify the ip address configured under WAN and LAN ports, associated MAC address for each port, VLANs that are configured to intercept, whether packets are intercepted properly or not, any errors, and so on.
WAE-7371-DC1#sh int inlinegroup 1/0
Interface is in intercept operating mode.
Standard NIC mode is off.
Disable bypass mode is off.
VLAN IDs configured for inline interception: All
Watchdog timer is enabled.
Timer frequency: 1600 ms.
Autoreset frequency 500 ms.
The watchdog timer will expire in 1282 ms.
Inline Port Statistics Of The Group:
WAN Port :
Device name : eth4. Bypass master interface.
Packets Received : 1380689
Packets Intercepted: 1379075
Packets Bridged : 1614
Packets Forwarded : 1604458
Packets Dropped : 0
Packets Received on native : 0
Active flows for this interface : 0
Ethernet Driver Status
Ethernet address:00:E0:ED:08:AB:C6
Internet address:
Broadcast address:
Maximum Transfer Unit Size:1500
Packets Received: 1380689
Input Errors: 0
Input Packets Dropped: 0
Input Packets Overruns: 0
Input Packets Frames: 0
Packet Sent: 1609953
Output Errors: 0
Output Packets Dropped: 0
Output Packets Overruns: 0
Output Packets Carrier: 0
Output Queue Length:1000
Collisions: 0
Base address:0x6080
LAN Port :
Device name : eth5. Bypass slave interface.
Packets Received : 36423391
Packets Intercepted: 36418020
Packets Bridged : 5371
Packets Forwarded : 17781828
Packets Dropped : 0
Packets Received on native : 0
Active flows for this interface : 0
Ethernet Driver Status
Ethernet address:00:E0:ED:08:AB:C7
Maximum Transfer Unit Size:1500
Packets Received: 36423391
Input Errors: 0
Input Packets Dropped: 57
Input Packets Overruns: 0
Input Packets Frames: 0
Packet Sent: 17783442
Output Errors: 0
Output Packets Dropped: 0
Output Packets Overruns: 0
Output Packets Carrier: 0
Output Queue Length:1000
Collisions: 0
Base address:0x60c0
The following output will show DRE fan-out details, the current device mode, compression type, DRE cache size, and so on.
WAE-7371-DC1#sh statistics dre config
DRE configuration:
Mac-id: 00:1a:64:66:ed:30
Replication-accelerator mode: ON
Comp-Type: DRE + LZ
Max concurrent connections: 5000, max fan-out: 9
TFO working thread number: 20
DRE cache data version: 7
Total DRE cache diskspace: 18431 MB
Total DRE cache memory: 2637 MB, hash table memory: 73 MB
PLZ memory: 2465 MB, PLZ buffer queue size: 8195
PLZ high-watermark=80, low-watermark=50
Total TFO buffer size: 632 MB
ACK TFO buffer queue cap: 158 MB, ACK queue diskspace cap: 1204 MB
Wait queue cap: 1024 MB
Max data-unit number: 25644
Sign-block read pool size: 275, data-segment read pool size: 68
Default anchor distance: 8
Default average chunk size: 256, default min chunk size: 32
Aggregation: OFF, Full tree: OFF, Full tree threshold: 90
Write threshold: 10, Cold pass threshold: 90
LZ algorithm: ZLIB, adaptive LZ: on
LZ compression level: 1
Adaptive threshold: 95, adaptive LZ use entropy: on
Big message checksum: off
WAE-7371-DC1#sh statistics dre
Status: Usable, Oldest Data (age): 15h
Total usable disk size: 18431 MB, Used: 39.08%
Hash table RAM size: 73 MB, Used: 95.00%
Connections: Total (cumulative): 4 Active: 4
Overall: msg: 345233, in: 11172 MB, out: 531 MB, ratio: 95.24%
DRE: msg: 345233, in: 11172 MB, out: 969 MB, ratio: 91.32%
DRE Bypass: msg: 0, in: 0 B
LZ: msg: 313205, in: 938 MB, out: 500 MB, ratio: 46.69%
LZ Bypass: msg: 32028, in: 32367 KB
Avg latency: 0.838 ms
Encode th-put: 39550 KB/s
Message size distribution:
0-1K=18% 1K-5K=2% 5K-15K=9% 15K-25K=9% 25K-40K=18% >40K=41%
Overall: msg: 224524, in: 19777 KB, out: 63505 KB, ratio: 68.86%
DRE: msg: 224524, in: 62850 KB, out: 63505 KB, ratio: 1.03%
DRE Bypass: msg: 0, in: 0 B
LZ: msg: 224524, in: 19777 KB, out: 62850 KB, ratio: 68.53%
LZ Bypass: msg: 0, in: 0 B
Avg latency: 0.064 ms
Decode th-put: 4441 KB/s
Message size distribution:
0-1K=99% 1K-5K=0% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0%

Additional Info

When device mode gets changed from `Application Accelerator' to `Replication Accelerator' or vice versa the WAE needs to be reloaded for the new configuration to take effect.
It is possible to have WAE-7371 on one end and WAE-7341 on the other end. E.g: In a scenario where one major DC has replication activity split between two remote DCs. The major DC may have WAE-7371 and the two remote DC may have WAE-7341.
At the same time, it is not recommended to install WAE-7371 on one end and WAE-7341 on the other end in a scenario where one-to-one replication is happening between two DCs.

Related Documents