IBM System z mainframes run business-critical applications for many of the world's largest enterprises, and many of these enterprises protect their most critical data using IBM z/OS Global Mirror, still widely known by its former name, IBM Extended Remote Copy, or XRC. This document describes how users of IBM z/OS Global Mirror can reduce costs and improve performance with a tightly integrated Cisco® MDS 9000 Family solution.
IBM XRC Overview
Operationally, IBM z/OS Global Mirror runs on one or more IBM System z logical partitions (LPARs) at the secondary location. This system is called the System Data Mover (SDM). In an initial copy operation, the SDM reads the data from the primary direct-access storage device (DASD) and writes it to the DASD at the secondary site. Thereafter, XRC keeps the DASD synchronized as reader processes in the SDM (or in multiple SDMs) read changed data from the primary DASD across the network and write it to the secondary DASD (Figure 1).
Figure 1. How XRC Works
Increasing XRC Performance
For local and metro distances, up to about 125 miles (200 km), the superior buffer credit capacity of the Cisco MDS 9000 Family FICON director can keep native or Coarse Wavelength-Division Multiplexing (CWDM) or Dense Wavelength-Division Multiplexing (DWDM) FICON links running at capacity, helping ensure the best possible throughput for mission-critical replication workloads. Beyond that distance, Fibre Channel over IP (FCIP) is typically used as the transport for FICON traffic, and the inherent increases in latency can slow the performance of the replication unless the traffic is accelerated. This is the problem that the Cisco MDS 9000 XRC Acceleration feature solves.
When this licensed feature is enabled, the intelligent line card in the Cisco MDS 9000 Family switch recognizes the Define Subsystem Operation (DSO) command that precedes the Read Record Set (RRS) commands that actually read the data from the primary DASD. The Cisco MDS 9000 Family switch reads the primary DASD in anticipation of the actual requests coming from the remote SDM reader process and transmits the data to the remote Cisco MDS 9000 Family switch, where it is buffered in anticipation of the requests from the SDM. When the SDM requests the records, they are already in the Cisco MDS 9000 Family switch in the secondary location and can be presented immediately. This approach allows IBM z/OS Global Mirror to run at the full capacity of its WAN links across almost any distance.
The Cisco MDS 9000 XRC Acceleration feature takes full advantage of the advanced compression and encryption features in the Cisco MDS 9000 18/4-Port Multiservice Module (MSM) line card and Cisco MDS 9222i Multiservice Modular Switch intelligent fabric switch. The compression engines on these platforms typically deliver a data compression ratio of about 4:1, though individual customer results may vary above or below this level. Data traveling over the WAN can also be encrypted using IP Security (IPsec) to help ensure its privacy and integrity. Both of these capabilities are built into the hardware.
The XRC Acceleration feature increases parallelism at every level of operation to deliver the best possible performance for the IBM z/OS Global Mirror (or XRC) solution. Multiple readers are supported within each SDM, and multiple SDMs are supported as well. IBM z/OS Parallel Access Volumes (PAVs) and HyperPAVs are supported. On the storage end, DASD from IBM, EMC and HDS are all supported.
The XRC Acceleration feature builds on the Cisco MDS 9000 Family FICON foundation that has been trusted by mainframe customers since 2002. XRC Acceleration is a separately licensed feature that depends on two other licenses as prerequisites:
• The Mainframe Package enables FICON support for the Cisco MDS 9000 Family chassis.
• The SAN Extension Package enables FCIP support on the Cisco MDS 9000 18/4-Port Multiservice Module line card. Note that when using the Cisco MDS 9222i, SAN extension capabilities are included with the base hardware, and you do not need to purchase and install the SAN Extension license. If an optional Cisco MDS 9000 18/4-Port Multiservice Module line card is installed in the slot of the Cisco MDS 9222i to run FCIP, a SAN Extension license is required for the Cisco MDS 9000 18/4-Port Multiservice Module card.
After the licenses are installed, the Cisco MDS 9000 Family administrator configures the FCIP links as usual and enables the XRC Acceleration feature for the desired FCIP tunnels between the SDM and the primary DASD.
Every Cisco MDS 9500 Series Multilayer Director ever sold can be upgraded to run this new feature. First-generation Cisco MDS 9506 and 9509 Multilayer Directors must have the current Cisco MDS 9500 Series Supervisor-2 Module supervisor cards, which are required to run all Cisco NX-OS Release 4 software. Newer Cisco MDS 9506 and 9509 directors, and all Cisco MDS 9513 Multilayer Directors ship with the Cisco MDS 9500 Series Supervisor-2 Module, so no supervisor upgrade is required.
IBM System z customers who are used to the Brocade USD-X have had to engage professional services to write complex scripts to make any changes. This process meant additional costs and delays in making changes to adapt to new workloads or conditions and was a major obstacle to achieving business agility.
With the Cisco MDS 9000 XRC Acceleration solution, enabling the feature is almost trivial, as described earlier. Even the job of tuning the TCP/IP parameters of an FCIP link to make it storage-friendly can be simplified using the Cisco SAN Extension Tuner tool, which is a standard feature of the Cisco MDS 9000 Family software.
After the solution is running, it can be monitored from a single pane through the Cisco Fabric Manager GUI. Cisco Fabric Manager is included with the Cisco MDS 9000 Family switches at no additional cost, and it allows real-time status and performance to be viewed at any time. If detailed historical performance statistics are desired, the Cisco Fabric Manager Server licensed feature can be used to accumulate, analyze, trend, and report this data for XRC Acceleration sessions, just as it does for other aspects of Cisco MDS 9000 Family FICON and distributed systems SANs.
For those who prefer their single pane to be an IBM z/OS Resource Measurement Facility (RMF) report on their IBM System z, port statistics and performance data can be passed to the IBM System z through the time-tested in-band IBM Control Unit Port (CUP) interface, enabling management using IBM z/OS-based tools.
Security and Resiliency
Mainframes are used for the most sensitive and mission-critical data that organizations have, and any time this data leaves the data center, it must be encrypted. The Cisco MDS 9000 18/4-Port Multiservice Module line card and Cisco MDS 9222i switch both have built-in encryption capabilities for their Gigabit Ethernet ports. This encryption can easily be enabled for XRC Acceleration, as well as for any other traffic running over the WAN.
However, three-site business continuance and disaster recovery (BC/DR) strategies are becoming more popular for delivering aggressive recovery point objectives (RPOs) and recovery time objectives (RTOs) using synchronous replication and failover to a local or metro-distance site, while replicating from that site asynchronously over a WAN for disaster recovery to guard against a regional disaster. The Cisco MDS 9500 Series can secure MAN data as well. All three of the 8-Gbps line cards can deliver Cisco TrustSec Fibre Channel link encryption, which offers protection for data in flight across a native FICON (or open systems Fibre Channel) Inter-Switch Link (ISL). Enabled by the optional Enterprise license, this feature can secure campus or MAN traffic and can even be used by the most security-conscious customer within a data center, as the encryption is performed in the hardware at full line rate (8, 4, or 2 Gbps).
Multiple links may be used between data centers to increase capacity and resiliency. The capability to combine physical ISLs into a logical ISL has been part of SAN architectures from the early days, but Cisco has been a leader in the capabilities it has delivered in this area. Cisco logical ISLs, called PortChannels, can combine up to 16 physical ISLs over native Fibre Channel, including campus or MAN links, or even over FCIP. Cisco's PortChannels have no restrictions limiting the links to only those of approximately the same length, so they can be used to add robustness to MAN configurations by including links in opposite directions on an optical loop. Because all ports in a PortChannel do not need to be located on the same line card application-specific integrated circuit (ASIC) (which would allow an ASIC or line card failure to take down the entire logical link), Cisco PortChannels can be spread over multiple line cards, reducing the impact of an ASIC or line card failure.
The XRC Acceleration feature will perform load balancing across FCIP links and enable traffic to fail over to surviving links should one or more of the links go down.
Reliability and Robustness
Since its introduction in 2002, the Cisco MDS 9000 has been built on the same crossbar architectural principles, providing consistent, predictable performance in the tradition of early FICON director pioneers McData and Inrange. In Cisco directors, unlike alternatives available in the marketplace:
• Slower ports (for example, 4-Gbps ports) with heavy traffic do not slow down faster ports (for example, 8-Gbps ports) to their level.
• Latency remains fair and consistent across all ports as load increases; it does not vary from port to port (competitive solutions may vary by a factor of 8 or more).
• Buffer credits are evenly distributed across all ports (in competitive solutions, nearly half the ports may be constantly running out of buffers).
• Traffic continues flowing across all ports even if a supervisor or switching fabric fails (in competitive solutions, applications such as those from Oracle can experience devastating traffic outages of 30 seconds or more).
The new capabilities of the Cisco MDS 9000 XRC Acceleration feature are clearly a significant advancement for IBM System z installations using IBM z/OS Global Mirror over a WAN. However, the real value derives from the combination of the feature with the advantages of the underlying Cisco MDS 9000 Family architecture and its intrinsic capabilities.