This guide is for sales engineers and storage administrators who want to understand the Cisco® Storage Media Encryption (SME) service. Basic knowledge of Cisco MDS 9000 family Fibre Channel concepts and SANs, including tape backup environments, is expected. Familiarity with the Fibre Channel redirect (FC-redirect) feature of the Cisco MDS 9000 SAN-OS Software is desirable.
This design guide provides details about Cisco SME data flow, supported topologies, and best practices for Cisco SME deployment in a tape backup environment.
Cisco MDS 9000 SAN-OS Software Release 3.3(1) provides scalability enhancements for large-scale Cisco SME deployments.
Cisco Storage Media Encryption Overview
Encryption of storage media in the data center has become a critical issue. Numerous high-profile incidents of lost or stolen tape and disk devices have underscored the risk and exposure that companies face when sensitive information falls into the wrong hands. Regulatory requirements arising from Sarbanes-Oxley, Health Insurance Portability and Accountability Act (HIPAA), Payment Card Industry Data Security Standard (PCI-DSS), and other laws have made encryption a top priority.
To meet these requirements, Cisco has introduced the Cisco SME solution. Cisco SME is a comprehensive network-integrated encryption service with enterprise-class key management that works transparently with existing and new SANs. Figure 1 shows a high-level view of a SAN with Cisco SME service deployed.
Figure 1. Cisco Storage Media Encryption
Cisco SME is a secure, integrated solution that delivers encryption as a SAN service (Figure 2). It provides intuitive provisioning, support for heterogeneous SAN devices, comprehensive key management, and role-based access control (RBAC). Using the clustering infrastructure, Cisco SME provides scalability, high availability, and load balancing.
Figure 2. Cisco SME: Secure, Integrated Solution
The Cisco SME solution offers numerous advantages over competitive solutions:
• Cisco SME installation and provisioning are simple and nondisruptive. Unlike other solutions, Cisco SME does not require rewiring or SAN reconfiguration.
• Encryption engines are integrated on the Cisco MDS 9000 18/4-Port Multiservice Module (MSM) and the Cisco MDS 9222i Multiservice Modular Switch, eliminating the need to purchase and manage additional switch ports, cables, and appliances.
• Traffic from any virtual SAN (VSAN) can be encrypted using Cisco SME, enabling flexible, automated load balancing through network traffic management across multiple SANs.
• No additional software is required for provisioning, key, and user role management; Cisco SME is integrated into Cisco Fabric Manager, therefore reducing operating expenses.
Cisco SME Terminology
The following Cisco SME terms are used in this document:
• Cisco SME interface: The security engine in the Cisco SME line card or fixed slot of a Cisco MDS 9222i fabric switch; each Cisco SME line card and Cisco MDS 9222i switch has one security engine
• Cisco SME cluster: A network of Cisco MDS 9000 family switches that are configured to provide the Cisco SME function; each switch includes one or more Cisco SME line cards, each module includes a security engine, and the switches in the cluster use IP connectivity through the management interface for communication
• Cisco SME cluster node: The Cisco MDS 9000 family switch that is part of a Cisco SME cluster
• Fabric: A physical fabric topology in the SAN as seen by Cisco Fabric Manager; the physical fabric can include multiple VSANs (logical fabrics)
• Tape group: A backup environment in the SAN; it consists of all the tape backup servers and the tape libraries that the servers access
• Tape device: A tape drive that is configured for encryption
• Tape volume: A physical tape cartridge identified by a bar code for a given use
• Tape volume group: A logical grouping of tape volumes that are configured for a specific use: for example, a group of tape volumes used to back up a database
• Cisco SME key hierarchy: The keys included in the Cisco SME key management system: master key, tape volume group keys, and tape volume keys; every backup tape has an associated tape volume key, tape volume group key, and master key (for more information about Cisco SME keys, refer to the Cisco Storage Encryption Media Configuration Guide)
– Master key: This encryption key is generated when a Cisco SME cluster is created. There is a unique master key for each cluster, and it is shared across all members of the cluster. The master key is used to wrap the tape volume group keys.
– Tape volume group key: This encryption key is used to encrypt and authenticate the tape volume keys: the keys that encrypt all tapes belonging to the same tape volume group. A tape volume group can be created on the basis of a bar code range for a set of backup tapes, or it can be associated with a specific backup application.
– Tape volume key: This key is used to encrypt and authenticate the data on the tapes. In unique key mode, the tape volume keys are unique for each physical tape. In shared key mode, one tape volume key is used to encrypt all volumes in a volume group.
• Smart card: A card (approximately the size of a credit card) with a built-in microprocessor and memory used for authentication; it is used to store the master key recovery shares for Cisco SME recovery officers
• Cisco SME administrator: A network administrator who configures Cisco SME
• Cisco SME recovery officer: A data security officer entrusted with smart cards and the associated personal identifier numbers (PINs); each smart card stores a share of the master key of the cluster. Recovery officers must present their cards and PINs to recover the key database of an archived cluster. A quorum of recovery officers is required to execute this operation
• FC-redirect: Capability in Cisco MDS 9000 SAN-OS Software that enables traffic from any switch port to be encrypted without SAN reconfiguration or rewiring
• Cisco SME line card: A module capable of providing Cisco SME services: the Cisco MDS 9000 18/4-Port MSM or the integrated supervisor module on the Cisco MDS 9222i switch; for simplicity, this document uses "Cisco SME line card"
Cisco SME Requirements
Software Requirements
All Cisco MDS 9000 family switches in the Cisco SME cluster must be running the current release of Cisco Fabric Manager and Cisco MDS 9000 SAN-OS Software:
• Cisco FMS must be running Cisco Fabric Manager 3.3(1) or later.
• Cisco MDS 9000 family switches attached to tape devices must be running Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
• All switches that include the Cisco MDS 9000 18/4-Port MSM must be running Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
• All FC-redirect-capable switches must be running Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
Hardware Requirements
Security Engine Capabilities
Cisco SME requires at least one encryption service engine in each cluster. The Cisco SME engines provide the transparent encryption and compression services to the hosts and storage devices. The following hardware supports Cisco SME:
• Cisco MDS 9000 18/4-Port MSM
• Integrated supervisor module on the Cisco MDS 9222i
FC-Redirect-Capable Switches
Cisco SME requires that each target switch be capable of FC-redirect. FC-redirect is supported on the following Cisco switches:
• Cisco MDS 9500 Series
• Cisco MDS 9222i
• Cisco MDS 9216i and 9216A Multilayer Fabric Switches
Cisco FMS and Cisco KMC Workstation Requirements
A separate dedicated workstation with the following configuration should be used for Cisco FMS and Cisco KMC for Cisco SME purposes:
• CPU: 2 GHz or more
• Memory: 2 GB or more
• Disk space: 20 GB or more
Smart Card Readers
To employ standard and advanced security levels, Cisco SME requires the following:
• Smart card reader for Cisco SME (DS-SCR-K9)
• Smart card for Cisco SME (DS-SC-K9)
The smart card reader is a USB device that is connected to a management workstation (Microsoft Windows platforms only). The Cisco SME web client on the management workstation is used to configure the Cisco SME cluster (Figure 3).
The smart card reader requires the smart card drivers that are included on the installation CD. These must be installed on the management workstation where the reader is attached.
The smart card reader is required only for initial configuration (and future recovery scenarios). It is not required for normal Cisco SME operation.
Figure 3. Smart Card Reader
License Requirements
Cisco SME requires a license for each Cisco MDS 9000 18/4 MSM and MDS 9222i switch with a security engine used for Cisco SME. License packages are summarized in Table 1.
Table 1. Cisco SME License Packages
Part Number
Description
Applicable Product
M9500SME1MK9
Cisco SME package for Cisco MDS 9000 18/4-Port MSM
Cisco MDS 9500 Series with 18/4-Port MSM
M9200SME1MK9
Cisco SME package for Cisco MDS 9000 18/4-Port MSM
Cisco MDS 9200 Series with 18/4-Port MSM
M9200SME1FK9
Cisco SME package for fixed slot
Cisco MDS 9222i switch only
Note: A separate license for Cisco FMS is not required if a Cisco SME license is installed.
Topology Requirements
Cisco SME supports a dual-fabric topology. In a dual-fabric SAN, one or more (up to a maximum of four) Cisco SME capable switches form a cluster (Figure 4). Cisco SME supports one cluster for each dual fabric in a data center SAN environment.
Figure 4. Cisco SME Cluster in a Dual-Fabric SAN
Cisco SME is fully supported in a dual-fabric SAN consisting of Cisco switches only. Cisco SME may be supported in some environments consisting of both Cisco and other vendors' switches; such a configuration must be evaluated on a case-by-case basis.
A Cisco SME cluster can span multiple VSANs in a fabric. In Figure 5, traffic in multiple VSANs is encrypted by the same Cisco SME cluster.
Figure 5. Cisco SME Cluster Spanning Multiple VSANs
For more guidelines on the topology design, refer to "Network Topologies" later in this document.
Zoning Requirements
Internal virtual N-ports are created by Cisco SME in the default zone. The default zone must be set to Deny, and these virtual N-ports must not be zoned with any other host or target.
FC-Redirect Requirements
• The Cisco MDS 9000 family switch with a Cisco SME line card must be running Cisco 9000 SAN-OS Software Release 3.3(1) or later.
• The target must be connected to a Cisco MDS 9500 Series, 9222i, 9216i, or 9216A switch running Cisco MDS 9000 SAN-OS Software Release 3.3(1).
• FC-redirect is limited to 32 targets per switch if the targets are connected to Generation 1 switching modules or if there are Inter-Switch Links (ISLs) on the switch from a Generation 1 switching module. To avoid any such limitation, it is recommended that targets for SME and ISLs use Generation 2 (or later) switching modules.
• Each FC-redirect target can be zoned with up to 128 hosts.
• Cisco Fabric Services must not be disabled on all FC-Redirect enabled switches.
• Servers and tape devices using Cisco SME cannot be part of an inter-VSAN routing (IVR) zone set.
• Cisco SME must not be used in conjunction with SAN device virtualization (SDV), Cisco Data Mobility Manager (DMM), or IVR.
• Advanced zoning capabilities such as QOS, LUN-zoning, Read-only LUNs must not be used forhosts and targets involved with FC-Redirect.
Configuration Requirements
• On a Cisco SME line card, either Small Computer System Interface over IP (iSCSI) or Cisco SME can be configured. iSCSI and Cisco SME cannot both be configured on the same Cisco SME line card.
• IVR cannot be enabled on the Cisco SME-enabled switches. Further, hosts and target devices using Cisco SME cannot be part of an IVR zone set.
• Fibre Channel over IP (FCIP) write acceleration and FCIP tape acceleration must not be configured on the Cisco SME data flow (that is, Cisco SME traffic between the host and the target must not pass through FCIP tunnels with FCIP write acceleration and FCIP tape acceleration enabled.
• FCIP and IP Security (IPsec) are not supported on modules running Cisco SME.
Cisco SME Data Flow
Single Cisco SME Switch
Figure 6 shows a single-fabric topology with a Cisco SME line card on one switch. In this case, the data from the server H2 is compressed and encrypted by Cisco SME. Data from the server H1 is not processed by Cisco SME.
Figure 6. Cisco SME Data Flow: Single Cisco SME Switch
Cisco employs an FC-redirect scheme that automatically redirects the traffic flow for the desired initiator-target nexus (I-T nexus) pair to an appropriate Cisco SME line card in the fabric. There is no appliance inline in the data path, and there is no SAN rewiring or reconfiguration. Encryption and compression services are transparent to the hosts and storage devices. These services are available for devices in any VSAN in a physical fabric and can be used without rezoning.
Cisco SME Clustering
Cluster technology provides scalability, reliability, and availability; automated load balancing; failover capabilities; and a single point of management.
A Cisco SME cluster consists of all Cisco SME-enabled switches in a dual-fabric (Figure 7). A Cisco SME cluster can consist of up to four Cisco SME-enabled switches. Scalability can be easily achieved by adding more Cisco SME line cards to Cisco SME-enabled switches. Each Cisco SME-enabled switch can have multiple Cisco SME line cards. Each switch can be part of only one cluster (consequently, each Cisco SME interface can be part of only one cluster).
With multiple Cisco SME line cards in a Cisco SME cluster, the traffic is automatically load balanced across these modules. If a Cisco SME line card or a Cisco MDS 9000 family switch fails, the traffic automatically fails over to another Cisco SME line card in the cluster within the same fabric.
The entire Cisco SME cluster can be managed through a single point using Cisco Fabric Manager.
Figure 7. Cisco SME Clustering
The Cisco SME cluster infrastructure uses the management interface to communicate with other switches in the cluster. A cluster view is defined as the set of switches that are part of the operational cluster. Only switches that are part of a cluster view participate in Cisco SME operations. A cluster requires a quorum of switches to be present. Refer to the Cisco Storage Encryption Media Configuration Guide for details.
Cisco SME Data Flow in a Cluster
After a Cisco SME cluster has been created and provisioned, the data is forwarded from the host to a Cisco SME module in the same fabric using FC-redirect. The data is compressed and encrypted and then sent to the target. When the data is read, it follows the reverse path. Only the traffic from configured I-T nexus pairs is redirected to a Cisco SME module. All other traffic is unaffected.
Each I-T nexus is bound to a specific Cisco SME interface. When multiple Cisco SME modules are present in a Cisco SME cluster, Cisco SME uses target-based load balancing. All I-T nexus pairs for a given target are always bound to the same Cisco SME interface. I-T nexus pairs for different targets are load balanced in the fabric across all available Cisco SME modules in the cluster. These Cisco SME modules can be on any Cisco SME capable switch in the cluster (multiple Cisco SME line cards on one switch are allowed).
In Figure 8, encryption traffic to target T1 (from hosts H1 and H2) flows through the Cisco SME module on switch SW1, and the encryption traffic to target T2 (from host H2) flows through the Cisco SME module on switch S2, all in fabric A. Traffic from host H2 to target T4 flows through switch SW3 in fabric B. Nonencrypted data flow from host H1 to target T3 does not go through the Cisco SME modules.
Figure 8. Cisco SME Data Flow in a Cluster
Failure Conditions
If the Cisco SME interface on switch SW2 fails (or if the entire switch SW2 fails), the traffic flow for I-T nexus pairs bound on the corresponding Cisco SME interfaces will be briefly interrupted (for example, traffic from host H2 to target T2) until the affected I-T nexus pairs are reassigned to other available Cisco SME interfaces in the cluster (Figure 9). The failure can cause some backup applications to stop backup jobs, and these may have to be restarted.
Note: Only Cisco SME interfaces within the same fabric are used for failover. For high availability, you should provision multiple Cisco SME interfaces in each fabric.
Figure 9. Failure Conditions
Cisco SME Clustering: High-Availability Considerations
Cisco SME cluster operations require successful communication among the switches in the Cisco SME cluster using the management interface. Failure of this communication channel can affect the cluster membership of a switch. If a member switch loses communication with other members for more than 20 seconds (and is no longer part of the cluster view), Cisco SME service is stopped on that switch. All the traffic fails over to other switches in the cluster.
For a Cisco SME cluster to be operational, it must include more than half the number of configured switches in the cluster view. Thus, in an N-node cluster, N/2 + 1 nodes form a cluster quorum. If N is even, the cluster quorum requires N/2 nodes and also the presence of the switch with the lowest node ID. The quorum logic helps ensure that in the event of cluster partitions, at most one partition can be operational. All other switches are nonoperational. This behavior helps ensure the consistency of the cluster.
If a Cisco SME cluster is configured with two switches, a quorum requires the presence of the switch with the lowest node ID (usually the master switch). If this switch fails, the entire Cisco SME cluster becomes nonoperational due to lack of a quorum. To avoid such a scenario, you should create a Cisco SME cluster with at least three switches. For more information about Cisco SME clustering and about quorum failures in two-, three-, and four-switch clusters, refer to the cluster quorum overview in the Cisco Storage Encryption Media Configuration Guide.
Network Topologies
Cisco SME is fully supported in fabrics that consist of only Cisco MDS 9000 family products. Cisco SME may be supported in some environments consisting of both Cisco and other vendors' switches; such a configuration must be evaluated on a case-by-case basis.
Cisco SME supports a dual-fabric topology. In a dual-fabric SAN, one or more (up to four) Cisco SME capable switches form a cluster. Dual-fabric topologies support single-path and multipath configurations.
In a single-path configuration, a cluster configuration includes only one path represented as an initiator-target path. In a multipath configuration, a cluster configuration includes all paths, which are represented as multiple initiator-target paths.
Topology Guidelines
When determining the provisioning and configuration requirements for Cisco SME, note the following guidelines related to SAN topology:
• The existing and new tape libraries must be connected to Cisco MDS 9500 and 9200 Series switches.
• Switches connected to tape libraries must be running Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
• Cisco MDS 9000 18/4-Port MSM is supported on Cisco MDS 9500 Series switches and on the Cisco MDS 9222i switch. The switch must be running Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
• For large-scale setups, targets for Cisco SME must be connected to Generation 2 (or later) Fibre Channel modules. Additionally, all ISLs must use Generation 2 (or later) Fibre Channel modules.
Cisco SME Load-Balancing Considerations
Cisco SME employs target-based load balancing, in which all the hosts for a specific target use the same Cisco SME interface for encryption. Additionally, to minimize ISL use, if one or more Cisco SME interfaces are available on the switch to which the target is connected, only these local Cisco SME interfaces are used for encryption for that target. The placement of Cisco SME interfaces in the SAN must take this behavior into consideration.
• For core-edge topologies, in which the targets are in the core, the Cisco SME interfaces should be provisioned in the target switches. For such a scenario, the number of Cisco SME interfaces on each switch should be proportional to the throughput requirement of the tape drives connected to it.
• For edge-core-edge topologies, in which the targets are on the edge, Cisco SME interfaces should be provisioned in the core switches.
• Cisco SME interfaces should not be provisioned so that some interfaces are on switches that have targets connected and some are on switches that do not have targets connected.
This section describes core-edge and edge-core-edge topologies. Additional examples for dedicated tape SANs are presented in the appendix.
Core-Edge Topology
In a core-edge topology, media servers are at the edge of the network, and tape libraries are at the core.
If the targets that require Cisco SME services are connected to only one switch in each fabric in the core (Figure 10), use Cisco SME line cards and provision Cisco SME on that switch in each fabric. The number of Cisco SME line cards depends on the throughput requirements (see "Sizing Guidelines" later in this document).
Figure 10. Core-Edge Topology: Targets on a Single Core Switch in Each Fabric
If the targets that require Cisco SME services are connected to multiple core switches (Figure 11), connect Cisco SME line cards and provision Cisco SME on all these switches. On the basis of the throughput requirements, derive the total number of Cisco SME line cards and spread them (in proportion to the expected traffic) across the switches to which the targets are connected. Whenever Cisco SME interfaces are available on the switch connected to a target, those Cisco SME interfaces are used for the encryption service, thus eliminating unnecessary traffic on ISLs. To handle failure of a Cisco SME interface, each Cisco SME switch should have more than one Cisco SME interface configured (see "Sizing Guidelines" later in this document for details).
Figure 11. Core-Edge Topology: Targets on Multiple Core Switches in Each Fabric
Note: If the Cisco SME line card is on a different switch than the tape library, additional traffic may cross the ISL.
Edge-Core-Edge Topology
In an edge-core-edge topology, the hosts and the targets are at the two edges of the network connected through core switches.
If the targets that require Cisco SME services are connected to only one switch on the edge in each fabric (Figure 12), use Cisco SME line cards and provision Cisco SME on that switch in each fabric. The number of Cisco SME line cards depends on the throughput requirements (see "Sizing Guidelines" later in this document).
Figure 12. Edge-Core-Edge Topology: Targets on a Single Edge Switch in Each Fabric
If the targets that require Cisco SME services are connected to multiple edge switches (Figure 13), connect Cisco SME line cards and provision Cisco SME on the core switches. On the basis of the throughput requirements, derive the total number of Cisco SME line cards and spread them evenly across the core switches. Any additional SME interfaces (for example, for additional throughput requirements) must go to the core switches. In this scenario, in which Cisco SME interfaces are provisioned on switches not connected to the targets, do not provision any more SME interfaces on the switches connected to the target.
Figure 13. Edge-Core-Edge Topology: Targets on Multiple Edge Switches in Each Fabric
Inserting Cisco SME in Existing Cisco SANs
The Cisco SME solution can be added to existing SAN fabrics in either of two ways:
• Upgrade switches connected to the target devices: Upgrade the Cisco MDS 9000 family switches connected to the targets to Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later and add Cisco SME line cards to these switches. Additionally, consider the configuration and zoning requirements specified in "Cisco SME Requirements" earlier in this document.
• Add new switches to the fabric and move the target devices: Add new Cisco MDS 9000 family switches with Cisco SME capabilities (using Cisco SME line cards) to the SAN and move the target devices needing Cisco SME to the new switch. This switch must be running Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
In both these solutions, the host-connected switches should be upgraded to Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later.
Sizing Guidelines
• There are two ways to size the Cisco SME environment:
– Provision a sufficient number of Cisco SME line cards to get the maximum line-rate throughput for each tape drive.
– Provision a sufficient number of Cisco SME line cards to get the aggregate throughput to meet the backup window.
• Each Cisco SME interface supports up to 450 MB/s throughput with compression and encryption enabled.
• The peak throughput to each tape drive with Cisco SME is about the same as that without Cisco SME. Addition of Cisco SME has minimal to no negative effect on the throughput to each tape drive.
• The peak throughput to each tape drive depends on the type of the drive, the server performance, the host bus adapter (HBA) speed (2 Gbps or 4 Gbps), and the backup data (compressibility). For Canterbury Corpus data, the observed compression ratio using Cisco SME is 4.7:1.
– For example, for an LTO-3 tape drive, the peak throughput would be in the range of 80 to 120 MB/s, depending on other factors.
• You should use the actual observed per-drive throughput for sizing calculations.
• The sizing calculations must be performed independently for each fabric in a dual-fabric SAN.
Sizing Using Line-Rate Tape Drive Throughput
• Estimate the per-drive peak throughput in the existing SAN environment.
• Use the per Cisco SME interface throughput of 450 MB/s to derive the number of tape drives that can be serviced by each Cisco SME interface.
– For example, if an LTO-3 drive achieves 80 MB/s peak throughput (as observed in the existing SAN), 5 such drives can be serviced by each Cisco SME interface.
• Derive the total number of Cisco SME interfaces required to support all the tape drives in the SAN.
• To account for failures, add 20 percent or one Cisco SME interface per switch (whichever is greater) to the total number calculated above.
• Detailed examples are provided in the appendix.
Sizing Using Backup Windows Calculations
• Estimate the aggregate backup throughput requirement to meet the backup window. Include projected growth in backup data.
– For example, assume that a total of 90 terabytes (TB) needs to be backed up in a time frame of 18 hours; this translates to 5 TB per hour, or about 1456 MB/s.
• Derive the total number of Cisco SME interfaces required to meet this aggregate throughput.
– In the preceding example, a total of 4 Cisco SME interfaces are required (1456/450).
• To account for failures, add 20 percent or one Cisco SME interface per switch (whichever is greater) to the total number calculated above.
– In the preceding above, if only one Cisco SME switch is used, a total of 4 Cisco SME interfaces should be provisioned. If two Cisco SME switches are used, a total 5 Cisco SME interfaces should be provisioned.
• Detailed examples are provided in the appendix.
Cisco SME Capabilities
Table 2 summarizes Cisco SME capabilities.
Table 2. Cisco SME Capabilities
Capability
Cisco MDS 9000 SAN-OS Release 3.2(3)
Cisco MDS 9000 SAN-OS Release 3.2(3)
Number of Clusters per Switch
1
1
Number of Cluster per Physical Fabric
1
1
Switches in a Cluster
4
4
Fabrics in a Cluster
1
2
Modules in a Switch
11
11
Cisco SME Interfaces in a Cluster
32
32
Initiator-Target Logical Unit Numbers (LUNs), or ITLs, per Cluster
128
1024
LUNs Behind a Target
32
32
Host Ports in a Cluster
128
128
Target Ports in a Cluster
128
128
Number of Hosts per Target
16
128
Tape Groups per Cluster
2
2
Tape Volume Groups in a Tape Group
4
4
Cisco KMC Number of Keys
32,000
32,000
Cisco FMS Guidelines
In small, dedicated tape SAN environments that use Cisco FMS for overall SAN management, Cisco FMS can also be used as the key management server and for the Cisco SME configuration.
However, for larger SAN setups, and especially when the Cisco FMS performance manager is being used, a separate server should be used for Cisco SME purposes.
Appendix: Sizing and Placement Deployment Examples
This appendix presents deployment examples and derives the requirements for the number and placement of Cisco SME interfaces. The calculations shown here are per fabric.
Example 1: Single-Switch Single-Fabric Backup Environment
• The backup environment consists of 16 media servers and 30 LTO-3 tape drives, all connected to a single Cisco MDS 9000 family switch.
• The switch has Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later installed.
• The LTO3 drives get about 80 MB/s peak throughput without Cisco SME.
• The backup time frame is 8 hours, in which about 50 TB of data is backed up every day.
Figure 14 shows the environment for Example 1.
Figure 14. Single Switch Backup Environment
Sizing Using Peak Tape Drive Throughput
With each LTO-3 drive getting 80 MB/s, 5 LTO-3 drives can be supported by one Cisco SME interface (450/80). To support 30 drives, a total of 6 Cisco SME interfaces are required. Adding the capacity for handling failures (20 percent or 1 interface per switch, whichever is greater), a total of about 8 Cisco SME interfaces (and hence 8 Cisco MDS 9000 18/4-Port MSMs) are required, all on the single Cisco MDS 9000 switch.
Sizing Using Backup Time Frame Calculations
50 TB in 8 hours is about 1820 MB/s. (50 x 1024 x 1024)/(8 x 60 x 60). This scenario requires 5 Cisco SME interfaces (1820/450). Adding the capacity for failures (20 percent or 1 interface per switch, whichever is greater), a total of 6 Cisco SME interfaces (and hence 6 Cisco MDS 9000 18/4-Port MSMs) are required, all on the single Cisco MDS 9000 switch.
Zoning Check
The total number of host-target pairs is 16 x 30 = 480, which is within the supported limit of 1024 ITL combinations, so, all the16 hosts can be zoned to all the 30 drives.
Example 2: Core-Edge Topology
• The backup environment consists of 16 media servers, 30 LTO-3 tape drives, and Cisco MDS 9000 family switches.
• The hosts are connected to the edge switches.
• The tape drives are connected to the core switches: 15 drives on each core switch.
• All the switches have Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later installed.
• The LTO-3 drives get about 80 MB/s peak throughput without Cisco SME.
• The backup time frame is 8 hours, in which about 50 TB of data is backed up every day.
Figure 15 shows the environment for Example 2.
Figure 15. Core-Edge Topology
In this topology, the Cisco SME interfaces will be provisioned on the core switches to which the tape drives are connected. For such a scenario, the number of Cisco SME interfaces on each switch should be proportional to the throughput requirement of the tape drives connected to it.
Sizing Using Peak Tape Drive Throughput
With each LTO-3 drive getting 80 MB/s, 5 LTO-3 drives can be supported by one Cisco SME interface (450/80). To support 15 drives on each switch, 3 Cisco SME interfaces are required. Similarly, 3 Cisco SME interfaces are required on the other switch. Adding the additional capacity for handling failures (20 percent or 1 interface per switch, whichever is greater), a total of 4 + 4 = 8 Cisco SME interfaces (and hence 8 Cisco MDS 9000 18/4-Port MSMs) are required, 4 on each switch.
Sizing Using Backup Time Frame Calculations
50 TB in 8 hours is about 1820 MB/s. (50 x 1024 x 1024)/(8 x 60 x 60). This scenario requires 4 Cisco SME interfaces (1820/450): 2 on each switch. Adding the capacity for failures (20 percent or 1 interface per switch, whichever is greater), a total of 6 Cisco SME interfaces (and hence 6 Cisco MDS 9000 18/4-Port MSMs) are required: 3 on each switch.
Zoning Check
The total number of host-target pairs is 16 x 30 = 480, which is within the supported limit of 1024 ITL combinations, so all the16 hosts can be zoned to all the 30 drives.
ISL Considerations
All the ISLs between the core and the edge switches should be on Generation 2 Fibre Channel modules.
Target Connectivity
All the tape drives should be connected to Generation 2 Fibre Channel modules.
Example 3: Edge-Core-Edge Topology with Restricted Zoning
• The backup environment consists of 80 media servers, 72 LTO-3 tape drives, and Cisco MDS 9000 family switches.
• The hosts are connected to the server-edge switches.
• The tape drives are connected to the target-edge switches.
• The core has 2 switches.
• All the switches have Cisco MDS 9000 SAN-OS Software Release 3.3(1) or later installed.
• The LTO-3 drives get about 80 MB/s peak throughput without Cisco SME.
• The backup timeframe is 16 hours, in which about 250 TB of data is backed up every day.
Figure 16 shows the environment for Example 3.
Figure 16. Edge-Core-Edge Topology with Restricted Zoning
In this topology, the Cisco SME interfaces are provisioned on the core switches, equally distributed.
Sizing Using Peak Tape Drive Throughput
With each LTO-3 drive getting 80 MB/s, 5 LTO-3 drives can be supported by one Cisco SME interface (450/80). To support 72 drives, 15 Cisco SME interfaces are required. Adding the capacity for handling failures (20 percent or 1 interface per switch, whichever is greater), a total of 18 Cisco SME interfaces (and hence 18 Cisco MDS 9000 18/4-Port MSMs) are required: 9 on each switch.
Sizing Using Backup Time Frame Calculations
250 TB in 16 hours is about 4551 MB/s (250 x 1024 x 1024)/(16 x 60 x 60). This scenario requires 11 Cisco SME interfaces (4551/450). Adding the capacity for failures (20 percent or 1 interface per switch, whichever is greater), a total of 14 Cisco SME interfaces (and hence 14 Cisco MDS 9000 18/4-Port MSMs) are required: 7 on each switch.
Zoning Check
The total number of host-target pairs is 80 x 72 = 5760, which is beyond the supported limit of 1024 ITL combinations. In this case, zoning restrictions need to be placed so that the total number of ITLs is below 1024. For example, the 80 hosts can be divided into 8 groups of 10. Similarly, the 72 tape drives can be divided into 8 groups of 9. Each of the groups of 10 hosts and 9 targets can be zoned with each other, resulting in 90 ITLs for each group, or a total of 8 x 90 = 720 ITLs, which is within the limit of 1024.
ISL Considerations
All the ISLs between the core and the edge switches should be on Generation 2 Fibre Channel modules.
Target Connectivity
All the tape drives should be connected to Generation 2 Fibre Channel modules.
