In today's data center, applications are driving increased demand for server processing and I/O. All of this is happening at a time when budgets are shrinking. To grow to meet demand, the data center manager is forced to duplicate, isolate, and waste resources, increasing overall complexity and cost. This is because servers come in fixed packages and prevent resources from being shared. In response, architects have begun making changes to simplify the datacenter as well as consolidate and share resources across multiple customers and applications.
One key area of datacenter complexity that can be simplified is server I/O architecture. Typically, a conventional server today is deployed with multiple network adapters that server three basic I/O requirements: LAN/WAN, SAN, and inter-process communications. Servers can be deployed with multiple Ethernet network interface cards (NICs), Fibre Channel host bus adapters (HBAs), and sometime dedicated clustering interconnects. For mission-critical applications such as databases, servers can require many expansion slots. In a large data center, managing these servers as well as the morass of cables can be difficult and costly, preventing the ability to change quickly to meet business demands. There has to be a better way.
Introducing the Cisco Server Switch, a new class of data center infrastructure that provides a platform to interconnect discreet server resources together into a high performance fabric, to connect that server fabric with shared pools of I/O and storage resources, and to map the resources together to deliver virtual "compute services" based on application or business policy and priority. The Cisco Server Switch enables the delivery of utility computing by consolidating server I/O into a shared pool, significantly simplifying the server I/O architecture and enabling the virtualization of compute resources.
The Cisco SFS 3000 Multifabric Server Switch family creates a unified fabric that dramatically simplifies the datacenter architecture by running multiple types of traffic over a single physical connection. With only one interface card in each server, all resources can be managed on one fabric, eliminating the need to install and manage multiple Ethernet, Fibre Channel, and IPC cards. All types of I/O can be aggregated and load-balanced on a single 10 Gbps InfiniBand cable, reducing the number of managed host ports and increasing availability. The Server Switch then connects servers to a pool of shared Fibre Channel and Ethernet ports via line-rate gateways, and creates virtual I/O subsystems on each host, including virtual HBAs and virtual IP interfaces. When used in conjunction with VFrame server virtualization software, the Server Switch can dynamically commission or decommission this virtual I/O pool to any physical server on-demand.
The SFS 3000 Server Switch Family includes the 3012, an enterprise-class modular chassis with twelve expansion modules, and the 3001, a fixed chassis with one expansion slot. The SFS 3012 Multifabric Server Switch includes up to 24 10 Gbps InfiniBand ports on two switch modules and up to 12 expansion modules, enabling up to 96 ports in a single enclosure. The SFS 3001 Multifabric Server Switch is a fixed switch with 12 ports of 10 Gbps InfiniBand and one expansion slot. The InfiniBand-to-Ethernet Gateway and InfiniBand-to-Fibre Channel Gateway are hot-pluggable expansion modules that allow hosts attached to a unified InfiniBand fabric to share a pool of aggregated Ethernet and Fibre Channel ports.
Figure 1. Cisco SFS 3001 and 3012 Multifabric Server Switches
Figure 2. Cisco SFS Fibre Channel and Ethernet Gateways
Expressly designed to apply networking principles to computing problems, the Cisco SFS 3012 uses a switched, 10-Gbps InfiniBand fabric to interconnect server resources at high performance levels. Servers can communicate with each other across the fabric using Remote Direct Memory Access (RDMA), seamlessly increasing bandwidth and decreasing latency, allowing applications to scale higher and faster by simply replacing the fabric.
Servers can also leverage the higher levels of bandwidth to communicate to existing SANs and LANs. By sharing aggregated gateway ports in conjunction with the 10Gbps InfiniBand connection, individual servers gain access to more bandwidth than an individual port would normally offer. The Server Switch also dynamically load balances across multiple gateways and multiple chassis.
With the ability to add additional hot-plug expansion modules, IT managers can manage and scale bandwidth centrally without bringing down servers. And with the ability to easily cascade multiple switches, IT managers can scale to meet datacenter requirements and protect their initial investments with the flexibility to upgrade over time.
TRANSPARENT TOPOLOGY ARCHITECTURE
The Cisco Transparent Topology Architecture allows existing IP and storage area network (SAN) tools to work out-of-the-box. Using Fibre Channel and Ethernet gateways, administrators can add virtual storage and IP interfaces to every host attached to the unified fabric. Critical storage management and network security models work out-of-the-box, such as multipathing, fabric zoning and storage-based access controls. This allows multiple hosts to share Fibre Channel connections, and still enable storage management tools to "see" each unique host. Also, by adding Ethernet gateways, IT managers can seamlessly bridge to an existing IP subnet, including VLANs, link aggregation, multicast, and other IP-based technologies.
Designed for enterprise-class redundancy, the Cisco SFS 3012 is architected for enterprise-class high availability and redundancy. By combining dynamic load balancing and port aggregation, The Cisco SFS 3012 minimizes failure points in switch blades, controllers, gateways, and ports. All removable components are also hot-swappable, including controllers, power, and cooling.
The Cisco SFS 3001 also meets high-availability requirements with redundant, hot-pluggable power and cooling. The optional expansion module is also hot-pluggable. Multiple Cisco SFS 3001s can also be deployed in redundant pairs to prevent single points of failure at the system level.
Figure 3. Cisco SFS 3001 and 3012 Rear Views
COMPLETE END-TO-END SOLUTION
The Cisco SFS 3000 Series of switches joins the Cisco SFS 7000 Series InfiniBand server switches, Cisco Catalyst
® 6000 Series switches, and Cisco MDS 9000 Series Multilayer SAN Switches to create the most complete datacenter switching solution in the industry. The Cisco SFS 3001 shares common software and expansion modules with the Cisco SFS 3012 Multifabric Server Switch, offering a clear growth path while protecting existing investments.
Configuration and maintenance are simplified with an element manager graphical user interface (GUI) and full command-line interface (CLI) using serial console, Telnet, or Secure Shell (SSH) Protocol, enabling remote monitoring, upgrades, and troubleshooting. With topology transparency, existing SAN and IP management tools work upon deployment. The Cisco SFS 3012 is also fully compatible with applications that support Simple Network Management Protocol (SNMP), allowing IT managers to reduce management time and total cost of ownership.
KEY FEATURES AND BENEFITS
Table 1. Fibre Channel Gateway Features
Virtual I/O for Fibre Channel
Allows a group of servers to share a pool of centralized Fibre Channel I/O resources. Translates between SCSI over InfiniBand (SRP) and FCP at the gateway, and allows an SRP initiator to concurrently talk through multiple shared connections.
Creates unique World-Wide-Names for every virtual HBA, enabling InfiniBand-attached hosts to seamless connect with existing Fibre Channel storage and management tools.
Enables sessions to failover and failback.
Seamless support for existing multipathing tools, including EMC Powerpath, etc.
Centralized connection manager dynamically distributes sessions across multiple gateways.
Storage Access Controls
Compatible with existing switch-based zoning and LUN-based access controls. Multifabric Server Switch also includes support for port and LUN access controls via storage management GUI.
Storage Traffic Monitoring
Creates graphs and reports on storage performance statistics on individual or aggregated ports.
Boot over SAN/LUN Remapping
Enables InfiniBand-attached servers to boot remotely over the SAN via LUN remapping.
Table 2. Ethernet Gateway Features
Virtual I/O for Ethernet
Allows a group of servers to share a pool of centralized Ethernet I/O resources. Translates between IP over InfiniBand and IP over Ethernet at the gateway, and allows an InfiniBand-attached host to seamlessly join an existing IP subnet.
Full IPv4 Multicast Support
Enables multicast-enabled applications across the InfiniBand network.
Choose from a variety of flexible options to prevent broadcast loops.
Jumbo Frame Support
Support for up to 9k Ethernet frames and wire-speed IP fragmentation.
VLAN and Partition Support
Transparent support for VLANs on the InfiniBand network while maintaining existing business and security rules.
Combines multiple ports to optimize use of aggregate bandwidth, as well as high availability. Supports a variety of metrics, including source/ destination IP, source/destination MAC, and round robin.
Supports redundancy groups across multiple gateways and multiple chassis.
Flexible deployment in active-active or active-passive modes to eliminate single points of failure.
DHCP Relay Support
Allows DHCP to work across Ethernet and InfiniBand fabrics.
Table 3. Product Specifications
• 12 ports 4X InfiniBand
• One modular expansion slot (2 Gbps Fibre Channel or 1 Gbps Ethernet)
• Redundant (active/standby) control processor modules with:
· One RS-232 serial port
· One Ethernet management port
Reliability and Availability
Redundant, hot-swappable AC power supply/cooling module; dual AC inputs. Hot-plug expansion module; Deployable in redundant pairs.
Redundant (active/standby) control processor modules; Redundant, hot-swappable AC power supply modules and cooling; Dual AC inputs; Hot-plug expansion modules; Fully passive backplane; Deployable in redundant pairs.
• Standard 19-inch rack-mountable
• 1 rack unit (RU) height (1.75 inch)
• 22 inch depth
• <30 lb
• Rack-mountable in a standard 19 inch EIA rack
• 7 inch (4RU) height
• 24 inch depth
• 30-95 lb
Approvals and Compliance
• FCC: CFR 47 Part 15, Subpart B Class A, UL60950, 3rd ed.
• ICES-003 Issue 2, CSA 22.2 No. 60950:2000
• EN 61000-3-2 (Harmonics), EN 61000-3-3 (Flicker), EN 55022:1998, EN 55024:1998; EN61000-4-1,2,3,4,5,6,8,11, IEC60950, EN60950, EN60825-1 and EN60825-2
• VCCI-V3/02.04, IEC60950
• Sound Pressure: 30dB at 25C ambient ISO 7779 and section 8.5 of ISO 3744:1994(E)
• Sound Power: 40dB at 25C ambient ISO 7779, section 8.6 of ISO 3744:1994(E)
• Sound Pressure: 38dB at 25C ambient ISO 7779 and section 8.5 of ISO 3744:1994(E)
• Sound Power: 50dB at 25C ambient ISO 7779, section 8.6 of ISO 3744:1994(E)
Table 4. SFS 3012 InfiniBand Switch Card
12 4X IB InfiniBand Ports, 10 Gbps each
12 CX4 4X InfiniBand Connectors
10 Gbps line speed, full duplex
IPoIB, SDP, SRP, uDAPL, MPI
Table 5. Ethernet Gateway Module
Six Gigabit Ethernet ports per gateway, Two internal 10 Gbps InfiniBand ports
Six external copper RJ-45
• 1 Gbps line speed per port, full duplex
• 12 Gbps aggregate throughput per module (6 Gbps full duplex); 144 Gbps aggregate per SFS 3012
Link Speed Negotiation
Automatic, 100/1000 Mbps
Transparent Topology emulation; IP over InfiniBand (IPoIB)
Table 6. Fibre Channel Gateway Module
Two 2-Gbps Fibre Channel ports per gateway; Two internal 10 Gbps InfiniBand Ports
Two external Small Form-Factor Pluggable LC
Up to 400 MB/s per channel full duplex, Unique memory-optimized architecture
• Translates between SRP FCP concurrently;
• Fibre Channel tape support
Fibre Channel Compatibility
Transparent Topology Emulation; NL_Port
Link Speed Negotiation
1 or 2-Gbps Autosensing
Class of Service
Class-3 Fibre Channel service
Table 7. Network Management
SFS 3001 and SFS3012
Embedded for out-of-box deployments
Easy configuration, monitoring, and maintenance in band and out of band
• Java-based element manager GUI, installs on a variety of platforms
• Web-based Chassis Manager GUI, point web browser to the switch to manage
• CLI via Telnet, SSH, or RS-232
• Store multiple system images and configuration files for validation and rollback.
• Includes recovery image for failsafe upgrades.
SNMP, NTP, DNS, FTP
MIB-II, Bridge MIB, Interface MIB, IP Forwarding MIB, Ethernet-like MIB, IB SM InfiniBand Subnet Manager MIB, IB SM InfiniBand Subnet Manager Agent, and private enterprise switch MIBs
Easy to use, Java-based storage management utility
• Secure Management: SSH v2, SSL, SNMP v3
• Local and RADIUS authentication and authorization
• Role-based access controls for Ethernet, Fibre Channel, and InfiniBand
• Local and streaming logging
• Configurable verbosity with log rotation and aging
• Ability to upload, aggregate, and filter log files via GUI
Monitoring and Troubleshooting
• Performance monitoring with graphical statistic graphing on a per port or aggregate port basis.
• Monitor switches via SNMP traps and MIBs using integrated or remote trap receiver.
Table 8. Cisco SFS Multifabric Server Switch Environmentals
• 90 to 132 VAC, 180 to 264 VAC autoranging, 47 to 63 Hz
• Power dissipation <200W
• Autoranging 90 to 264 VAC, 47 to 63 Hz
• Power dissipation <650W depending on number of gateway modules
Cisco offers a wide range of services programs to accelerate customer success. These innovative services programs are delivered through a unique combination of people, processes, tools, and partners, resulting in high levels of customer satisfaction. Cisco services help you to protect your network investment, optimize network operations, and prepare the network for new applications to extend network intelligence and the power of your business. For more information about Cisco Services, see
Cisco Technical Support Services or
Cisco Advanced Services.