Introduction
RDMA Over Converged Ethernet (RoCEv2)
Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCEv2) allows direct memory access over the network. It does this by encapsulating an Infiniband (IB) transport packet over Ethernet. There are two RoCE versions: RoCEv1 and RoCEv2. RoCEv1 is an Ethernet link layer protocol and hence allows communication between any two hosts in the same Ethernet broadcast domain. RoCEv2 is an internet layer protocol, which means that RoCEv2 packets can be routed.
The RoCEv2 protocol exists on top of either the UDP/IPv4 or the UDP/IPv6 protocol. The UDP destination port number 4791 has been reserved for RoCEv2. Since RoCEv2 packets are routable, the RoCEv2 protocol is sometimes called Routable RoCE.
RoCEv2 is supported on the Windows, Linux and ESXi platforms.
This document provides information to configure RoCEv2 in Mode 1 and Mode 2 using Cisco Integrated Management Controller (Cisco IMC). This document does not provide detailed steps to configure vNIC properties. For detailed steps to configure vNIC properties, refer the configuration guide for your Cisco IMC release.
NVMe over Fibre
NVMe over Fabrics (NVMeoF) is a communication protocol that allows one computer to access NVMe namespaces available on another computer. The commands for discovering, connecting, and disconnecting a NVMeoF storage device are integrated into the nvme utility provided in Linux. The NVMeoF fabric that Cisco supports is RDMA over Converged Ethernet version 2 (RoCEv2). The eNIC RDMA driver works in conjunction with the eNIC driver, which must be loaded first when configuring NVMeoF.