C1-Tetration Cluster Device Cabling
Pay attention to the following configuration information when you cable the M5 Virtual Interface Card (VIC) on the 39-RU rack:
-
There are two private interfaces for all nodes.
-
The 39-RU rack has one public interface for 20 nodes.
-
The M5 hardware has four ports per VIC.
-
The names for the bare metal interface—the physical servers in the cluster known as the base, compute, and serving nodes—begin with "eno" (Ethernet onboard).
The following diagram shows the device cabling for the public/external configuration for the C1-Tetration rack. For a detailed list of the connections, see the tables that follow the diagrams.
The following diagram shows the device cabling for the internal/management configuration for the C1-Tetration rack. For a detailed list of the connections, see the following tables.
The following figure shows which ports on the M5 server correspond to the "eno" ports in the figures above:
1 |
Leaf 1 private Server interface port = eno6 CIMC designation = adapter 1/physical port 0/vic-1-eth0 |
2 |
Leaf 1 or leaf 2 public depending on the server Server interface port = eno5 CIMC designation = adapter 1/physical port 2/vic-1-eth1 |
3 |
Not used Server interface port = eno7 CIMC designation = adapter 2/physical port 2/vic-2-eth1 |
4 |
Leaf 2 private Server interface port = eno8 CIMC designation = adapter 2/physical port 0/vic-2-eth0 |
5 |
CIMC Server interface port = eno1 CIMC designation = LOM 1 |
6 |
MGMT 2.2.2.2 Server interface port = eno2 CIMC designation = LOM 2 |
Spine Port |
Connection Type |
Connection | |||
---|---|---|---|---|---|
Device |
RU in Single Rack |
RU in Dual Rack |
Port |
||
1/1 |
CIMC VLAN (1 Gigabit) |
UCS server host 1 (compute node) |
RU 36 |
Rack 1 RU 17 |
eno1 |
1/2 |
CIMC VLAN (1 Gigabit) |
UCS server host 2 (compute node) |
RU 35 |
Rack 1 RU 16 |
eno1 |
1/3 |
CIMC VLAN (1 Gigabit) |
UCS server host 3 (compute node) |
RU 34 |
Rack 1 RU 15 |
eno1 |
1/4 |
CIMC VLAN (1 Gigabit) |
UCS server host 4 (compute node) |
RU 33 |
Rack 1 RU 14 |
eno1 |
1/5 |
CIMC VLAN (1 Gigabit) |
UCS server host 5 (compute node) |
RU 32 |
Rack 1 RU 13 |
eno1 |
1/6 |
CIMC VLAN (1 Gigabit) |
UCS server host 6 (compute node) |
RU 31 |
Rack 1 RU 12 |
eno1 |
1/7 |
CIMC VLAN (1 Gigabit) |
UCS server host 7 (compute node) |
RU 30 |
Rack 1 RU 11 |
eno1 |
1/8 |
CIMC VLAN (1 Gigabit) |
UCS server host 8 (compute node) |
RU 29 |
Rack 1 RU 10 |
eno1 |
1/9 |
CIMC VLAN (1 Gigabit) |
UCS server host 9 (compute node) |
RU 28 |
Rack 1 RU 8 |
eno1 |
1/10 |
CIMC VLAN (1 Gigabit) |
UCS server host 10 (compute node) |
RU 27 |
Rack 1 RU 7 |
eno1 |
1/11 |
CIMC VLAN (1 Gigabit) |
UCS server host 11 (compute node) |
RU 26 |
Rack 1 RU 6 |
eno1 |
1/12 |
CIMC VLAN (1 Gigabit) |
UCS server host 12 (compute node) |
RU 25 |
Rack 1 RU 5 |
eno1 |
1/13 |
CIMC VLAN (1 Gigabit) |
UCS server host 13 (compute node) |
RU 24 |
Rack 1 RU 4 |
eno1 |
1/14 |
CIMC VLAN (1 Gigabit) |
UCS server host 14 (compute node) |
RU 23 |
Rack 1 RU 3 |
eno1 |
1/15 |
CIMC VLAN (1 Gigabit) |
UCS server host 15 (compute node) |
RU 22 |
Rack 1 RU 2 |
eno1 |
1/16 |
CIMC VLAN (1 Gigabit) |
UCS server host 16 (compute node) |
RU 21 |
Rack 1 RU 1 |
eno1 |
1/17 |
CIMC VLAN (1 Gigabit) |
UCS server host 17 (serving node) |
RU 20 |
Rack 2 RU 21 |
eno1 |
1/18 |
CIMC VLAN (1 Gigabit) |
UCS server host 18 (serving node) |
RU 19 |
Rack 2 RU 20 |
eno1 |
1/19 |
CIMC VLAN (1 Gigabit) |
UCS server host 19 (serving node) |
RU 18 |
Rack 2 RU 19 |
eno1 |
1/20 |
CIMC VLAN (1 Gigabit) |
UCS server host 20 (serving node) |
RU 17 |
Rack 2 RU 18 |
eno1 |
1/21 |
CIMC VLAN (1 Gigabit) |
UCS server host 21 (serving node) |
RU 16 |
Rack 2 RU 17 |
eno1 |
1/22 |
CIMC VLAN (1 Gigabit) |
UCS server host 22 (serving node) |
RU 15 |
Rack 2 RU 16 |
eno1 |
1/23 |
CIMC VLAN (1 Gigabit) |
UCS server host 23 (serving node) |
RU 14 |
Rack 2 RU 15 |
eno1 |
1/24 |
CIMC VLAN (1 Gigabit) |
UCS server host 24 (serving node) |
RU 13 |
Rack 2 RU 14 |
eno1 |
1/25 |
CIMC VLAN (1 Gigabit) |
UCS server host 25 (base node) |
RU 12 |
Rack 2 RU 12 |
eno1 |
1/26 |
CIMC VLAN (1 Gigabit) |
UCS server host 26 (base node) |
RU 11 |
Rack 2 RU 11 |
eno1 |
1/27 |
CIMC VLAN (1 Gigabit) |
UCS server host 27 (base node) |
RU 10 |
Rack 2 RU 10 |
eno1 |
1/28 |
CIMC VLAN (1 Gigabit) |
UCS server host 28 (base node) |
RU 9 |
Rack 2 RU 9 |
eno1 |
1/29 |
CIMC VLAN (1 Gigabit) |
UCS server host 29 (base node) |
RU 8 |
Rack 2 RU 8 |
eno1 |
1/30 |
CIMC VLAN (1 Gigabit) |
UCS server host 30 (base node) |
RU 7 |
Rack 2 RU 7 |
eno1 |
1/31 |
CIMC VLAN (1 Gigabit) |
UCS server host 31 (base node) |
RU 6 |
Rack 2 RU 6 |
eno1 |
1/32 |
CIMC VLAN (1 Gigabit) |
UCS server host 32 (base node) |
RU 5 |
Rack 2 RU 5 |
eno1 |
1/33 |
CIMC VLAN (1 Gigabit) |
UCS server host 33 (base node) |
RU 4 |
Rack 2 RU 4 |
eno1 |
1/34 |
CIMC VLAN (1 Gigabit) |
UCS server host 34 (base node) |
RU 3 |
Rack 2 RU 3 |
eno1 |
1/35 |
CIMC VLAN (1 Gigabit) |
UCS server host 35 (base node) |
RU 2 |
Rack 2 RU 2 |
eno1 |
1/36 |
CIMC VLAN (1 Gigabit) |
UCS server host 36 (base node) |
RU 1 |
Rack 2 RU 1 |
eno1 |
1/49 |
Internal VLAN (40 Gigabit) |
Leaf switch 1 (RU 41 in single rack or RU 40 in rack 1 of dual rack) |
RU 40 |
Rack 1 RU 40 |
1/49 |
1/50 |
Internal VLAN (40 Gigabit) |
Leaf switch 2 (RU 40 in single rack or RU 40 of rack 2 in dual rack) port 49 |
RU 41 |
Rack 2 RU 40 |
1/50 |
Leaf 1 Port | Connection Type | Connection | |||
---|---|---|---|---|---|
Device |
RU in Single Rack | RU in Dual Rack | Port | ||
1/1 |
Internal VLAN (10 Gigabit) |
UCS server host 1 (compute node) |
RU 36 |
Rack 1 RU 17 |
eno6 |
1/2 |
Internal VLAN (10 Gigabit) |
UCS server host 2 (compute node) |
RU 35 |
Rack 1 RU 16 |
eno6 |
1/3 |
Internal VLAN (10 Gigabit) |
UCS server host 3 (compute node) |
RU 34 |
Rack 1 RU 15 |
eno6 |
1/4 |
Internal VLAN (10 Gigabit) |
UCS server host 4 (compute node) |
RU 33 |
Rack 1 RU 14 |
eno6 |
1/5 |
Internal VLAN (10 Gigabit) |
UCS server host 5 (compute node) |
RU 32 |
Rack 1 RU 13 |
eno6 |
1/6 |
Internal VLAN (10 Gigabit) |
UCS server host 6 (compute node) |
RU 31 |
Rack 1 RU 12 |
eno6 |
1/7 |
Internal VLAN (10 Gigabit) |
UCS server host 7 (compute node) |
RU 30 |
Rack 1 RU 11 |
eno6 |
1/8 |
Internal VLAN (10 Gigabit) |
UCS server host 8 (compute node) |
RU 29 |
Rack 1 RU 10 |
eno6 |
1/9 |
Internal VLAN (10 Gigabit) |
UCS server host 9 (compute node) |
RU 28 |
Rack 1 RU 8 |
eno6 |
1/10 |
Internal VLAN (10 Gigabit) |
UCS server host 10 (compute node) |
RU 27 |
Rack 1 RU 7 |
eno6 |
1/11 |
Internal VLAN (10 Gigabit) |
UCS server host 11 (compute node) |
RU 26 |
Rack 1 RU 6 |
eno6 |
1/12 |
Internal VLAN (10 Gigabit) |
UCS server host 12 (compute node) |
RU 25 |
Rack 1 RU 5 |
eno6 |
1/13 |
Internal VLAN (10 Gigabit) |
UCS server host 13 (compute node) |
RU 24 |
Rack 1 RU 4 |
eno6 |
1/14 |
Internal VLAN (10 Gigabit) |
UCS server host 14 (compute node) |
RU 23 |
Rack 1 RU 3 |
eno6 |
1/15 |
Internal VLAN (10 Gigabit) |
UCS server host 15 (compute node) |
RU 22 |
Rack 1 RU 2 |
eno6 |
1/16 |
Internal VLAN (10 Gigabit) |
UCS server host 16 (compute node) |
RU 21 |
Rack 1 RU 1 |
eno6 |
1/17 |
Internal VLAN (10 Gigabit) |
UCS server host 17 (serving node) |
RU 20 |
Rack 2 RU 21 |
eno6 |
1/18 |
Internal VLAN (10 Gigabit) |
UCS server host 18 (serving node) |
RU 19 |
Rack 2 RU 20 |
eno6 |
1/19 |
Internal VLAN (10 Gigabit) |
UCS server host 19 (serving node) |
RU 18 |
Rack 2 RU 19 |
eno6 |
1/20 |
Internal VLAN (10 Gigabit) |
UCS server host 20 (serving node) |
RU 17 |
Rack 2 RU 18 |
eno6 |
1/21 |
Internal VLAN (10 Gigabit) |
UCS server host 21 (serving node) |
RU 16 |
Rack 2 RU 17 |
eno6 |
1/22 |
Internal VLAN (10 Gigabit) |
UCS server host 22 (serving node) |
RU 15 |
Rack 2 RU 16 |
eno6 |
1/23 |
Internal VLAN (10 Gigabit) |
UCS server host 23 (serving node) |
RU 14 |
Rack 2 RU 15 |
eno6 |
1/24 |
Internal VLAN (10 Gigabit) |
UCS server host 24 (serving node) |
RU 13 |
Rack 2 RU 14 |
eno6 |
1/25 |
Internal VLAN (10 Gigabit) |
UCS server host 25 (base node) |
RU 12 |
Rack 2 RU 12 |
eno6 |
1/26 |
Internal VLAN (10 Gigabit) |
UCS server host 26 (base node) |
RU 11 |
Rack 2 RU 11 |
eno6 |
1/27 |
Internal VLAN (10 Gigabit) |
UCS server host 27 (base node) |
RU 10 |
Rack 2 RU 10 |
eno6 |
1/28 |
Internal VLAN (10 Gigabit) |
UCS server host 28 (base node) |
RU 9 |
Rack 2 RU 9 |
eno6 |
1/29 |
Internal VLAN (10 Gigabit) |
UCS server host 29 (base node) |
RU 8 |
Rack 2 RU 8 |
eno6 |
1/30 |
Internal VLAN (10 Gigabit) |
UCS server host 30 (base node) |
RU 7 |
Rack 2 RU 7 |
eno6 |
1/31 |
Internal VLAN (10 Gigabit) |
UCS server host 31 (base node) |
RU 6 |
Rack 2 RU 6 |
eno6 |
1/32 |
Internal VLAN (10 Gigabit) |
UCS server host 32 (base node) |
RU 5 |
Rack 2 RU 5 |
eno6 |
1/33 |
Internal VLAN (10 Gigabit) |
UCS server host 33 (base node) |
RU 4 |
Rack 2 RU 4 |
eno6 |
1/34 |
Internal VLAN (10 Gigabit) |
UCS server host 34 (base node) |
RU 3 |
Rack 2 RU 3 |
eno6 |
1/35 |
Internal VLAN (10 Gigabit) |
UCS server host 35 (base node) |
RU 2 |
Rack 2 RU 2 |
eno6 |
1/36 |
Internal VLAN (10 Gigabit) |
UCS server host 36 (base node) |
RU 1 |
Rack 2 RU 1 |
eno6 |
1/37 |
Public VLAN (10 Gigabit) |
UCS server host 33 (base node) |
RU 3 |
Rack 2 RU 3 |
eno5 |
1/38 |
Public VLAN (10 Gigabit) |
UCS server host 35 (base node) |
RU 1 |
Rack 2 RU 1 |
eno5 |
1/39 |
Internal VLAN (10 Gigabit) |
Customer router 1 |
— |
— |
— |
1/40 |
Internal VLAN (10 Gigabit) |
Leaf 1 |
RU 40 |
Rack 1 RU 40 |
1/40 |
1/41 |
Public VLAN (10 Gigabit) |
UCS server host 1 (compute node) |
RU 35 |
Rack 1 RU 16 |
eno5 |
1/42 |
Public VLAN (10 Gigabit) |
UCS server host 3 (compute node) |
RU 33 |
Rack 1 RU 14 |
eno5 |
1/43 |
Public VLAN (10 Gigabit) |
UCS server host 5 (compute node) |
RU 31 |
Rack 1 RU 12 |
eno5 |
1/44 |
Public VLAN (10 Gigabit) |
UCS server host 7 (compute node) |
RU 29 |
Rack 1 RU 10 |
eno5 |
1/45 |
Public VLAN (10 Gigabit) |
UCS server host 9 (compute node) |
RU 27 |
Rack 1 RU 8 |
eno5 |
1/46 |
Public VLAN (10 Gigabit) |
UCS server host 11 (compute node) |
RU 25 |
Rack 1 RU 6 |
eno5 |
1/47 |
Public VLAN (10 Gigabit) |
UCS server host 13 (compute node) |
RU 23 |
Rack 1 RU 4 |
eno5 |
1/48 |
Public VLAN (10 Gigabit) |
UCS server host 15 (compute node) |
RU 21 |
Rack 1 RU 2 |
eno5 |
1/49 |
Internal VLAN (40 Gigabit) |
Spine switch |
RU 42 |
Rack 1 RU 42 |
1/49 |
1/50 |
— |
— |
— |
— |
— |
1/51 |
— |
— |
— |
— |
— |
1/52 |
— |
— |
— |
— |
— |
1/53 |
Internal VLAN (40 Gigabit) |
Leaf switch 1 |
RU 40 |
Rack 1 RU 40 |
1/53 |
1/54 |
Internal VLAN (40 Gigabit) |
Leaf switch 1 |
RU 40 |
Rack 1 RU 40 |
1/54 |
Leaf 2 Port | Connection Type | Connection | |||
---|---|---|---|---|---|
Device | RU in Single Rack | RU in Dual Rack | Port | ||
1/1 |
Internal VLAN (10 Gigabit) |
UCS server host 1 (compute node) |
RU 36 |
Rack 1 RU 17 |
eno8 |
1/2 |
Internal VLAN (10 Gigabit) |
UCS server host 2 (compute node) |
RU 35 |
Rack 1 RU 16 |
eno8 |
1/3 |
Internal VLAN (10 Gigabit) |
UCS server host 3 (compute node) |
RU 34 |
Rack 1 RU 15 |
eno8 |
1/4 |
Internal VLAN (10 Gigabit) |
UCS server host 4 (compute node) |
RU 33 |
Rack 1 RU 14 |
eno8 |
1/5 |
Internal VLAN (10 Gigabit) |
UCS server host 5 (compute node) |
RU 32 |
Rack 1 RU 13 |
eno8 |
1/6 |
Internal VLAN (10 Gigabit) |
UCS server host 6 (compute node) |
RU 31 |
Rack 1 RU 12 |
eno8 |
1/7 |
Internal VLAN (10 Gigabit) |
UCS server host 7 (compute node7) |
RU 30 |
Rack 1 RU 11 |
eno8 |
1/8 |
Internal VLAN (10 Gigabit) |
UCS server host 8 (compute node) |
RU 29 |
Rack 1 RU 10 |
eno8 |
1/9 |
Internal VLAN (10 Gigabit) |
UCS server host 9 (compute node) |
RU 28 |
Rack 1 RU 8 |
eno8 |
1/10 |
Internal VLAN (10 Gigabit) |
UCS server host 10 (compute node) |
RU 27 |
Rack 1 RU 7 |
eno8 |
1/11 |
Internal VLAN (10 Gigabit) |
UCS server host 11 (compute node) |
RU 26 |
Rack 1 RU 6 |
eno8 |
1/12 |
Internal VLAN (10 Gigabit) |
UCS server host 12 (compute node) |
RU 25 |
Rack 1 RU 5 |
eno8 |
1/13 |
Internal VLAN (10 Gigabit) |
UCS server host 13 (compute node) |
RU 24 |
Rack 1 RU 4 |
eno8 |
1/14 |
Internal VLAN (10 Gigabit) |
UCS server host 14 (compute node) |
RU 23 |
Rack 1 RU 3 |
eno8 |
1/15 |
Internal VLAN (10 Gigabit) |
UCS server host 15 (compute node) |
RU 22 |
Rack 1 RU 2 |
eno8 |
1/16 |
Internal VLAN (10 Gigabit) |
UCS server host 16 (compute node) |
RU 21 |
Rack 1 RU 1 |
eno8 |
1/17 |
Internal VLAN (10 Gigabit) |
UCS server host 17 (serving node) |
RU 20 |
Rack 2 RU 21 |
eno8 |
1/18 |
Internal VLAN (10 Gigabit) |
UCS server host 18 (serving node) |
RU 19 |
Rack 2 RU 20 |
eno8 |
1/19 |
Internal VLAN (10 Gigabit) |
UCS server host 19 (serving node) |
RU 18 |
Rack 2 RU 19 |
eno8 |
1/20 |
Internal VLAN (10 Gigabit) |
UCS server host 20 (serving node) |
RU 17 |
Rack 2 RU 18 |
eno8 |
1/21 |
Internal VLAN (10 Gigabit) |
UCS server host 21 (serving node) |
RU 16 |
Rack 2 RU 17 |
eno8 |
1/22 |
Internal VLAN (10 Gigabit) |
UCS server host 22 (serving node) |
RU 15 |
Rack 2 RU 16 |
eno8 |
1/23 |
Internal VLAN (10 Gigabit) |
UCS server host 23 (serving node) |
RU 14 |
Rack 2 RU 15 |
eno8 |
1/24 |
Internal VLAN (10 Gigabit) |
UCS server host 24 (serving node) |
RU 13 |
Rack 2 RU 14 |
eno8 |
1/25 |
Internal VLAN (10 Gigabit) |
UCS server host 25 (base node) |
RU 12 |
Rack 2 RU 12 |
eno8 |
1/26 |
Internal VLAN (10 Gigabit) |
UCS server host 26 (base node) |
RU 11 |
Rack 2 RU 11 |
eno8 |
1/27 |
Internal VLAN (10 Gigabit) |
UCS server host 27 (base node) |
RU 10 |
Rack 2 RU 10 |
eno8 |
1/28 |
Internal VLAN (10 Gigabit) |
UCS server host 28 (base node) |
RU 9 |
Rack 2 RU 9 |
eno8 |
1/29 |
Internal VLAN (10 Gigabit) |
UCS server host 29 (base node) |
RU 8 |
Rack 2 RU 8 |
eno8 |
1/30 |
Internal VLAN (10 Gigabit) |
UCS server host 30 (base node) |
RU 7 |
Rack 2 RU 7 |
eno8 |
1/31 |
Internal VLAN (10 Gigabit) |
UCS server host 31 (base node) |
RU 6 |
Rack 2 RU 6 |
eno8 |
1/32 |
Internal VLAN (10 Gigabit) |
UCS server host 32 (base node) |
RU 5 |
Rack 2 RU 5 |
eno8 |
1/33 |
Internal VLAN (10 Gigabit) |
UCS server host 33 (base node) |
RU 4 |
Rack 2 RU 4 |
eno8 |
1/34 |
Internal VLAN (10 Gigabit) |
UCS server host 34 (base node) |
RU 3 |
Rack 2 RU 3 |
eno8 |
1/35 |
Internal VLAN (10 Gigabit) |
UCS server host 35 (base node) |
RU 2 |
Rack 2 RU 2 |
eno8 |
1/36 |
Internal VLAN (10 Gigabit) |
UCS server host 36 (base node) |
RU 1 |
Rack 2 RU 1 |
eno8 |
1/37 |
Public VLAN (10 Gigabit) |
UCS server host 34 (base node) |
RU 4 |
Rack 2 RU 8 |
eno5 |
1/38 |
Public VLAN (10 Gigabit) |
UCS server host 36 (base node) |
RU 2 |
Rack 2 RU 6 |
eno5 |
1/39 |
Internal VLAN (10 Gigabit) |
Customer router 1 |
— |
— |
— |
1/40 |
Internal VLAN (10 Gigabit) |
Leaf switch 2 |
RU 41 |
Rack 2 RU 40 |
1/40 |
1/41 |
Public VLAN (10 Gigabit) |
UCS server host 2 (compute node) |
RU 36 |
Rack 1 RU 17 |
eno5 |
1/42 |
Public VLAN (10 Gigabit) |
UCS server host 4 (compute node) |
RU 34 |
Rack 1 RU 15 |
eno5 |
1/43 |
Public VLAN (10 Gigabit) |
UCS server host 6 (compute node) |
RU 32 |
Rack 1 RU 13 |
eno5 |
1/44 |
Public VLAN (10 Gigabit) |
UCS server host 8 (compute node) |
RU 30 |
Rack 1 RU 11 |
eno5 |
1/45 |
Public VLAN (10 Gigabit) |
UCS server host 10 (compute node) |
RU 28 |
Rack 1 RU 9 |
eno5 |
1/46 |
Public VLAN (10 Gigabit) |
UCS server host 12 (compute node) |
RU 26 |
Rack 1 RU 7 |
eno5 |
1/47 |
Public VLAN (10 Gigabit) |
UCS server host 14 (compute node) |
RU 24 |
Rack 1 RU 5 |
eno5 |
1/48 |
Public VLAN (10 Gigabit) |
UCS server host 16 (compute node) |
RU 22 |
Rack 1 RU 3 |
eno5 |
1/49 |
Internal VLAN (40 Gigabit) |
Spine switch |
RU 42 |
Rack 1 RU 42 |
— |
1/50 |
— |
— |
— |
— |
1/50 |
1/51 |
— |
— |
— |
— |
— |
1/52 |
— |
— |
— |
— |
— |
1/53 |
Internal VLAN (40 Gigabit) |
Leaf 1 switch |
RU 40 |
Rack 1 RU 40 |
1/49 |
1/54 |
Internal VLAN (40 Gigabit) |
Leaf 2 switch |
RU 41 |
Rack 2 RU 40 |
1/50 |