Cisco SFS InfiniBand Software Configuration Guide, Release 2.10
Bridging InfiniBand and Ethernet Networks
Downloads: This chapterpdf (PDF - 226.0KB) The complete bookPDF (PDF - 5.21MB) | Feedback

Bridging InfiniBand and Ethernet Networks

Table Of Contents

Bridging InfiniBand and Ethernet Networks

About Bridge-Groups

Bridge-Group Options

Bridge-Group Configuration Overview

Configuring Bridge-Groups with Default P_Key and No VLAN

Creating a Single Bridge-Group

Assigning Ethernet Ports to the Bridge-Group

Assigning Ethernet Ports Without Link Aggregation

Assigning a Bridge-Group to a Link Aggregation Group

Assigning the Gateway Slot for the Bridge-Group

Configuring Broadcast Forwarding

Configuring Bridge-Groups with a Non-Default P_Key and a VLAN

Prerequisites

Creating a Single Bridge-Group

Assigning Ethernet Ports and VLAN ID to the Bridge-Group

Assigning Ethernet Ports and VLAN ID Without Link Aggregation

Assigning a Bridge-Group and VLAN ID to a Link Aggregation Group

Assigning the Gateway Slot for the Bridge-Group

Associating the Bridge-Group to a Non-Default Partition

Configuring Broadcast Forwarding

Adding Bridge-Groups to the Larger Network Configuration


Bridging InfiniBand and Ethernet Networks


This chapter describes how to configure bridging between IB and Ethernet networks and includes the following sections:

About Bridge-Groups, page 14-1

Bridge-Group Options

Bridge-Group Configuration Overview, page 14-3

Configuring Bridge-Groups with Default P_Key and No VLAN, page 14-4

Configuring Bridge-Groups with a Non-Default P_Key and a VLAN, page 14-6

Adding Bridge-Groups to the Larger Network Configuration, page 14-8

About Bridge-Groups

A bridge-group is an entity that operates on the Ethernet gateway and enables the bridging of one IPoIB partition to one VLAN (see Figure 14-1).

The Ethernet gateway acts like a Layer 2 bridge between IB and Ethernet. Each Ethernet gateway must be configured for Layer 2 bridging (with or without link aggregation and redundancy groups).

Configuring Layer 2 bridging enables the system to learn everything it needs to know about the location of nodes on the network with very little input from the administrator. The bridge-group bridges only IP protocol and drops all others.

Figure 14-1 Bridge-Groups Connect InfiniBand Networks to Ethernet Networks

Bridge-Group Options

The parameters that you define for the bridge-group determine how the traffic is handled between the IB network and the Ethernet network.

The bridge port has certain attributes that are always attached to the bridge port entity, and certain attributes that are optional to the bridge group. (See Figure 14-2.)

Figure 14-2 Required and Optional Bridge-Group Attributes

Bridge-Group Configuration Overview

For information about configuring bridging between the Ethernet gateway and an Ethernet switch, see the following sections in the sequence listed:

"Creating Partitions" section on page 12-3

"Configuring Link Aggregation" section on page 13-1

"Configuring Bridge-Groups with Default P_Key and No VLAN" section

"Creating a Single Bridge-Group" section

"Associating the Bridge-Group to a Non-Default Partition" section

"Assigning a Bridge-Group to a Link Aggregation Group" section

"Assigning Ethernet Ports and VLAN ID to the Bridge-Group" section

Figure 14-3 shows an example of two bridge-groups created on two separate gateways in a single chassis.

Figure 14-3 Create a Bridge-Group to Bridge a Single Broadcast Domain

Configuring Bridge-Groups with Default P_Key and No VLAN

This section describes how to configure bridge-groups with default P_Key and no VLAN and includes the following topics:

Creating a Single Bridge-Group

Assigning Ethernet Ports to the Bridge-Group

Assigning the Gateway Slot for the Bridge-Group

Configuring Broadcast Forwarding

Creating a Single Bridge-Group

To create a single bridge-group, perform the following steps:


Step 1 Create a partition only if you want non-default partitions.


Note The default partition is created automatically and all bridge-groups are members.


Step 2 (Optional) Create a link aggregation interface.

For more details, see the "Configuring Link Aggregation" section on page 13-1.

The following example shows how to create a link aggregation interface:

SFS-3504 > enable
SFS-3504 # configure terminal
SFS-3504 (config)# interface trunk 1
SFS-3504 (config-if-trunk)# enable
SFS-3504 (config-if-trunk)# interface ethernet 1/1-1/6
SFS-3504 (config-if-ether-1/1-1/6)# trunk-group 1
SFS-3504 (config-if-ether-1/1-1/6)# exit

Step 3 Create the bridge-group by entering the bridge-group command, assigning a number, and assigning a subnet.

Step 4 Enter bridge-group bridge-group# subnet-prefix address of subnet-prefix to be bridged and length.

SFS-3504 (config)# bridge-group 1 subnet-prefix 10.0.0.0 8

Step 5 (Optional) Assign a name to the bridge-group:

SFS-3504 (config)# bridge-group 1 name PrimaryBG
SFS-3504 (config)#


Assigning Ethernet Ports to the Bridge-Group

You must assign Ethernet ports 1-6 to a bridge-group. This can be done either by assigning individual Ethernet ports on the gateway, or by assigning a link aggregation group (with multiple ports) to the bridge-group. This section describes how Ethernet ports are assigned bridge-groups and includes the following methods:

Assigning Ethernet Ports Without Link Aggregation, page 14-5

Assigning a Bridge-Group to a Link Aggregation Group, page 14-5

Assigning Ethernet Ports Without Link Aggregation

(Optional) If you are not using link aggregation, assign individual Ethernet ports to the bridge-group.


Step 1 Specify the type of interface that is assigned to the bridge-group, which is the Ethernet interface:

SFS-3504 (config)# interface ethernet 1/1

Step 2 Specify the bridge-group:

SFS-3504 (config-if-ether-1/1)# bridge-group 1


Assigning a Bridge-Group to a Link Aggregation Group

If you are using link aggregation, you must associate a bridge-group to a trunk interface. To assign a bridge-group, perform the following steps:


Note If you are not using trunking (link aggregation), you must assign the interface ethernet port to the bridge-group as described in the preceding section.



Step 1 Specify the type of interface that is to be assigned to the bridge-group.

The following example shows how to specify the trunk interface:

SFS-3504 (config)# interface trunk 1
SFS-3504 (config-if-trunk)#

Step 2 Specify the bridge-group:

SFS-3504 (config-if-trunk)# bridge-group 1


Assigning the Gateway Slot for the Bridge-Group

Associate the bridge-group to the slot number in the chassis. By assigning the gateway slot, the internal gateway port is associated by default. See the "Internal Gateway Ports" section on page 10-2.

SFS-3504 (config)# interface gateway 1
SFS-3504 (config-if-gw-1/2)# bridge-group 1
SFS-3504 (config)#

Configuring Broadcast Forwarding

If you are using redundancy groups, broadcast forwarding should be configured on the redundancy group and not on the individual bridge-groups. However, if you are not using redundancy groups, broadcast forwarding parameters are configured on the individual bridge-group. Enable broadcast forwarding if you are using a DHCP server.

The following example shows how to enable broadcast forwarding:

SFS-3504 # config
SFS-3504 (config#) bridge-group 1 broadcast-forwarding
SFS-3504 (config#)

Configuring Bridge-Groups with a Non-Default P_Key and a VLAN

This section describes how to configure bridge-groups with a non-default P_Key and a VLAN and includes the following topics:

Prerequisites

Creating a Single Bridge-Group

Assigning Ethernet Ports and VLAN ID to the Bridge-Group

Assigning the Gateway Slot for the Bridge-Group

Associating the Bridge-Group to a Non-Default Partition

Configuring Broadcast Forwarding

Prerequisites

You must create the following tasks before you create a bridge-group:

1. (Optional) Create a non-default partition. See the "Creating Partitions" section on page 12-3. The default partitions are created automatically.

2. Create a VLAN on your Ethernet switch, if you have not already done so.

Creating a Single Bridge-Group

To create one or more bridge-groups, perform the following steps:


Step 1 (Optional) Configure link aggregation.

For more details, see the "Configuring Link Aggregation" section on page 13-3.

Step 2 (Optional) Create IB partition.

For more details, see the "Creating Partitions" section on page 12-3.

Step 3 Create the bridge-group by entering the bridge-group command, assigning a number and assigning a subnet.

Enter bridge-group bridge-group# subnet-prefix address of subnet-prefix to be bridged and length

SFS-3504 (config)# bridge-group 1 subnet-prefix 10.0.0.0 8

Step 4 Assign the Ethernet interface and VLAN ID to the bridge group.

For more details, see the "Assigning Ethernet Ports and VLAN ID to the Bridge-Group" section.

Step 5 Assign an IB interface (gateway port) to the bridge group.

For more details, see the "Assigning the Gateway Slot for the Bridge-Group" section


Assigning Ethernet Ports and VLAN ID to the Bridge-Group

This section describes how to assign Ethernet ports and VLAN ID to a bridge-group and includes the following topics:

Assigning Ethernet Ports Without Link Aggregation, page 14-5

Assigning a Bridge-Group to a Link Aggregation Group, page 14-5

You must assign a minimum of one and a maximum of six Ethernet ports to a bridge-group. This can be done either by assigning individual Ethernet ports on the gateway, or by assigning a link aggregation group (with multiple ports) to the bridge-group.

Assigning Ethernet Ports and VLAN ID Without Link Aggregation

To assign Ethernet ports and VLAN ID without link aggregation, perform the following steps:


Note (Optional) If you are not using link aggregation, assign individual Ethernet ports to the bridge-group.



Step 1 Specify the type of interface that is assigned to the bridge-group, which is the Ethernet interface:

SFS-3504 (config)# interface ethernet 1/1

Step 2 Specify the bridge-group:

SFS-3504 (config-if-ether-1/1)# bridge-group 1 vlan-tag 4


Assigning a Bridge-Group and VLAN ID to a Link Aggregation Group

If you are not using trunking (link aggregation), you must assign the interface Ethernet port to the bridge-group, as described in the "Assigning Ethernet Ports and VLAN ID Without Link Aggregation" section.


Step 1 Specify the type of interface that is to be assigned to the bridge-group, which is the trunk interface in this case:

SFS-3504 (config)# interface trunk 1
SFS-3504 (config-if-trunk)#

Step 2 Specify the bridge-group and VLAN ID of the tagged Ethernet switch port:

SFS-3504 (config-if-trunk)# bridge-group 1 vlan-tag 4


Assigning the Gateway Slot for the Bridge-Group

Associate the bridge-group to the slot number in the chassis. By assigning the gateway slot, the internal gateway port is associated by default. Unless you specify otherwise, internal port 2 is defined. See the "Internal Gateway Ports" section on page 10-2.

The following example shows how to assign the gateway slot:

SFS-3504 (config)# interface gateway 1
SFS-3504 (config-if-gw-1/2)# bridge-group 1
SFS-3504 (config)#

Associating the Bridge-Group to a Non-Default Partition

(Optional) If you have created a partition, as described in "Creating a P_Key Partition ID" section on page 12-3, create the bridge-group and assign the partition key.

The following example shows how to create the bridge-group and assign the partition key:

SFS-3504 (config)# interface gateway 1
SFS-3504 (config-if-gw-1/2)# bridge-group 1 pkey 80:01

Configuring Broadcast Forwarding

If you are using redundancy groups, broadcast-forwarding must be configured on the redundancy group.

However, if you are not using redundancy groups, broadcast-forwarding parameters are configured on the individual bridge-group. Enable broadcast forwarding if you are using a DHCP server.

The following example shows how to enable broadcast forwarding:

SFS-3504 # config
SFS-3504 (config#) bridge-group 1 broadcast-forwarding
SFS-3504 (config#)

Adding Bridge-Groups to the Larger Network Configuration

If you want your IB network to reach a remote subnet (any destination outside your bridged subnet), you must configure the ib-next-hop. See Chapter 15, "Configuring Subnet Forwarding" for more details.