Dynamic Breakout Ports

This chapter contains the following sections:

Configuration of Dynamic Breakout Ports

Breakout cables are suitable for very short links and offer a cost effective way to connect within racks and across adjacent racks.

Breakout enables a 40 Gigabit (Gb) port to be split into four independent and logical 10Gb ports or a 100Gb port to be split into four independent and logical 25Gb ports.

Before you configure breakout ports, connect a 40Gb port to four 10Gb ports or a 100Gb port to four 25Gb ports with one of the following cables:

  • Cisco QSFP-4SFP10G

  • Cisco QSFP-4SFP25G

The 40Gb to 10Gb dynamic breakout feature is supported on the access facing ports of the following switches:

  • N9K-C9332PQ

  • N9K-C93180LC-EX

  • N9K-C9336C-FX

The 100Gb to 25Gb breakout feature is supported on the access facing ports of the following switches:

  • N9K-C93180LC-EX

  • N9K-C9336C-FX2

  • N9K-C93180YC-FX in the 3.2 and later releases

Observe the following guidelines and restrictions:

  • In general, breakout ports and port profiles (ports changed from uplink to downlink) are not supported on the same port.

    However, from Cisco Application Policy Infrastructure Controller (APIC) release 3.2, dynamic breakouts (both 100Gb and 40Gb) are supported on profiled QSFP ports on the N9K-C93180YC-FX switch.

  • Fast Link Failover policies are not supported on the same port with the dynamic breakout feature.

  • Breakout ports cannot be used for Cisco APIC connectivity.

  • Breakout subports can be used in the same way other port types in the policy model are used.

  • When a port is enabled for dynamic breakout, other policies (expect monitoring policies) on the parent port are no longer valid.

  • When a port is enabled for dynamic breakout, other EPG deployments on the parent port are no longer valid.

  • A breakout sub-port can not be further broken out using a breakout policy group.

  • If the LACP transmit rate on port channels that have breakout sub-ports need to be changed, then all the port channels that include breakout sub-ports need to use the same LACP transmit rate configuration. You can configure an override policy to set the transmit rate as follows:

    1. Configure/change the default port channel member policy to include Fast Transmit Rate (Fabric > Access Policies > Policies > Interface > Port Channel Member).

    2. Configure all the PC/vPC interface policy groups to include the above default port channel member policy under the override policy groups (Fabric > Access Policies > Interfaces > Leaf Interfaces > Policy Groups > PC/vPC Interface).

Configuring Dynamic Breakout Ports Using the APIC GUI

Configure a Breakout Leaf Port with an Leaf Interface Profile, associate the profile with a switch, and configure the sub ports with the following steps.


Note


You can also configure ports for breakout in the APIC GUI by navigating to Fabric > Inventory, and clicking Topology or Pod, or expanding Pod and clicking Leaf. Then, enable configuration and click the Interface tab.


Procedure

Before you begin

  • The ACI fabric is installed, APIC controllers are online, and the APIC cluster is formed and healthy.

  • An APIC fabric administrator account is available that will enable creating the necessary fabric infrastructure configurations.

  • The target leaf switches are registered in the ACI fabric and available.

  • The 40GE or 100GE leaf switch ports are connected with Cisco breakout cables to the downlink ports.

Procedure


Step 1

On the menu bar, choose Fabric > Access Policies.

Step 2

In the Navigation pane, expand Interfaces and Leaf Interfaces and Profiles.

Step 3

Right-click Profiles and choose Create Leaf Interface Profile.

Step 4

Type the name and optional description, click the + symbol on Interface Selectors

Step 5

Perform the following:

  1. Type a name (and optional description) for the Access Port Selector.

  2. In the Interface IDs field, type the slot and port for the breakout port.

  3. In the Interface Policy Group field, click the down arrow and choose Create Leaf Breakout Port Group.

  4. Type the name (and optional description) for the Leaf Breakout Port Group.

  5. In the Breakout Map field, choose 10g-4x or 25g-4x.

    For switches supporting breakout, see Configuration of Dynamic Breakout Ports.

  6. Click Submit.

Step 6

To assign a Breakout Port to an EPG, perform the following steps:

On the menu bar, choose Tenant > Application Profiles > Application EPG. Right-click on Application EPG to open Create Application EPGdialog box, and perform the following steps:

  1. Select the Statically Link with Leaves/Paths check box to gain access to the Leaves/Paths tab in the dialog box.

  2. Complete one of the following sets of steps:

Option Description
If you want to deploy the EPG on... Then
A node
  1. Expand the Leaves area.

  2. From the Node drop-down list, choose a node.

  3. In the Encap field, enter the appropriate VLAN.

  4. (Optional) From the Deployment Immediacy drop-down list, accept the default On Demand or choose Immediate.

  5. (Optional) From the Mode drop-down list, accept the default Trunk or choose another mode.

A port on the node
  1. Expand the Paths area.

  2. From the Path drop-down list, choose the appropriate node and port.

  3. (Optional) In the Deployment Immediacy field drop-down list, accept the default On Demand or choose Immediate.

  4. (Optional) From the Mode drop-down list, accept the default Trunk or choose another mode.

  5. In the Port Encap field, enter the secondary VLAN to be deployed.

  6. (Optional) In the Primary Encap field, enter the primary VLAN to be deployed.

Step 7

To associate the Leaf Interface Profile to a the leaf switch, perform the following steps:

  1. Expand Switches and Leaf Switches, and Profiles.

  2. Right-click Profiles and select Create Leaf Profiles.

  3. Type the name and optional description of the Leaf Profile.

  4. Click the + symbol on the Leaf Selectors area.

  5. Type the leaf selector name and an optional description.

  6. Click the down arrow on the Blocks field and choose the switch to be associated with the breakout leaf interface profile.

  7. Click the down arrow on the Policy Group field and choose Create Access Switch Policy Group.

  8. Type a name and optional description for the Access Switch Policy Group.

  9. Optional. Enable other policies.

  10. Click Submit.

  11. Click Update.

  12. Click Next.

  13. In the Associations Interface Selector Profiles area, choose the Interface Selector Profile you previously created for the breakout port.

  14. Click Finish.

Step 8

To verify the breakout port has been split into four sub ports, perform the following steps:

  1. On the Menu bar, click Fabric > Inventory.

  2. On the Navigation bar, click the Pod and Leaf where the breakout port is located.

  3. Expand Interfaces and Physical Interfaces.

    You should see four ports at the position where the breakout port was configured. For example, if you configured 1/10 as a breakout port, you should see the following:
    • eth1/10/1

    • eth1/10/2

    • eth1/10/3

    • eth1/10/4

Step 9

To configure the sub ports, perform the following steps:

  1. On the Menu bar, click Fabric > Access Policies.

  2. On the Navigation bar, expand Interfaces, Leaf Interfaces, Profiles, and the breakout leaf interface profile you previously created.

    You will see a port selector with the breakout cable. Instead of defining a sub port block under already existing port selector, you need to define on a new access port selector.

  3. On the Navigtion bar, right click the higher level interface profile and select Create Access Port Selector.

  4. In the Name field, enter the sub port name.

  5. In the Interface IDs field, enter the IDs for the four sub ports in a format such as 1/10/1-4.

  6. In the Interface Policy Group field, select Create Leaf Access Port Policy Group.

  7. Click Submit.

Step 10

To apply the Policy Group to an individual interface that links the AAEP to the port, perform the following steps:

  1. In the Name field, enter the name for the Leaf Access Port Group Policy.

  2. In the Link Level Policy field, select link-level_auto.

  3. In the CDP Policy field, select cdp_enabled.

  4. In the LLDP Policy field, select default.

  5. In the Attached Entity Profile field, select the AAEP profile to attach to the policy group.

  6. Click Submit.


Configuring Dynamic Breakout Ports Using the NX-OS Style CLI

Use the following steps to configure a breakout port, verify the configuration, and configure an EPG on a sub port, using the NX-OS style CLI.

Before you begin

  • The ACI fabric is installed, APIC controllers are online, and the APIC cluster is formed and healthy.

  • An APIC fabric administrator account is available that will enable creating the necessary fabric infrastructure configurations.

  • The target leaf switches are registered in the ACI fabric and available.

  • The 40GE or 100GE leaf switch ports are connected with Cisco breakout cables to the downlink ports.

Procedure

  Command or Action Purpose

Step 1

configure

Example:

apic1# configure

Enters configuration mode.

Step 2

leaf ID

Example:

apic1(config)# leaf 101

Selects the leaf switch where the breakout port will be located and enters leaf configuration mode.

Step 3

interface ethernet slot/port

Example:

apic1(config-leaf)# interface ethernet 1/16

Identifies the interface to be enabled as a 40 Gigabit Ethernet (GE) breakout port.

Step 4

breakout 10g-4x | 25g-4x

Example:

apic1(config-leaf-if)# breakout 10g-4x

Enables the selected interface for breakout.

Note

 

For switch support for the Dynamic Breakout Port feature, see Configuration of Dynamic Breakout Ports.

Step 5

show run

Example:

apic1(config-leaf-if)# show run
# Command: show running-config leaf 101 interface ethernet 1 / 16
# Time: Fri Dec  2 18:13:39 2016
  leaf 101 
    interface ethernet 1/16
      breakout 10g-4x
apic1(config-leaf-if)# exit
apic1(config-leaf)# exit

Verifies the configuration by showing the running configuration of the interface and returns to global configuration mode.

Step 6

tenant tenant-name

Example:

apic1(config)# tenant tenant64

Selects or creates the tenant that will consume the breakout ports and enters tenant configuration mode.

Step 7

vrf context vrf-name

Example:

apic1(config-tenant)# vrf context vrf64
apic1(config-tenant-vrf)# exit

Creates or identifies the Virtual Routing and Forwarding (VRF) instance associated with the tenant and exits the configuration mode.

Step 8

bridge-domain bridge-domain-name

Example:

apic1(config-tenant)# bridge-domain bd64

Creates or identifies the bridge-domain associated with the tenant and enters BD configuration mode.

Step 9

vrf member vrf-name

Example:

apic1(config-tenant-bd)# vrf member vrf64
apic1(config-tenant-bd)# exit

Associates the VRF with the bridge-domain and exits the configuration mode.

Step 10

application application-profile-name

Example:

apic1(config-tenant)# application app64

Creates or identifies the application profile associated with the tenant and the EPG.

Step 11

epg epg-name

Example:

apic1(config-tenant)# epg epg64

Creates or identifies the EPG and enters into EPG configuration mode.

Step 12

bridge-domain member bridge-domain-name

Example:

apic1(config-tenant-app-epg)# bridge-domain member bd64
apic1(config-tenant-app-epg)# exit
apic1(config-tenant-app)# exit
apic1(config-tenant)# exit

Associates the EPG with the bridge domain and returns to global configuration mode.

Configure the sub ports as desired, for example, use the speed command in leaf interface mode to configure a sub port.

Step 13

leaf leaf-name

Example:


apic1(config)# leaf 1017
apic1(config-leaf)# interface ethernet 1/13
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# switchport trunk allowed vlan 20 tenant t1 application AP1 epg EPG1

Note

 

The vlan-domain and vlan-domain member commands mentioned in the above example are a pre-requisite for deploying an EPG on a port.

Associates the EPG with a break-out port.

Step 14

speed interface-speed

Example:


apic1(config)# leaf 101
apic1(config-leaf)# interface ethernet 1/16/1
apic1(config-leaf-if)# speed 10G
apic1(config-leaf-if)# exit

Enters leaf interface mode, sets the speed of an interface, and exits the configuration mode.

Step 15

show run

Example:

apic1(config-leaf)# show run 

After you have configured the sub ports, entering this command in leaf configuration mode displays the sub port details.

The port on leaf 101 at interface 1/16 is confirmed enabled for breakout with sub ports 1/16/1, 1/16/2, 1/16/3, and 1/16/4.

Example

This example configures the port for breakout:

apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface ethernet 1/16
apic1(config-leaf-if)# breakout 10g-4x

This example configures the EPG for the sub ports.

apic1(config)# tenant tenant64
apic1(config-tenant)# vrf context vrf64
apic1(config-tenant-vrf)# exit
apic1(config-tenant)# bridge-domain bd64
apic1(config-tenant-bd)# vrf member vrf64
apic1(config-tenant-bd)# exit
apic1(config-tenant)# application app64
apic1(config-tenant-app)# epg epg64
apic1(config-tenant-app-epg)# bridge-domain member bd64
apic1(config-tenant-app-epg)# end

This example sets the speed for the breakout sub ports to 10G.

apic1(config)# leaf 101
apic1(config-leaf)# interface ethernet 1/16/1
apic1(config-leaf-if)# speed 10G
apic1(config-leaf-if)# exit                                                                          
apic1(config-leaf)# interface ethernet 1/16/2
apic1(config-leaf-if)# speed 10G
apic1(config-leaf-if)# exit
apic1(config-leaf)# interface ethernet 1/16/3
apic1(config-leaf-if)# speed 10G
apic1(config-leaf-if)# exit
apic1(config-leaf)# interface ethernet 1/16/4
apic1(config-leaf-if)# speed 10G
apic1(config-leaf-if)# exit

This example shows the four sub ports connected to leaf 101, interface 1/16.

apic1#(config-leaf)# show run
# Command: show running-config leaf 101
# Time: Fri Dec  2 00:51:08 2016
  leaf 101 
    interface ethernet 1/16/1
      speed 10G
      negotiate auto
      link debounce time 100
      exit
    interface ethernet 1/16/2
      speed 10G
      negotiate auto
      link debounce time 100
      exit
    interface ethernet 1/16/3
      speed 10G
      negotiate auto
      link debounce time 100
      exit
    interface ethernet 1/16/4
      speed 10G
      negotiate auto
      link debounce time 100
      exit
    interface ethernet 1/16
      breakout 10g-4x
      exit
    interface vfc 1/16

Configuring Dynamic Breakout Ports Using the REST API

Configure a Breakout Leaf Port with an Leaf Interface Profile, associate the profile with a switch, and configure the sub ports with the following steps.

For switch support for the breakout feature, see Configuration of Dynamic Breakout Ports.

Procedure

Before you begin

  • The ACI fabric is installed, APIC controllers are online, and the APIC cluster is formed and healthy.

  • An APIC fabric administrator account is available that will enable creating the necessary fabric infrastructure configurations.

  • The target leaf switches are registered in the ACI fabric and available.

  • The 40GE or 100GE leaf switch ports are connected with Cisco breakout cables to the downlink ports.

Procedure


Step 1

Configure a breakout policy group for the breakout port with JSON, such as the following example:

Example:

In this example, we create an interface profile 'brkout44' with the only port 44 underneath its port selector. The port selector is pointed to a breakout policy group 'new-brkoutPol'.

{
            "infraAccPortP": {
              "attributes": {
                "dn":"uni/infra/accportprof-brkout44",
                "name":"brkout44",
                "rn":"accportprof-brkout44",
                "status":"created,modified"
                },
                "children":[ {
                     "infraHPortS": {
                        "attributes": {
                          "dn":"uni/infra/accportprof-brkout44/hports-new-brekoutPol-typ-range",
                          "name":"new-brkoutPol",
                          "rn":"hports-new-brkoutPol-typ-range",
                          "status":"created,modified"
                     },
                     "children":[ {
                        "infraPortBlk": {
                          "attributes": {
                            "dn":"uni/infra/accportprof-brkout44/hports-new-brkoutPol-typ-range/portblk-block2",
                            "fromPort":"44",
                            "toPort":"44",
                            "name":"block2",
                            "rn":"portblk-block2",
                            "status":"created,modified"
                         },
                         "children":[] }
                  }, {
            "infraRsAccBaseGrp": {
              "attributes":{
                 "tDn":"uni/infra/funcprof/brkoutportgrp-new-brkoutPol",
                 "status":"created,modified"
              },
              "children":[] 
             }
           }
         ]
       }
     }
   ]
 }
}

Step 2

Create a new switch profile and associate it with the port profile, previously created, with JSON such as the following example:

Example:

In this example, we create a new switch profile 'leaf1017' with switch 1017 as the only node. We associate this new switch profile with the port profile 'brkout44' created above. After this, the port 44 on switch 1017 will have 4 sub ports.

Example:

{
   "infraNodeP": {
     "attributes": {
       "dn":"uni/infra/nprof-leaf1017",
       "name":"leaf1017","rn":"nprof-leaf1017",
       "status":"created,modified"
      },
      "children": [ {
        "infraLeafS": {
          "attributes": {
            "dn":"uni/infra/nprof-leaf1017/leaves-1017-typ-range",
            "type":"range",
            "name":"1017",
            "rn":"leaves-1017-typ-range",
            "status":"created"
      },
      "children": [ {
        "infraNodeBlk": {
          "attributes": {
            "dn":"uni/infra/nprof-leaf1017/leaves-1017-typ-range/nodeblk-102bf7dc60e63f7e",
            "from_":"1017","to_":"1017",
            "name":"102bf7dc60e63f7e",
            "rn":"nodeblk-102bf7dc60e63f7e",
            "status":"created"
           },
           "children": [] }
          }
        ]
      }
    }, {
      "infraRsAccPortP": {
        "attributes": {
          "tDn":"uni/infra/accportprof-brkout44",
          "status":"created,modified"
     },
     "children": [] }
      }
    ]
  }
}

Step 3

Configure the subports.

Example:

This example configures subports 1/44/1, 1/44/2, 1/44/3, 1/44/4 on switch 1017, for instance, in the example below, we configure interface 1/44/3. It also creates the infraSubPortBlk object instead of the infraPortBlk object.

{
   "infraAccPortP": {
     "attributes": {
       "dn":"uni/infra/accportprof-brkout44",
       "name":"brkouttest1",
       "rn":"accportprof-brkout44",
       "status":"created,modified"
     },
     "children": [{
       "infraHPortS": {
         "attributes": {
           "dn":"uni/infra/accportprof-brkout44/hports-sel1-typ-range",
           "name":"sel1",
           "rn":"hports-sel1-typ-range",
           "status":"created,modified"
        },
        "children": [{ 
          "infraSubPortBlk": {
            "attributes": {
              "dn":"uni/infra/accportprof-brkout44/hports-sel1-typ-range/subportblk-block2",
              "fromPort":"44",
              "toPort":"44",
              "fromSubPort":"3",
              "toSubPort":"3", 
              "name":"block2",
              "rn":"subportblk-block2",
              "status":"created"
             },
             "children":[]}
             },
           {
             "infraRsAccBaseGrp": {
                 "attributes": {
                   "tDn":"uni/infra/funcprof/accportgrp-p1",
                   "status":"created,modified"
                 },
                 "children":[]}
                 }
               ]
             }
           }
         ]
       }
     }

Step 4

Deploy an EPG on a specific port.

Example:

<fvTenant name="<tenant_name>" dn="uni/tn-test1" >
    <fvCtx name="<network_name>" pcEnfPref="enforced" knwMcastAct="permit"/>
    <fvBD name="<bridge_domain_name>" unkMcastAct="flood" >
        <fvRsCtx tnFvCtxName="<network_name>"/>
    </fvBD>
    <fvAp name="<application_profile>" >
        <fvAEPg name="<epg_name>" >
            <fvRsPathAtt tDn="topology/pod-1/paths-1017/pathep-[eth1/13]" mode="regular" instrImedcy="immediate" encap="vlan-20"/>
        </fvAEPg>
    </fvAp>
</fvTenant>