Connecting the Switch to the ACI Fabric

ACI Fabric Topology

The ACI fabric topology includes the following major components:

  • Application Centric Infrastructure Controller (APIC) appliance (cluster of APICs)

  • Leaf switches (Cisco Nexus 93108TC-EX, 93120TX, 93128TX, 93180LC-EX, 93180YC-EX, 9332PQ, 9372PX, 9372PX-E, 9372TX, 9372TX-E, 9396PX, and 9396TX switches)

  • Spine switches (Cisco Nexus 9336PQ, 9504, 9508, and 9516 switches)

As shown in the following figure, each APIC is connected to one or two leaf switches and each leaf switch should be connected to every spine switch in the same fabric.


Note

To prevent sub-optimal forwarding between endpoints, connect every leaf switch in the fabric to every spine switch in the same fabric.



Preparing to Connect to Other Devices

When preparing to connect the fabric devices, consider the following for each type of interface, and gather all of the required equipment before making the connections:

  • Cabling type required for each interface type

  • Distance limitations for each signal type

  • Additional interface equipment required


Note

When running power and data cables in overhead or subfloor cable trays, we strongly recommend that you locate power cables and other potential noise sources as far away as practical from network cabling that terminates on Cisco equipment. In situations where long parallel cable runs cannot be separated by at least 3.3 feet (1 meter), we recommend that you shield any potential noise sources by housing them in a grounded metallic conduit.

The optical transceivers that are not already assembled to their cables come separate from their cables. To prevent these transceivers and their cables from being damaged, we recommend that you keep the transceivers disconnected from their cables when installing them in ports and then insert the optical cable into the transceiver. When removing transceivers from ports, remove their cables before removing the transceivers.

To maximize the effectiveness and life of your transceivers and optical cables, do the following:

  • Wear an ESD-preventative wrist strap that is connected to an earth ground whenever handling transceivers. The switch is typically grounded when you install transceivers and provides an ESD port to which you can connect your wrist strap. If you cannot find an ESD port, connect the wrist strap to an earth ground (such as the grounding connection for the chassis).

  • Do not remove or insert a transceiver more often than necessary. Repeated removals and insertions can shorten its useful life.

  • Keep the transceivers and fiber-optic cables clean and dust free to maintain high signal accuracy and to prevent damage to the connectors. Attenuation (loss of light) increases with contamination and should be kept below 0.35 dB.

    • Clean these parts before installing them to prevent dust from scratching the fiber-optic cable ends.

    • Clean the connectors regularly; the required frequency of cleaning depends upon the environment. In addition, clean connectors if they are exposed to dust or accidentally touched. Both wet and dry cleaning techniques can be effective; refer to your site's fiber-optic connection cleaning procedures.

    • Do not touch the ends of connectors. Touching the ends can leave fingerprints and cause other contamination.

  • Inspect routinely for dust and damage. If you suspect damage, clean and then inspect fiber ends under a microscope to determine if damage has occurred.

Connecting Leaf Switches to APICs

You must downlink one or two (recommended for redundancy) ACI-mode leaf switches (Cisco Nexus 93108TC-EX, 93120TX, 93128TX, 93180LC-EX, 93180YC-EX, 9332PQ, 9372PX, 9372PX-E, 9372TX, 9372TX-E, 9396PX, or 9396TX) to each Application Policy Infrastructure Controller (APIC) in your ACI fabric. The type of virtual interface card (VIC) installed on the APIC determines the types of interface cables that you can use to connect the leaf switches to the APICs.

  • The VIC1225 module supports optical transceivers, optical cables, and switches with optical downlink ports (Cisco Nexus 93180LC-EX, 93180YC-EX, 9332PQ, 9372PX, 9372PX-E, and 9396PX switches).

  • The VIC1225T module supports copper connectors, copper cables, and switches with copper downlink ports (Cisco Nexus 93108TC-EX, 93120TX, 93128TX, 9372TX, 9372TX-E, and 9396TX switches).

Before You Begin

The APIC and leaf switches in the fabric must be fully installed in their racks and grounded.

    Step 1   Connect an interface cable to one of the two ports on the virtual interface card (VIC) installed on the APIC. If the cable is not already assembled to its transceivers, insert the transceiver into the VIC port and then connect the optical interface cable to the transceiver.
    Step 2   Connect the other end of the interface cable to a downlink port on a leaf switch.

    • For a Cisco 10GBASE-LR or -SR transceiver and cable, insert the transceiver into a downlink optical port on a leaf switch before connecting the cable to the transceiver.

    • For Cisco SFP+ Active Optical Cables, insert the transceiver on the cable into a downlink optical port on a leaf switch.

    • For a 10GBASE-T copper cable, insert the RJ-45 connector on the cable into a downlink BASE-T port on a leaf switch.

    Note   

    To determine which transceivers and cables are supported by this switch, see http:/​/​www.cisco.com/​c/​en/​us/​support/​interfaces-modules/​transceiver-modules/​products-device-support-tables-list.html.

    Connecting Leaf Switches to Spine Switches

    For optimal forwarding between endpoints, you must connect each leaf switch (Cisco Nexus 93108TC-EX, 93120TX, 93128TX, 93180LC-EX, 93180YC-EX, 9332PQ, 9372PX, 9372PX-E, 9372TX, 9372TX-E, 9396PX, or 9396TX) to every spine switch (Cisco Nexus 9336PQ, 9504, 9508, or 9516) in the same ACI fabric. The following table lists the number of ports that you can connect on each type of leaf switch and the supported speeds for those ports.

    Leaf Switch

    Maximum Number of Uplink Connections

    Supported Transmission Speeds (Uplink Ports)

    Cisco Nexus 93108TC-EX

    6 QSFP28 fixed ports

    10, 40, and 100 Gigabits1

    Cisco Nexus 93120TX

    6 QSFP+ fixed ports

    40 Gigabits

    Cisco Nexus 93128TX

    with M6PQ or M4PQ-E uplink module

    6 QSFP+ ports

    40 Gigabits

    with M12PQ uplink module

    8 QSFP+ ports (leftmost 8 ports are supported on 12-port module)

    1, 10, or 40 Gigabits1

    Cisco Nexus 93180LC-EX

    6 QSFP28 fixed ports

    40, and 100 Gigabits

    Cisco Nexus 93180YC-EX

    6 QSFP28 fixed ports

    10, 40, and 100 Gigabits1

    Cisco Nexus 9332PQ

    6 QSFP+ fixed ports

    10 or 40 Gigabits1

    Cisco Nexus 9372PX

    Cisco Nexus 9372PX-E

    Cisco Nexus 9372TX

    Cisco Nexus 9372TX-E

    6 QSFP+ ports

    40 Gigabits

    Cisco Nexus 9396PX

    Cisco Nexus 9396TX

    with M6PQ or M4PQ-E uplink module

    6 QSFP+ ports

    40 Gigabits

    with M12PQ uplink module

    12 QSFP+ ports

    10 or 40 Gigabits1
    1 For 10-Gigabit support, use a QSFP-to-SFP adapter (such as CVR-QSFP-SFP10G) with a SFP+ or SFP transceiver.

    The following table lists the number of ports that you can connect on each ACI-mode line card installed in a modular spine switch (Cisco Nexus 9504, 9508, or 9516 switch) or on each 9336PQ spine switch. The number of line cards in a spine switch depends on the model of the spine switch (the Cisco Nexus 9504 supports four line cards, the Cisco Nexus 9508 supports up to eight line cards, and the Cisco Nexus 9516 supports up to 10 line cards in ACI-mode).

    Spine Switch or Modular Line Card

    Maximum Number of Uplink Connections

    Supported Transmission Speeds

    Cisco Nexus 9336PQ

    36 QSFP+ fixed ports

    10 or 40 Gigabits

    N9K-X9736PQ line card supported by Cisco Nexus 9504, 9508, and 9516 modular switches

    36 QSFP+ fixed ports

    1, 10, or 40 Gigabits

    To determine which transceivers, adapters, and cables are supported by this switch, see http:/​/​www.cisco.com/​c/​en/​us/​support/​interfaces-modules/​transceiver-modules/​products-device-support-tables-list.html. To see the transceiver specifications and installation information, see http:/​/​www.cisco.com/​c/​en/​us/​support/​interfaces-modules/​transceiver-modules/​products-installation-guides-list.html.


    Warning

    Statement 1053—Class 1M Laser Radiation

    Class 1M laser radiation when open. Do not view directly with optical instruments.


    Warning

    Statement 1055—Class I and Class 1M Laser

    Class I (CDRH) and Class 1M (IEC) laser products.


    Warning

    Statement 1056—Unterminated Fiber Cable

    Invisible laser radiation may be emitted from the end of the unterminated fiber cable or connector. Do not view directly with optical instruments. Viewing the laser output with certain optical instruments (for example, eye loupes, magnifiers, and microscopes) within a distance of 100 mm may pose an eye hazard.

    Before You Begin

    • The leaf and spine switches in the fabric must be fully installed in their racks and grounded.

    • If there are modular switches in the fabric, their ACI-mode line cards must already be installed. The line cards can be of the following types:

      • 36-port 40-Gigabit (N9K-X9736PQ)

      • 32-port 100-Gigabit (N9K-X9732C-EX) (supported by Cisco Nexus 9504 and 9508 modular switches)


    Note
    You cannot include NX-OS line cards in the same chassis when running in ACI mode.
      Step 1   For the transceivers with removable cables, make sure that the transceivers are separated from their interface cables.
      Step 2   Insert the appropriate transceiver into an active uplink port on the leaf switch.
      Note   

      For 1- or 10-Gigabit Ethernet on a Cisco Nexus 93108TC-EX or 93180YC-EX switch or a N9K-X9732C-EX, N9K-X9736PQ, or N9K-X9432PQ line card, insert a QSFP-to-SFP adapter (CVR-QSFP-SFP10G) before inserting an SFP+ transceiver.

      Step 3   Insert the same type of transceiver in the spine switch port on a X9736PQ line card.
      Step 4   For transceivers with removable cables, insert the interface cable into the open end of each of those transceivers.
      Step 5   Repeat Steps 1 through 4 for each spine switch in the ACI fabric. The leaf switch is connected to each spine switch in the ACI fabric.
      Step 6   Repeat Steps 1 through 5 for each leaf switch in the ACI fabric. Each leaf switch in the ACI fabric is connected to each spine switch in the network,

      The fabric automatically implements Equal Cost Multi-Pathing (ECMP) and enables all links. You do not need to configure the links.

      Setting Up an Optional Console Interface

      You can optionally set up a console interface for performing the initial configuration of the switch. To do this, use the interface cable provided in the accessory kit to connect the switch to your console device. You can connect the console port on the switch to a modem. If you do not connect it to a modem, make the connection either before powering up the switch or after completing the boot process for the switch.

      Before You Begin

      The console device must support VT100 terminal emulations and asynchronous transmissions.

        Step 1   Configure the terminal emulator program to match each of the following default port characteristics:

        • 9600 baud

        • 8 data bits

        • 1 stop bit

        • No parity

        Step 2   Insert the RJ-45 connector on the interface cable found in the accessory kit into the RS-232 port on the switch and insert the DB-9 connector on the other end of the cable to the serial port on the console device.
        What to Do Next

        You can now perform the initial configuration for the switch (see the Cisco ACI Getting Started Guide).

        Setting Up an Optional Management Connection

        You can optionally set up an out-of-band management connection for monitoring and troubleshooting purposes. To do this, you connect either the RJ-45 management port or the SFP management port on the switch to an external hub, switch, or router.

        Before You Begin

        To prevent an IP address conflict, you must complete the initial configuration for the switch and establish an IP address before you create the management connection.

          Step 1   Connect the appropriate modular interface cable to only one of the two management ports on the switch.

          • For the RJ-45 management port, use a copper interface cable with RJ-45 connectors (can be used for shorter connections).

          • For the SFP management port, use an optical interface cable with LH or SX SFP transceivers (can be used for longer connections).

          Note   

          If you use the management interface, connect only one of the two management ports. The switch does not support your use of two management ports at the same time.

          Step 2   Connect the other end of the cable to an external hub, switch, or router.

          Maintaining Transceivers and Optical Cables

          Transceivers and fiber-optic cables must be kept clean and dust free to maintain high signal accuracy and prevent damage to the connectors. Attenuation (loss of light) is increased by contamination and should be below 0.35 dB.

          Consider the following maintenance guidelines:

          • Transceivers are static sensitive. To prevent ESD damage, wear an ESD-preventative wrist strap that is connected to the grounded chassis.

          • Do not remove and insert a transceiver more often than is necessary. Repeated removals and insertions can shorten its useful life.

          • Keep all optical connections covered when not in use. Clean them before using to prevent dust from scratching the fiber-optic cable ends.

          • Do not touch the ends of connectors. Touching the ends can leave fingerprints and cause other contamination.

          • Clean the connectors regularly; the required frequency of cleaning depends upon the environment. In addition, clean connectors if they are exposed to dust or accidentally touched. Both wet and dry cleaning techniques can be effective; refer to your site's fiber-optic connection cleaning procedures.

          • Inspect routinely for dust and damage. If you suspect damage, clean and then inspect fiber ends under a microscope to determine if damage has occurred.