Welcome to the end to end life cycle presentation of an AI optimized fabric using Cisco Nexus dashboard. Today, we explore the journey from initial provisioning and lossless configuration to real time performance validation.00:05Focusing on how intelligent packet Spring manages elephant flows in your AI backend. This session is crafted by a Porva talker, a customer success specialist, and is aimed at providing you with detailed insights on improving your network efficiency. Today's agenda will provide a structured overview of our session.00:18Initially, we'll start with an overview of AI and ML infrastructure. Then discuss the Cisco nexus 9300 hardware solutions followed by exploring various network architecture options and AI fabric building blocks. We will address traffic load balancing challenges.00:35And then move on to discuss dynamic load balancing and intelligent packet flow. Finally, we will conclude with a demonstration of these concepts in action. Designing for AI requires evolving beyond traditional data center architecture. We employ a two tier approach, the front end network handles standard user to app interactions.00:51While the back end network, the core of the AI cluster is a high bandwidth, lossless fabric for low latency, inner GPU communication, essential for intensive training, segregating these prevents management traffic from bottlenecking AI tasks. Distributed training is offline and compute intensive.01:11Requiring high node to node bandwidth to maximize training speed. Conversely, production inference operates live prioritizing high availability and low latency for real time responses. Differentiating these is pivotal for infrastructure planning.01:30Cisco's nexus 9300 series 400 G switches provide high speed throughput and deep buffering critical for lossless AI fabrics. The 800 G switch offers 51.2 tb capacity and 256 mb buffers doubling previous capacities and ensuring future proofing with.01:48Ultra ethernet readiness.02:08Nexus dashboard integrates existing ACI or vxlane EVPN fabrics. We discuss two design options, the classic design where nodes connect to a single leaf and the rail optimized design, where GPU, Nick pairs connect to unique rails across multiple leaves to optimize all reduced patterns both.02:11To use non blocking BGP fabrics.02:31To achieve a lossless fabric, we use RASAVV two to bypass the CPU PFC and ECN to manage buffer occupancy and DCQCN/red for congestion control. Advanced load balancing ensures optimal link utilization. Next we focus on load balancing techniques.02:35Traditional ECMP five tuple hashing often fails in AI networks causing traffic polarization where one spine is overburdened. Dynamic load balancing DLB, solves this by monitoring link utilization in real time and rerouting traffic using flow led hashing.02:56Real world data shows DLB maintains 50 % utilization across ports maximizing throughput. Effective traffic flow management is pivotal in optimizing an AI network as shown in the current slide. On one side, we have the ECMP static pinning approach by assigning network flows to specific paths, static.03:16The pinning offers predictability.03:36Critical for troubleshooting and maintaining consistent network behavior in traditional data centers. Contrastingly, for ai's long lived flows, static pinning may lead to congestion. This is where per packet or packet spring becomes beneficial. Instead of confining a flow to one path, packets are distributed across multiple available spine lengths.03:38By utilizing all available paths, packet spring mitigates bottlenecks, leveraging the aggregate bandwidth of all spines. Key outcomes include static pinning provides the predictability needed for standard traffic. Packet spring reduces congestion for ai's extensive data flows.03:57Cisco Nexus switches utilize these intelligent packet techniques offering tailored load balancing strategies to meet diverse workload demands ensuring both consistency and high performance in any AI fabric. Combining Cisco per packet DLB with NVIDIA spectrum X adaptive routing yields a 37 % performance gain.04:12Improving jane's fairness index to 1.000.04:32With cisco's per packet load balancing and NVIDIA spectrum X, the results are shown here. This is what we mean by optimized traffic distribution for AI networks. By distributing packets across all links, we eliminate congestion and ensure there are NO underutilized paths. For aI/ML workloads, such uniformity leads to consistent, high speed training.04:38Maximizing infrastructure ROI and efficiency.04:59Supported hardware includes specific nexus 9300 platforms and NVIDIA Bluefield three supernix. Firmware must support the LLDP TLV handshake for adaptive routing. Licensing is per switch. Use the XF two or XF three license family based on your hardware PID.05:04This diagram demonstrates a classic leaf spine fabric with both 100 gbps and 400 gbps connections. The diagram shows high speed 400 gbps backbone lengths in dashed green and 100 gbps links in-light blue. The solid blue lines highlight active traffic paths from compute hosts, hosts.05:24One through five communicating with a centralized storage node at interface E1/eleven on leaf L two oh six, this many to one pattern can create congestion.05:44These screenshots shows what are the real time traffic rates on links between leaf two oh six and different spines. As we can observe here, the links are not getting equally utilized. Some links are overutilized while others are under utilized. Monitoring ECN and output discards identifies congestion management.05:55We observe that there are some output discards from the leaf towards the storage endost. Additionally, we see that the packets are not marked with ECN even though there is congestion and packet drops. We are also able to see that the storage end host is sending PPP pause frames to notify about congestion. In the next steps, we will see how Cisco's deal.06:14ALB and other features of lossless Ethernet are able to solve this issue.06:34Now, let's move to the Nexus dashboard to set up dynamic load balancing with priority flow control to provision, navigate to manage fabrics in the Nexus dashboard. Select create new LAN fabric, and then select fabric type as AI and select AI routed sub.06:39Type.06:59Give a name to the fabric. Define BGP ASN, 400 G fabric and telemetry settings. In the advanced settings stage, we have the option to customize further, but for this demonstration, we'll stick with the recommended defaults. These defaults provide a balance of flexibility and reliability designed with industry best practices.07:02We'll click next to proceed to the summary before starting the fabric creation.07:22We'll click next to proceed to the summary before starting the fabric creation. On the next page, you can review the accuracy of all information. After verifying the summary, submit to trigger automated creation.07:28With a submission complete, we are now, in the fabric creation phase, now the fabric is created. Once the fabric is created.07:45Discover switches using seed IP and credentials. Select site 1L1L2S1 and S two and add them to the fabric. Then assign spine roles to. S one and S two.08:03Acknowledge the recalculate and deploy warning to validate topology. Access the actions menu and select recalculate and deploy. With recalculate and deploy initiated, the Nexus dashboard is recalculating the required configurations to our selected switches.08:23This process ensures all configurations aligned perfectly with our fabric design, supporting an AI optimized infrastructure. On this page, we can preview configs, compare running, pending, and expected states, then deploy.08:44In this screen, the nexus dashboard offers a detailed view of pending config for site 1L1 device. We're looking at the site 1L1 switch specifically. The dashboard displays the exact commands that will be implemented, including quality of service, queuing policies, and MTU settings tailored for our AI network.09:02After the preview, we have now proceeded with deploying the configs on all the devices. Once deployment is complete, the switch show commands tool can be used to verify operational status directly from the Nexus dashboard. We've entered the show run ipco's command to check the quality of service settings on our site 1L1 switch.09:24The real time output confirms that our COZ policies including Rosav two classifications are correctly established. Turning to host connectivity, we focus on managing interfaces through the connectivity tab.we selected ethernet one/five on site 1L1, which is connected to our storage host in this lab.09:43Edit interface E1/five for AI workloads enabling PFC and coast configuration. Again, we can observe here that Nexus dashboard is generating the necessary conflicts for the ethernet port one 5th. This display shows the current state of the switch port configuration.10:04Here, we explore the pending config view. This display outlines the specific commands generated by the Nexus dashboard to transition from the current configuration to your desired design. Let's move on to the expected config view. This serves as a projection of what the configuration will look like once all pending changes are deployed for interface.10:25We are now in the deploy progress stage where the Nexus dashboard is actively applying the optimized settings to interface ethernet one/five on site 1L1. This automation seamlessly moves from the expected config to the actual switch, eliminating manual entry and reducing human errors.10:47Once the config is applied, we observe now in production, the results show perfectly balanced traffic across interfaces, delivering production grade performance for i/ML applications. The leaf interfaces now utilize all the interfaces equally. Thank you for watching. Subscribe for more advanced networking topics.11:05
