AI Networking with Nexus Dashboard

Hello, good morning, everyone, and thank you for joining us for today's expert insight series on AI ready infrastructure with Nexus dashboard.00:10My name is Don and I will be moderating our session today. With that I have two excellent experts Carlos and Jeffrey. They're gonna go over the topic today and the content.00:20We also have some other colleagues of mine in the panelists that are gonna be here to help with any Q and A that you may have. Speaking of Q and A, we are gonna be utilizing Slido for the Q and A today as well as some poll questions that we may have throughout the presentation.00:35Please feel free to join us@Slido.com. You can do so by scanning this QR code and putting in the passcode of in the_AI. You can also go to slido.com and use the meeting code in the_AI and the passcode as well in the_AI.00:55You can also join the Slido here in Webex by using the apps link in the lower right hand corner, clicking that, and then clicking Slido. We're gonna go ahead and start with a, just a quick icebreaker. If you don't mind joining in Slido, let us know that.01:13It's working by letting us know where you're joining us from today. So I'm gonna go ahead and start that question. So you should be able to now go in and provide your response and let us know where in the world are you today and where you, where you joining us from?01:33Okay, I'm not seeing anything in the slido, but I do see in the chat we got someone joining us from Quebec. Very cool. Seattle. Awesome. Good morning out in Seattle, Texas. Very cool. Colorado. Nice.02:11Now we get in the, the coming to Denver Philippines but from Tennesssee a long way from home. That's very cool. Miami, not far from me, I'm just up the road for Miami. Excellent. I appreciate everyone participating and letting us know where you're joining from and just helping us make sure that slide.02:30It was working properly, let's go ahead and move forward.02:50Today we're gonna start with talking about networking for AI cluster and lossless features, traffic load balancing, Nexus dashboard for AI clusters and Nexus dashboard monitoring. To help me with this content, I am gonna turn things over to my steam club.02:56Carlos, after this next Slido question, so this next question, you know.03:15What do you find to be the most significant technical hurdle when deploying lossless internet in your current data center? So what are some of the things that are maybe holding you up or preventing you or or or might be speed bumps that.03:21Kind of keep you up tonight when you talk about doing that in your data center for your AI workloads. So that, that poll is now open. So go ahead and answer that.03:38I got a couple options there for you, so let us know, kind of what your concerns are, ok? I see we've got lack of end to end visibility across your NICs and GPO use and networks, initial configuration and QS policy tuning, that's yeah that's a.03:56Bit of tricky one. Looks like those two are the two biggest concerns so far out of the votes that we've got, we'll give them another couple seconds to squeeze in another vote or two if you.04:13Yeah, there we go and 5056 votes total, for initial configuration and QS policy and lack of end to end visible across next GPUs and networks. Okay.04:30So we're definitely gonna talk about both of those things today and to do so, I will go ahead and turn things over to Carlos to kick us off on that. Thank you, Don. Hello everyone.04:46So let's start by looking at networking for AI and some relevant aspects of networking within an AI cluster. So how does AI work and what do you need to have in place to make a.05:04AI work for you. There are two main operations. They are not limited to these two, but these are kind of the two main ones. The 1st operation is, you know, distributed training and you know 2nd one is production inference. So how can we distinguish between the two?05:20In essence, distributed training is about building and preparing the model often using significant computational power While production inference is about, you know, deploying and using the model to generate real time predictions efficiently.05:37When we talk about AI networking, it's important to understand what's, understand that it's not that single unified network but several networks. This slide is just to give you a very high level overview of how.05:57An AI cluster is built from a networking perspective. So we have our GPU GPU notes. Those might be, for instance, you know, UCS servers with GPU, GPUs installed and those nodes are connected to several networks.06:151st, we have a front end network which allows users and applications to access the AI cluster. E.g., to send a request to an AI model and receive a response. This could be your existing AI.06:34Or your existing ACI fabrics e.g.. Then there is an storage network which is going to give the cluster nodes access to, you know, shared storage devices that provide the data sets they need.06:51And, in some designs, the storage network can be converged with the front end network and to enable communication between GPUs.07:09Or between GPU notes, you are going to need a dedicated inter GPU backend network. This is far the most demanding segment as the bandwidth and latency requirements are significant. It's where the lossless networking and the load balancing features we're gonna be talking about today.07:24Become more important. Finally, there is a management network which provides, you know, out of band connectivity to the devices of the or of your AI infrastructure. And.07:44So we have several networks in a cluster. Another important concept is, that within an AI cluster, some traffic runs over TCP, and this is generally the inference traffic from users.08:00On the front end network While other portion of the traffic comes over Rocky V two in the backend network. Rocky V two stands for, you know, RDMA over converged ethernet version two, and it's the protocol that allows RDM.08:18Made to run over standard Ethernet networks.08:38And it uses UDP as if its transport protocol. Our DMA enables one server in an AI cluster to directly access the memory of another servers over the network without involving, you know, the CPU, the operating system or the full network.08:42Or networking stack in every step. So it provides ultra low latency, high throughput, and minimal CPU overhead. And keep in mind that UDP as transferred protocol doesn't have a building mechanism to detect and retransmit packets.09:02Lost in the network. That's why we need the the network to be us and you know, it also needs to be or provide high throughput and low latency. So rocky V two requires a loss network.09:22That's why we use a combination of well known protocols. We are gonna use priority flow control BFC to prioritize rocky B two traffic across the layer two and the layer three network. We also need ECM.09:40ECM marking or you know explicit congestion to to send the notifications when a link approaches ingestion. As I said earlier, the transport network.09:56Must provide high throughput and low latency. So the Cisco nexus 9000 is a very good choice for this with it's you know 2025 .6 /s of bandwidth per ASIC.10:15Here we're we're talking about the silicon 100 asic. This bring us to our second poll for today. So back to you, Don. So let me get this poll started.10:32Oh, bear with me just a second. Here we go. Waiting for votes now. So as you scale your AI cluster, what is.10:50That's your primary infrastructure concern over the next twelve or so months. What is it that's on the top of your mind and the, you know, top of the to do list to see we've got scaling bandwidth and moving from moving to 400 or 800 gig?11:10That's a big one. Operational simplicity and automated fabric life cycle management. That's, that's understandable a big one and real time visibility into GPU to GPU communication, that kind of goes hand in hand with what we saw earlier when one of your concerns was lack of visibility into your GPU to GPU communication, so.11:25Yeah, seems to be kind of split right along lines here of, of scaling, you know, moving up and getting in bandwidth and that real time visibility and then of course trying to do this as, as automated and and simplified as possible, so.11:45I'm glad you all are here because these are the sort of things that we're, we're thinking about as well and that we're doing. So I'm gonna go ahead and close this slido poll and Carlos I'm gonna shift things back over to you sir. Sure.12:01Thank you, Don. So the 1st thing we need to do is prepare our network to operate as the last network, so we're gonna use several features. The 1st one is ECM, which is an end to end congestion signaling mechanism designed to, you know, help prevent packet drops.12:18So how it works is part of the IP header, it uses the two list significant bits of the eight bit type of service field. The blue table in the screen shows the four combinations of those two bits.12:38001001 and one one. Each value represents a different status. The main ones are one zero and zero one which are going to indicate that the traffic is isn't capable.12:55A value of zero zero means that the traffic is not easy and capable. If the field is marked as one one, it means that a device in the network has detected congestion and marked the packet as congestion.13:11Encountered or see. Now, if we look at the diagram above the table, as an example, we have a computer on the left, storage on the right, and two switches connecting them. And let's say that 2nd switch experiences.13:27Congestion. When the ECN capable traffic reaches that switch, it marks is the ECM field as one one. It is going to indicate to the destination that congestion has been detected along the path.13:46Once the storage receives the market packets, it generates a congestion notification packet or C CMP, and the.14:01Directly to the source as unicast traffic. In this way, the storage notifies the sources that the network is approaching congestion and that it needs to, you know, reduce its its transmission rate.14:17So this is going to give the network time to clear the buffers and resume normal traffic flow. So by slowing down the source, we can prevent further congestion. All these ECM packets.14:33Marked with one one in the network. There are different algorithms for doing this. The 1st one, which is widely used is double red or, you know, wedded random every detection. So how does it work in the switch? Basically, it works within the queues.14:53When you configure W red on the switch, you choose two thresholds. W red mean and W red math. In our 1st moment, if the buffered usage is lower than W red mean threshold, nothing happens.15:12The packets continues continue to flow. They are marked as ECM capable but NO other actions are taken. If in a second moment the buffer usage rises above the mean threshold, the system marks a certain number of packets with one.15:29One, and the number of packets that are going to be marked.15:48Depends on the drop probability configured on the switch. If that doesn't solve the problem, buffered usage continues to grow and eventually in a 3rd moment, it's going to cross the double red max threshold.15:53And in this case, every ECM capable packet is going to be marked as one one. This is going to tell the server that, that is a severe congestion and that it needs to slow down.16:10So when the send, when the sender receives the congestion of notification, it's going to slow down the specific flow that matches the, the notification. Another feature we can use is the approximate fail.16:26Drop or AFD. It's very similar to W red, but works slightly different.16:46This feature help us decide which packets should be marked with ECN. We can use AFD to identify which flows are considered elephant flows and which ones are considered non elephant or.16:54Mice flows. We do not want to impact the mice flows and make them you know to reduce their transmission rate. Instead, we want to, we want the notification packet to send.17:09Or to be sent for the elephant flows so that they can reduce their rate. To do that, when we configure AFD, we set a parameter called ideal depth, which refers to the ideal depth of the buffer. So if the.17:26Over usage for a flow exceeds that limit, we consider it an elephant flow and its traffic can be marked with ECN one one.17:46A number of packets marked with ECN depends on how much bandwidth the flow is using. E.g., a flow running at one gig has fewer packets marked than a flow running at five gigs. So AFD triggers a different.17:57Number of market packets proportionate to the size of the flow. So an advantage AFD has overred it's his ability to distinguish which set of flows are.18:16Causing the most congestion. Another feature we use is priority control or BFC. PFC lets list and lossy traffic share the same internet link.18:35PFC allows us to logically divide a single physical connection, e.g. 100 G connection into up to eight independent, independent classes. Keep in mind that in PFC, the classes are called priorities.18:53So when congestion builds PFC can pause traffic on an affected priority while traffic on another on or other priorities continues to flow. So how does PFC work?19:11And how does it generate pause frames? Similar to wred, EFC also uses two thresholds. X on and XR. XRF is the threshold that is going to indicate when a pause frame should be sent.19:31At that point, buffered usage is high, so the source needs to slow down. Above the exhaust threshold, there is a headroom to accommodate any packets that are already in transit on the wire. The headroom.19:50You know, absorbs, absorbs those packets and places them in the buffer while the source is being paused. Once the buffer usage falls below the exhaum threshold, the EFCS stops generating pause frames.20:07Another concept is Nick PFC and storms on how they are solved. In an AI network, if a note like host C in the diagram enters in an error state or.20:28Becomes extremely slow, it may continuously send PFC pass frames to its access switch. In this case needs L three. So L three or L three's buffer.20:46And then fill up causing it to send pause frames to spine S one. The spine in turn sends pause frames to L one and L two, you know, blocking hosts that aren't experiencing any issues. This is known and this is known as.21:03Head of line blocking across the public.21:23This is where PFC watch doc comes in. PFC PFC watch doc is going to start the timer for the affected queue. If the timer expires, all packets in the affected queue are cleared.21:29Breaking the blackage. So in this example dropping the packets, testing for host C, the the problematic note, please that space on leave three. So every then stops sending post frames to the spine.21:45Although our goal is network, it's better to lose packets destined for a single failing node than, than, you know, let that node affect the performance of a cluster with many other nodes.22:06Now, EFC W red, and ECN can work together and that's called, you know, DCQC. The key is how we position thresholds within the non drop queue. So 1st we set the W red threshold in the lower part of the buffer.22:24This is going to allow the switch to act proactively.22:44So, in a 1st moment, as traffic reaches those thresholds, ECM notifications are sent to the endpoints. So this is our early, you know, our early warning mechanism giving the endpoint a chance to grad gradually reduce their transmission.22:49Great.23:09However, if a board soft traffic is so intense that the buffer keeps failing despite the, you know, the ECN notifications PFC is going to come into play to prevent packet drops. So we set the PFC X off threshold higher in the queue.23:13Well above the wred thresholds. So it only takes effect in critical situations. If traffic reaches Exox threshold, the switch sends a PFC pause frame, you know, back to our descender, so managing the congestion in that way.23:32So those are QoS features we have for our last list network. So before we jump into the next section section of the presentation I'm gonna hand it over to Don. So thank you Carlos.23:54We have one additional Slido question for everyone in your current AI cluster environment, which specific visibility gap is the most disruptive to your operation? So when you think about your day to day and you think about a lossess network and AI networking.24:12What is the visibility gap that's most concerning to you in your environment? So feel free to take a look at these options and let us know of these are most concerning to you. We've got a few answers coming in determining if a latency spike is due to the network or the compute node that has been a battle since we've had networks.24:29Hasn't it. So, yeah, so that is definitely something that the Nexus dashboard can help you with. Tracking these storage I/O performance across the fabric.24:49Yep, that, that's always a challenge as well. We'll leave the, the poll open for just a few more moments to see if we can squeeze another few votes in and see if anything else any of these other options are top of mind. Looks like we're battling out between those bottom two of track.24:58Tracking storage, I/O performance across the fabric, and then determining if all the latency spike is due to the network or a compute node.25:18Those are two, two big ones. That's, that's for sure. Okay, well I think we'll go ahead and call it. I think we've we've seen exactly what is on top of most people's minds. So Carlos I'm gonna let you jump back in.25:26Thanks on.25:45Let me, can you please share the, the file again? I'm sorry. Yeah, NO worries, I got, I got it. Here you go. Okay, let me.25:48We already covered them. So, our next section is about traffic load balancing. Real quick Carlos, I think you need to grab control of the.26:16Okay sure control file. Thank you. So why is it important the the traffic load balancing in in an AI network environment? So let's talk about the problem it solved.26:281st, let's say that we start with a non blocking network. So the idea is that the network has enough capacity for servers to communicate that their full connection speeds even when several servers are actively or active simultaneously.26:47And that is gonna depend on the balance between the downlink capacity and the uplink capacity from the leaf switches to the spines. If you look at the topology, leave one has four servers connected to it.27:07Each with 100 gig link and it also has a 400 gig connection to each spine. The same is true for all the leaf switches, so we're we have a non blocking fabric, right?27:21So in a, in, in ideal conditions, this fabric gives us enough bandwidth so that servers don't experience congestions congestion due to a lack of link capacity.27:38So what's the potential problem? Let's look at what happens when ECMP distributes traffic across across uplinks. On the left hand side.27:56This slide you can see two switches representing two different designs. Both provide 200 gigs of total uplink capacity.28:11Imagine 1110 gigs, ten gig flows coming from the downlink ports. So together they need 110 gigs, right? But the question is, where do those flows go? As you probably do remember ECMP assigns each flow.28:26One uplink based on a hash with 2010 gig uplinks, the flows may end up on the same link while other links remain idle. So we say that the probability that all eleven flows choose different uplinks is only about 3 %.28:45Now, with two, 100 gig up links multiple flows can share each link. So both links can handle the traffic unless too many flows end up on the same one. The prob the probability that both links stay with.29:06Within capacity is about, in this case 99 %. So the takeaway here is that total uplink capacity is in the whole story. With ECMP, how traffic is distributed?29:24Across the links matters too. Remember, we are talking about all all of this because this is relevant when you turn the QS configuration for your AI network. Here is the issue.29:40CCMP needs enough variation in its hash inputs to distribute rocket flows effectively, but there is a potential traffic distribution problem. Usually all servers in a backend network are in the same so in the same.29:56A small IP subnet, so their IP addresses shared the same 1st two or three opts. Also uses the same destination port 4791 for all the flows. So this can lead to a traffic polarization across the uplinks, meaning that, you know.30:13Some uplinks become busy while other, others remain under used. So how's the hash calculated? You know, the default ECMP algorithm?30:33Uses the source and destination IP addresses and the layer four protocol along with the layer four source and destination ports ports to calculate a hash. So again with two traffic, only the source port may provide some entropy.30:52So, how can this be enhanced? You can use something called user defined field or UDF. So using this, the Nexus nine K look deeper into the packet and use a.31:12Field beyond the IP and the layer four headers when calculating the hash. Every rocky packet includes an infini event transport header. Inside that packet, it's it's a field called destination.31:32That can provide useful variation between rocky conversations. If we look at the command output at the, at the bottom of the screen, we can see that the load sharing algorithm is using not only the source and destination, but also the UDF.31:51Which is going to use the destination queue per field. The offset there, the offset of 33 tells the switch where that field begins in the packet layout. And the length of 24 bits tells it how much to read.32:10So in this case, three bytes. So the bottom line here is that we can give ETMP another field so it can it can improve the distribution of rocky B two traffic across the available up links.32:30DLB or dynamic load balancing is another way to perform hashing more efficiently. It doesn't do flow hashing but does flow led hashing. Illustrated in the diagram the switch at the bottom has four.32:53Possible paths through the network. Instead of selecting a path based only on a hash and keeping that choice regardless of conditions, DLB can take part congestion into account. It can rebalance traffic at the flow level.33:12Or at the floatled level. A flow let is a burst of packets within a flow and DLB identifies, identifies flows based on the gap between the burst.33:30Packets in a flow. So each flowlet can be transmitted on a less congested path while helping to, you know, limit back three ordinary. Okay, so far it's looked at the features.33:45That help manage traffic across the AI fabric, including, you know, QS and load balancing. Now Nexus dashboard provides a way to onboard and, onboard the fabric and apply those.34:05Settings consistently. As a very brief overview to the product, Nexus dashboard brings you brings you you know provisioning, security, management and analysis.34:23Together in one platform. It supports different type types of fabrics, classical internet, IPFM, sand, and in this case also AA fabrics. So it helps us.34:40Deploy and manage the AI fabric, then analyze it to see what's happening so we can identify and troubleshoot issues faster. Now, deploying an AI backend fabric.34:55It requires you know error prone manual.35:16Tuning or two. With Nexus dashboard onboarding and AA fabric is straightforward. In this 1st in in in in his 1st demo.35:19Today Jeffrey is going to walk you through that process. You will see that ND offers different QS configuration templates that you can use as a starting point. You can also fine tune those settings to match your fabric needs and deploy them on.35:36As part of the creation process. So.35:56It's going to automatically configure PFC, ECN, you know, load balancing, combining different algorithms or specific traffic, and it also handles the underlay routing by automatically allocating IP subnets, loopback.35:59And configuring BEP across the fabric. Nexus dashboard can also help us or help you monitor your AI cluster.36:19In basically four key areas. 1st, end to end visibility gives you a broader view of the network from, you know, switch ASICs and optical links to server links and GPU utilization.36:38Next, topology visualization is going to help you understand how the physical topologies relate to the communication patterns used by your AI training jobs. Nexus dashboard also provides real time metrics including.36:55You know of usage PFC paul's frames and marking rates. These metrics are gonna help you identify potential congestion and understand what's happening across your fabric. Jeffrey is also going to walk you through all of this.37:14Finally, proactive troubleshooting helps you detect abnormal behaviors.37:34And narrow down issues before they affect your, you know, your training jobs. So, that, with that I wrapped up, they showed me how part of this session, so.37:40In the 1st part, we covered the concepts behind lossless networks or AI cluster, the QoS features needed and how Nexus dashboard can help you configure and monitor AI fabrics more easily.37:59In 2nd part, you will see all that in action with Jeffrey. So I'm going to pass the presenter role to Jeffrey. Thanks Carlos. Appreciate it.38:14Alright, let me share out my demo environment here and we'll get right into it. Good morning, everyone, thanks again for joining us today. I hope everything Carlos has showed you so far was insightful. Now we're gonna take a lot of that and just dive right into the lab environment and and take a look at some of these things in action. So I actually have three different demos.38:35Plan for today.38:55We're gonna start with some provisioning with Nexus dashboard. So if you're not familiar with Nexus dashboard, it is basically our centralized unified platform for operating, automating and monitoring network fabrics. One of the many use cases is provisioning of.38:57I ML network fabrics. And the way it does this is we utilize best practice templates that Cisco builds and then we basically use those to apply configuration to our switches. So let's go see this in action.39:17So I'm gonna jump right in here. We'll log into our Nexus dashboard. And 1st thing you see when you log into Nexus dashboard is an overview page, right? It's gonna give you an overview of your onboarded sites, your fabrics, the inventory. At this point, we don't have any so that we don't really see much here. So let's get started with.39:35Actually creating an AI fabric and onboarding some switches.39:55So we're gonna jump over to the managed tab here on the left. We'll go to fabrics, we'll go to the actions menu on the right, create fabric. And here we have a few different options. We do have support for ACI fabrics for telemetry, but again, the use case for today is AI.39:59By fabrics. We have the ability to create a new fabric, and we also have some support for onboarding brownfield environments. So if you already have a fabric stood up and you want to start managing the configuration with Nexus dashboard where you want to use.40:19Telemetry on Nexus dashboard, you can onboard existing deployments. So we're gonna build a brand new AI fabric today, so we'll go ahead and select create new land fabric, go to next. And now we have the ability to select a type of fabric. So.40:39We can use Nexus dashboard to build various types of topologies, right? And the reason we're asking you to select a type is this is basically going to be used to figure out which template we want to use to build the configuration for the switches, right? So if you think of like a VX.40:56Plan fabric spine leaf architecture, we have routed interfaces between our spines and our leaf switches.41:16We use vni's, we use VXLAN. That configuration is very different from let's say a classic land where we have spanning tree, right? Maybe a three tier architecture. So all you're doing here is you're telling Nexus dashboard.41:23This is the type of fabric I want to build and in turn that is going to drive the configuration template that is used to generate the configuration for each individual switch. So we're gonna select the AI fabric. Again, it says automate NXOS for top performance AI networks using meraki v.41:39Two. Now on the right hand side here we have two different subtypes that we can build. We can build in a routed fabric using ebgp or we can build an AI VXLAN EVPN fabric. So we'll go with the AI VXLAN EVPN fabric today.41:59Now just so you can see all the configuration options, I'm gonna toggle this config mode to advanced. I'll give it a fabric name, let's say AI fabric one and give it a location that puts it on a nice map on the overview page. We'll use IBGP as our.42:17Overlay routing protocol and then we'll give a BGP asn, right? So that's gonna be.42:35The BGPAS running inside of our fabric between all of our leafs and our spines. I'll select premier licensing just so we can see all the different features. Hit next. Now under this AI settings section, we have this policy selection mode, which we can.42:40Basically define what our link speed they're going to be. The other thing we can toggle here is dynamic load balancing, which Carlos talked about in the presentation portion. So if you turn this on, you have the ability to to select different modes for DLB based on what you want to achieve.43:00I'm gonna leave this off in this demo just cause these are virtual switches, they don't support DLB, so we'll start seeing errors when we try to do deployments, but that is how you would turn that on here. And we can go back and edit a lot of these properties after the fabric is created if we need to down the road.43:20Now the last thing you'll see here for because I toggled the advanced settings is all of these different parameters that we're looking at here are best practices that Cisco has generated, right? So we have this template for to build this.43:43Fabric and now we're giving you some customizability.44:00So, things like what do I want my subnet mass to be on my point to point links between my spines and leafs 30 or 31? Which underlay routing protocol do I want to use? OSPF or ISIS? So this is all built into that Cisco best practice template.44:03That we ship with Nexus dashboard, but we're giving you some flexibility to modify these if you want to. So some other things you might look at are loopback ranges, right? If I need to configure a loopback on one of my switches and you want Nexus Dashboard to auto generate an IP.44:22For it, what is the range it should pull from? Right, so those are the types of things that you're going to see in.44:42This customizability, right? Other things are gonna be like your layer two VNI range, your layer three VNI range for your VRF deployments. Dynamic VRF VLAN ranges and dynamic network VLAN ranges. So all that is adjustable, but you don't have to touch any of this if you just.44:51Want to go ahead and start building the fabric. The one thing I will adjust here just so our switches don't reload and we don't have to wait, is we will turn on add switches without reload. Alright, so we'll submit that.45:10Nexus dashboard is now building kind of the base template for this fabric, and then what we're gonna see is we'll have the ability to onboard our switches. And we'll talk about how that works as we're onboarding them.45:29Alright, so now the fabric has been created. We'll go to view fabric details. Just like the main overview page of Nexus dashboard, we have a nice pane of glass here that shows you an overview of the particular fabric that we just built. So it gives you the name, BGP ASN, the type of fabric it is, right? We see.45:48Gives you some status breakdowns. Again, there's not much here yet because we don't have any switches. Some analysis here at the bottom, and then on the right hand side we have more inventory type information. So number of switches VPC pairs, audit log.46:10Interface inventory, accounts for like number of endpoints, so learned max, IP addresses, and then like logical configuration entities, so like vrfs, networks, and then AI job status on the bottom right.46:25So let's go ahead and add switches and see how we push configuration. So we'll jump to the inventory tab here. And before we do this, let me just show you an example of a switch right now so you can see the configuration that's on it. So we'll jump to a leaf that we're going to onboard and let's just do a.46:42Quick show run. So you can see there's not really any config on this particular switch.47:01Basic default route in the management VRF, NO config on the interfaces. We've got our management IP there and that's really about it. So let's jump back to Nexus dashboard. We'll go to the actions menu here on the right.47:07Add switches, and we're gonna provide what's called a seed IP. So this seed IP is basically telling next nexus dashboard which switch am I going to reach out to to onboard it, right? So that switch that we just looked at leaf one is reachable via a management IP. So we're gonna.47:22Give it the management IP.47:42As well as some credentials to log into it. Now, not only is Nexus dashboard gonna onboard this particular switch, but while it's onboarding this particular switch, it's going to parse the CDP neighbors of this switch and see what other device.47:44Devices it can reach. That way I don't have to onboard each switch one by one. So it's gonna do its best to figure out.48:04Is as much of the topology as it can in this one discovery flow, and we'll see that here when we click on discover.48:11Oh alright, so here we see a list of the discovery results. So not again not only did we.48:30Log into that particular seed IP, the management IP of leaf one, but once that happened, we looked at CDP neighbors and we basically went link by link to see alright, can I log into any of those other devices using those same credentials? And we can see here it came back with a list of switches, and this is.48:46Basically everything I want to onboard it. So today we're not gonna worry about the external router, we're gonna onboard our three leafs, our two spines.49:06And that's really all we're going to need for today. Now, I'm gonna click add switches and at this point we're.49:15Working on registering each of these particular switches into Nexus dashboard. So basically taking over.49:36Managing the configuration of these devices. Now, one thing to note is at this point we are not pushing any configuration to these devices. We are simply looking at the running configuration and registering them into Nexus dashboard.49:43Yeah once that process is completed, we can go ahead and take a look at actually pushing configuration to those switches.49:59While we wait for this, I'll also note that what's nice about having Nexus dashboard manage the configuration for all these switches is that you always get Cisco best practices for the actual configuration that gets applied. Again, it comes from that template that we generate. The.50:16And when you need to replace a device, let's say a switch needs to be rma'd or you're simply upgrading hardware maybe to a newer model, you can use a replacement workflow in Nexus dashboard, and it will basically take all the configuration from that old device, the old serial number.50:36And move it to a new serial number, right? All you have to do is click a few buttons and your device is replaced. So NO more copying and pasting config from box to box, really simple to do that workflow in Nexus dashboard.50:54It can also all be automated. So, if you want to use like Ansible or Terraform, you can use the Nexus dashboard APIs to accomplish all of that. So everything that we're doing in the UI today can also be done via the API.51:11So this registration should be finishing up here. This part typically takes a little longer, it is a virtual environment, so it's a little slower than if you were to have a physical deployment, so just give it another minute here.51:29Yeah, so we have our three leafs and our two spines on boarded here. We'll go ahead and close this out.51:47You can see our dashboard here is starting to populate with some of the inventory information. We'll jump back to inventory. And the one next thing I wanna talk about here is this role. So by default, we have the role set to leaf. We need to adjust that for our two spine switches. So I'm gonna check these two.52:03And go to actions menu, set role.52:22Spine. And then it's gonna give me a warning saying I need to do a recalculate and deploy, which we'll talk about here in a second. Before we do that, I want to just explain the importance of the role property here. So the role property is basically telling Nexus dashboard.52:27The type of configuration that needs to be generated for that particular device, right? So if you're familiar with VXLAN, right? Spine switches have very different configuration than leaf switches, right? On the leaf switches, e.g., that's where our workloads connect. That's where we have the SVI default gateway for the clients, right?52:45That's where we extend the VLAN down to a particular host. That's where the user VRF is deployed versus on a spine, right? That's typically more of our transit.53:05Portion of the network. We don't have our user VRF deployed there typically, right? It's usually just routed connectivity between the spine and its member leaf switches. So that role is basically how.53:17We determine which type of of configuration template to use when we're generating the config for that device. So what I'll do next is we'll do that recalculate and deploy, which is basically our foot 1st and initial.53:31Full configuration push to these devices from Nexus dashboard. So we'll go to this actions menu here in the top right. We'll go to recalculate and deploy, and now Nexus dashboard is going to.53:48Look at the running configuration of each of these devices and it's going to compare the running configuration to the intent. And the intent is basically what Nexus dashboard thinks the configuration should be based on what you have configured.54:05As your intent, right? So we've built an AI fabric, which is built using a template. We have some customizability, we might create a VRF e.g.. So it's basically just going to say, alright, here's the configuration the user has inputted on Nexus dashboard. Does.54:24Does that match what is in the running config on the switch? And if not, how do we get there?54:43And again, I'll stress, even at this point, we still have not pushed any config or made any config changes on the switches. We will always explicitly call that out when we are going to do a deployment to the switch, right? And we have the ability to either accept it.54:49Or cancel it. And we'll also see when it's ready to do the configuration push, we have the ability to look at what configuration commands are going to be entered on each particular device. So I'll show you that here in a second when this finishes.55:07Yeah While we're waiting for this, the other thing I'll mention is if, if a user were to make configuration changes directly on a switch on the CLI.55:43That will put the configuration in an out of sync state on Nexus dashboard. So there'll be a few different ways to reconcile that configuration, but just keep that in mind. Typically.56:03Once we start managing the devices with Nexus dashboard, we want Nexus Dashboard to be our single source of truth. Alright, so here is the result of that initial recalculate and deploy. So we can see all of our switches are in an out of sync state, meaning that the roaming configuration.56:13Does not match the configured intent in Nexus dashboard and that's NO shock, right? We have a brand new fabric here. There's NO configuration yet on these switches. And again when we again at this point we haven't pushed anything, but we can click on this 633 lines and we can see exactly.56:32What changes Nexus Dashboard is going to make on the switches? So here's all the configuration commands.56:52That Nexus dashboard is going to run. So turning on a bunch of features, enabling VXLAN, creating multicast configuration, BGP configuration for our connectivity.57:00In the underlay and overlay, you can see that we are configuring different policy maps or QoS. Again, that's the really nice thing when Carlos was talking and during the presentation, a lot of that can be very challenging to configure, but all that is built in by default.57:15In this Cisco AI template in Nexus dashboard. So you don't have to configure any of that. It's done automatically for you.57:35So you can always take a look at everything here that it's going to push and if you're satisfied with that, you can go ahead and click the deploy button here at the bottom right. And at this point, this is when we are actually going to start changing the config on each of these switches.57:45So we'll let this run and then we'll go take a quick look at the running config of that same leaf where we saw it was pretty much default configuration.58:03Okay, usually a little faster with a physical deployment, it's a virtual lab with virtual nine case, so definitely not as speedy as an actual deployment with physical gear.58:23Alright, so it looks like our two spines are complete, just waiting on the leafs.58:41And again, those templates that we push to the switches, that's gonna include, you know, in terms of rocky V two and the lossless network we are building, it's gonna include PFC and ECN. All those best practices are built in automatically with this fabric.59:10Alright, so this is done, we'll close this out and let's just do a quick show run again on this switch. And you can see all that configuration we were just looking at is now pushed to this leaf one. Right, so all of that. So there's loopbacks, we've got our BGP process and everything, the spine upload.59:25Links are automatically configured and I'll call that out quickly here for the spine uplinks in particular, if we look at like eth one one, I'll just take a look at this as an example. So here's our leaf to spine uplink.59:45Nexus dashboard automatically added a description to it based on where it's connecting. We turned on PFC, we have QoS enabled on this interface. The point to point IP was automatically pulled from the pool and established on this interface. So.01:00:03We never told Nexus dashboard, here's where all the devices connect to each other physically, right? It figured that out automatically based on CDP and applied this configuration on the right uplink interfaces. So that's really nice. That's all dynamic.01:00:19All right, now that's our initial config push. So let's do a quick deployment of like a VRF and a network to our switch. So let's go to the segmentation and security tab. Go to vrfs.01:00:36We'll just create a brand new VRF here, we'll call it AI prod. We don't have to put anything else here, so it picked this VRF ID from that pool from the fabric template we looked at when we 1st started building the fabric.01:00:51And then we're gonna jump to the networks tab, and we'll build an actual network for our hosts. So we'll call this, I don't know, like AI cluster one, we'll put it in the AI prod VRF.01:01:07And we'll give a default gateway ten 1010 one/twenty four. Just keep things quick and easy here. Alright, so now we have our network. Let's, let's put it on some interfaces on our leaf switches. Let's say we wanted to add this particular network, this VLAN essentially to Eth one five on.01:01:22All of our leaf switches. So let's go ahead and check it. We'll go to actions multi attach, we'll select all of our switches, and then I'll go ahead and hit select interfaces and we'll check ETH one five.01:01:42And I'll just go ahead and copy this. That way we don't have to click, and then I'll click next and save. Now just like when we did that initial recalculate and deploy.01:02:01It's gonna provide you the same information. So it's saying based on that change you just made, that is causing an out of sync condition because we haven't pushed the config into the switch yet. Again, we can click on this pending config here, see exactly what it's gonna push. So configuring the SVI, making some BGP adjustments, VN side.01:02:19Segment edition with the VLANs.01:02:38So we'll go ahead and deploy it. Alright, now let's take a quick look at our switch, so there's a show run in Eth one five.01:02:42So there is our VLAN deployed. Here's our AI prod VRF, there's our SVI VLAN 2300, which you can see is the access VLAN on this ETH one five, so all that was automatically applied. Now.01:03:01What about, QoS configuration? We see on this interface it's not there.01:03:20So we're building and trying to build a lossless fabric here. Let's add that same QoS configuration that's on the spine uplinks to this particular interface. So to do that, I can do that all very easily here in the UI. Let's go to the connectivity tab, which is where we're gonna find all of our interfaces.01:03:25I'll do a quick filter interface contains one/5I want to apply the same config on each eth one five across our three leafs. So I'll select those, go to the actions menu, edit config.01:03:45I'll scroll all the way to the bottom, which is where we'll find that configuration. So we'll enable priority flow control and then we'll also enable QoS. So I can create a custom policy if I want, but if you don't enter anything, it's going to use the default best practice policy called QoS classification. And that's the one we have built into the.01:04:03Input by default.01:04:22And then I'll do that same thing on the other two interfaces here and then once that is complete, we'll go ahead.01:04:24Add and deploy it to our eth one five interface across the three leaf switches. So we'll save and deploy. Again, we have the ability to check.01:04:44The configuration commands that it's going to push. I'll just say it's acceptable, We'll deploy.01:05:02And once that is complete, we should see that change on our ETH one five. There is the configuration. We have our QoS classification policy and we have priority flow control. So again.01:05:27That is using Nexus Dashboard to provision an AI fabric, in this case VXLAN EVPN AI fabric. Again, all the best practices included in those templates, so really easy to jump right in and build this lossless network.01:05:44Alright, let's move right into demo number two. Just because of a difference in the demo environments you'll see, still be Nexus dashboard, but the fabric is gonna be slightly different because it is a different environment. And what we're gonna focus on here is the telemetry aspect of Nexus Dashboard and the monitoring.01:06:01A portion. So let's take a look at this London AI fabric.01:06:20And we can see here we have already built fabric. We see the same type of information on the overview page here. How can I use Nexus Dashboard to do monitoring of my fabric? Well.01:06:26Another use case of Nexus Dashboard is what we call telemetry. So Nexus Dashboard can gather data from the switches. The switches also stream telemetry data in real time to the Nexus dashboard. So that's done via an established.01:06:41Session between Nexus dashboard and the switches. So I can use this to look at interface statistics. I can look for flow telemetry, what conversations are occurring in my fabric between my hosts. Alright, so if I wanted to see, e.g., some of my most utilized interfaces, I'm gonna.01:06:57I click on this new interface statistics here.01:07:17We can select how do we wanna look at top by receive transmit and so on. We'll go with receive. So e.g., eth one two is most utilized within the past 2 h in terms of receive utilization. So it looks like we're almost actually.01:07:20Maxed out here on this interface at particular points in time. And I can always adjust this time selector here to look at, data that is sprouted across a larger period of time, so like the last 6 h or even the last 24 h if I want to. So if I'm trying to understand like trends within.01:07:39In my environment, you can kind of get an understanding of what's going on in the fabric at that point in time.01:07:58Now Nexus dashboard, as it's collecting all of this data, is storing it in the database and it's using it to do correlation between different events. It's also storing it so you can go back and look at it, right? So if I want to understand, alright, I saw this particular event happen today, I.01:08:07I can change this time selector, go back in time typically a few weeks at least and see alright, is this something that's occurring every day or is this a new behavior that I'm seeing?01:08:26Now, we can take all of this data, all this raw data and look at it. That's really nice, but Nexus dashboard is constantly monitoring and analyzing all of these different collections that it's doing. So if it is.01:08:39Collecting a parameter about an interface, e.g., let's say they receive utilization. If it starts to increase and get close to maxing out, Nexus dashboard is gonna alert you about that via what we call an anomaly. So an anomaly is basically how we alert you that there's a problem, right? We don't expect you.01:08:55To come in here and look at all this raw data manually every day. So it's gonna keep an eye on all these statistics it's collecting and when something.01:09:14Is trending in a negative direction. It's going to raise what we call an anomaly and say hey hey you know we see this particular problem in your fabric or something that might become a problem. The other thing I want to show here is what we call traffic analytics and traffic analytics is.01:09:24Basically, looking at conversations between your endpoints, between your hosts. So the way this works is the switches are going to stream analytical data about the conversations, whether it be TCP or UDP.01:09:44That are happening between your clients and we're going to score them using three different main metrics here. So you can see latency, congestion, and drops. So for each particular flow.01:10:04Or conversation, we're gonna say, what is the latency of it? Is there any congestion detected? How do we detect congestion? Well, we can use a few different methods. One might be looking at flags in the TCP header if there is a congestion ECN bit set. And then finally drop.01:10:19Right? How do we detect that there's drops? Well, the leaf switches and the spine switches can report, hey, Nexus dashboard, I had to drop this particular packet because my buffer is full or I had to drop this packet because I don't have a route to the destination.01:10:39So Nexus dashboard, as it is receiving this flow data from the switches, again, it's looking at all of that, making basically like a baseline and saying, alright, this is what this particular flow typically looks like. It has this much latency.01:10:57You know, it's probably not usually congested and it doesn't have any drops. And when any of these start to differ in a negative direction, whether it be increased latency congestion or an increase in packet drops, that is when Nexus Dashboard will raise that anomaly and say, hey.01:11:15There's a problem with this flow. I can also look at the, the raw data, right? Again, we don't expect you to do that every day, but here it is, right? So I can see all the different conversations that are occurring within my fabric. These happen to be TCP flow.01:11:35So here's a particular server that is listening on this port, and if I click on it, I can get a breakdown of all the clients that are talking to this particular service. In addition to that I can see, you know, if it's healthy in this one, in this case it's not cause we're having a buffer drop for this particular flow.01:11:56And it gives you, you know, input, output information where the devices are connected. If there's like TCP resets that we're seeing, it will alert you of that as well.01:12:16And those are all the different things that will generate those system anomalies. And that's the last thing we'll look at here quickly, is these anomalies. So this is how we say, hey, there's a problem that we're, you know, either seeing live in the environment or something that may.01:12:28May become a problem. So e.g., here's that anomaly generated cause there's a buffer drop. So I can click on this to get more information about it. It will show me there's a buffer drop on leaf one. It was detected at this time. It was last seen at this particular time.01:12:47And then it's going to try to give you some sort of impact, right? What flows are impacted and what endpoints are impacted. So it looks like for this particular anomaly, for buffer drop, it's this client to server flow here.01:13:05Happens to be a UDP conversation. So this is where next nexus dashboard can be really handy in terms of monitoring the fabric. Doesn't have to specifically be AI, but in this particular instance, we're talking about AI fabrics so we're using that.01:13:20As the example today, but next is dashboard telemetry, and any time that there is a detected issue, we will raise an anomaly and that can, you know, those anomalies anomalies that you see here can be sent externally, so if you have like an external monitoring system, you can have these.01:13:40Export to that system and you can generate tickets or alert, you know, the appropriate administrators or users to come and take a look at it. Now one additional thing before we move on from telemetry onto demo three.01:14:00There is a feature in the latest version of Nexus dashboards, four.three. You can actually get extended information about AI jobs within Nexus dashboard. So I'll show you just quickly what that looks like in this video demo. So there's an AI jobs tab in Nexus dashboard, and we.01:14:18Can collect statistics from the server NICs, and we can basically correlate.01:14:38The information from the actual compute side and the switch side, the network side into one nice pane of glass. So you can see here they're demonstrating two AI jobs that are running, and if we go look at one of those specifically.01:14:45You can get a summary of the job, some GPU statistics. If there's any anomalies that are detected for this particular training job, it will show you that. You'll also see.01:15:03Some analytical data, so things like any packet drops, any CRC errors, you'll see RX and TX power for the modules. And then, you know, if there's any sort of like congestion occurring, so things like pause frames, congestion notification.01:15:19Packets, right? It's keeping an eye on all of that, but it's.01:15:39Nice thing to have because it actually, not only do you get data from the network, but you can also get the data from the compute side and nexus dashboard can correlate different events that it's seeing to help you troubleshoot if there are errors.01:15:43Alright, so let's jump to our last demo, and for this particular demo, again, it's a different environment, but we're gonna showcase what we have called Cisco Cloud control. And this is our new unified AI native operations platform.01:16:01That basically brings our entire portfolio together. So.01:16:19We're gonna try to use this to get some details about our AI fabric and and then also ask some capacity queries. Now, we'll see how this works out with AI, we know AI can be a little bit unpredictable, so if things start to go sideways, we'll fall back to a video that I recorded earlier.01:16:24Let's give it a shot and see what we can get.01:16:44So I have a particular AI fabric, so let's just get a quick summary of it. So I'll say give me a summary of the RTP AI one fabric on Nexus dashboard.01:16:46Alright, so it rendered some information here. I asked, again, I asked for a summary of this particular fabric. So this is a an AI fabric that lives on a Nexus dashboard cluster. So I've got the fabric called RTP AI one. It's fabric type AI ML VXLAN IP.01:17:38BGP, so that's the same type of fabric we built earlier, give some status here. Connectivity to the switches good. Looks like we have some anomalies and advisories. Everything's in sync, which is good. That means the configuration on the switches matches what.01:17:55The intent is a nexus dashboard. It gave me a nice breakdown of all of my switches, so it looks like we have two border leafs, two leafs, two border spines, the uptime, the model, and then it also broke down or broke out the anomaly and advisory severity per device, which is kind of nice. Looks like I have some issues I need to go.01:18:12Look at, gave me interface utilization or sorry, interface inventory. Looks like we have 326 total interfaces, number of endpoints, and now let's see if this can work yep here's a nice little table that if I wanted to maybe export it.01:18:31To send it out to my team, I could do that. So nice summary there. Let's see what else we can use. Let's say let's say I'm wondering if any of these switches in this particular fabric are coming up on end of life. But let's say I'm also interested in software. Like, are they running.01:18:50Owning the latest software version or the recommended software version.01:19:09So let's say there are any switches near hardware or software and of like, you know, with a smaller fabric, you could always, you can kind of look at this information manually, but as you start.01:19:13The scale, you might have different models, really convenient to be able to just have these manual query for this type of information. And nexus dashboard tracks that, right? This where this information is coming from. So cloud control has hooks into Nexus dashboard and Nexus Dashboard is the one that.01:19:33Has all this information about the devices.01:19:52So it looks like it's running queries to check software switch recommendations and advisory details, sounds like it's actually giving us more than what we asked for, which isn't terrible. So it says they're not on the recommended version, so they're all running 1062. Current recommended is 1055.01:20:17And if we care about the latest, it's 1064. So.01:20:38Looks like there's some advisory data, none flagged for end of life or end of support, which is good. So NO hardware or software end of life or end of sale detected. Running version is not the current recommended.01:20:45So we could potentially make some adjustments based on that, right? If we wanna, either upgrade to the latest ten six or we could go back to the current recommended 1055. The bottom line note hardware, software end of life concerns are flagged.01:21:02But running a version that is not on the recommended track. So we could continue querying it about this if we wanted to understand maybe like how I would potentially do this software upgrade or downgrade, get more information about some of these P certs that are in here.01:21:17And so on. Let's try a capacity question. Let's go with, give me a fall per interface RXTX table, sorted by utilization.01:21:33For all interfaces over the past, let's say 6 h.01:21:53That looks like that stopped let's try it again.01:22:16So while we wait for this query to complete, what's nice about cloud control here is it has hooks into so many of our different solutions and different portfolios, right? So we can query it about campus network for catalyst center, we can query nexus dashboard. We can talk to Meraki.01:22:57Splunk Intersight, right? So we have so many different.01:23:17Connections into our different portfolios, it brings it into one pane of glass and you can actually generate what's called a canvas and you can use canvases to troubleshoot problems with your peers. So you can basically share data from these queries onto a shared canvas and use that.01:23:23Have to kind of work through problems together with your team.01:23:43Yeah, this is the one that sometimes doesn't query the right way. Just so you can see it, what I'll do is let me show you this particular one in.01:23:49My pre recorded video. Let me just switch my screens here for a second. So here's this recording from earlier.01:24:08So I asked it looking at all spine leaf interfaces, give me a breakdown of their utilization or any near maximum and it gave me this nice table with a breakdown of top interfaces by utilization. So it looks like leaf three.01:24:22Is 55 % utilized. So if I was looking for, you know, do I have any capacity concerns with like my uplinks? Do I need to look at maybe they're like a hundred gig and I need to consider.01:24:39You know, upgrading them to 400 gig. This is a nice query for that, right? So it doesn't look like I would like I have any sort of issues there so there are those key takeaways at the bottom, NO interfaces are near max, the highest is 55.5 % on leaf one and leaf three, which is symmetrical AI workload.01:24:53Traffic passing through the downlinks. So you know AI is still kind of finicky sometimes, so, usually you can work through the queries and and get the data that you want. Sometimes it just takes a little bit of time. Alright, and then one last question we'll ask it.01:25:13Before we wrap up here is, let's see, do I have let's say I want to add like six more leaf switches do to my fabric. Do I have enough free spine ports to add six new leaf switches?01:25:31So, theoretically what it should do, it should go out to the fabric.01:26:03Check the switch model, understand which ports are in use and which are not. It should also understand that we typically connect a leaf to each of our spine switches or at least two of them and and kind of understand that to use the calculation on if I have enough interfaces to add these six new switches.01:26:07So, it's telling me my fabric has two spine switches for this particular model. So.01:26:3264 port, 400 gig spine switch.01:26:52It's give me a count of the interfaces that are up and down and it's also telling me I need to ask another query if I want to fully confirm this.01:26:55Again, you can work through this and and it should give you an answer. I can show again from earlier on my recording, so I said do I have enough free spine ports to add six new leaf switches?01:27:12Spine port capacity, basically went through the same thing, understood what model it is, how many ports there are, and it says yes, you almost certainly have enough more than, more than enough free spine ports to add six new leaf switches. So it says in a standard AI design, each new leaf switch takes one uplink to each spine.01:27:34So six new leaves require six ports on spine one, six ports on spine two, and basically work to that to understand yes, that we have enough free ports.01:27:54So that is utilizing cloud control. Again, it's our new canvas that has hooks into all of our different solutions. So encourage you to give that a try at some point.01:28:04And with that, that concludes our last demo, so I hope everyone found that useful and I'll turn it back over to Donn. Thank you Jeffrey, and thank you Carlos both for those insightful presentations and demos, learned a ton.01:28:19So that brings us everyone to completion of this EIS. There have been a couple of questions that have come in throughout the presentation. I believe those have been answered. We also do have a survey that is now open in Slido. We would really appreciate if you take a couple minutes and just answer if you.01:28:37Those questions, let us know how we did, let us know if there's anything you'd like to see in a future session that would be really helpful to us.01:28:56If there are NO final questions that would come through, which I don't see any, and hang on actually I do see. Okay, I see this is recorded, we will have access to the slides.01:29:05I don't think they put the slides out, but the recording will be posted on the Cisco community page, apologize for that one live. If there are NO other questions, we're gonna go ahead and.01:29:20Wrap up, we appreciate everyone sticking around and your participation and your questions and your feedback. Everyone have a great day. This has again been Cisco's expert insight series on AI networking with Nexus dashboard. Have a great day, everyone.01:29:37