ACI Distributed Networking fundamentals

Oh multi pod was introduced in 2016, just two years after ACI was introduced. Yeah.00:00With multi pod, you can have multiple leaf spine topologies that you see on the screen. All of these interconnected through a layer three transport layer, which is what we would call an IPN, right? The interpod network. So that's basically a layer three network to connect your different leaf and spine topology.00:20The overlay is your VXLAN.00:40And this architecture is usually used when you want to have an active active data center, which is managed by a single epic cluster. So it's a single fabric, but in a multi pod deployment for an active active use case. But let's say you want disaster recovery for your fabric.00:43Which means you want a complete backup of your primary data center. Or you want your standby DC to have all replicas of your primary DC. Then in that case, we would go with what you see in the bottom left of the screen, which is your multi site architect.01:03So in this multi site architecture.01:24Each of your site is a fabric. It has its own leaf, spine and epic cluster as well. So each of the fabric is managed by its own epic cluster, and you also have a component called as Nexus dashboard. So this Nexus dashboard helps us to.01:27Establish control plane and data plane connectivity across the sites and also Nexus dashboard is where we define the tenants, the bridge domains, vrfs et cetera, that we want to stretch across the sites. So for this East west communication, Nexus dashboard takes care of program.01:47Giving the policies.02:07Now, what if you have a use case where you want to extend this VXLAN overlay towards a remote data center? But you don't have the justification to have an entire leaf spine architecture. Then in that case, we have this concept called as the remote leaf, which you see.02:09Be as part of the multi part, right? So we have a remote leaf architecture since 2017.02:29This is mainly seen in service provider environments where you may have a remote data center location with only one or two racks of equipment. So in that case, you can have the remote leaf architecture. This architecture is very similar to your multi pod architecture, but the only difference.02:35This you don't need a spine. So this remote leaf will directly connect to your IPN devices. You don't need a spine for this and this is what is the remote leaf architecture. Now what is one thing that is common with whatever we have discussed so far, right? In the left of the screen?02:55Is that all of these are ACI fabrics. But what if you have an environment which is NXOS based.VXLAN EVPN? Then how would you connect it to an ACI fabric? Now, one thing to consider is ACI works on a vxlan, which is called as I vxlan. It's a little different from the standards based V.03:13BGP, VXLAN BGP VPN VXLAN.03:32So we have introduced this concept since 2024, which is called as border gateways. Let me see if I can just annotate on the screen. Just give me one sec, ok. So I think you will be able to see here. So now this is the.03:36Border gateways that you see on the screen. So what this border gateway does is, so between your IVXLAN, which is the VXLAN that's on your ACI fabric and your BGP EVPN standards based VXLAN that is there on your Nexus NXOS environment, using these border gateways, you will be able to communicate.03:57With the standards based VXLAN. So this border gateway that you see on the leaf will be able to convert one to another so that the communication can happen between your NXOS VXLAN and your ACI VXLAN. So essentially meaning your NXOS fabrics can also talk to your ACI fabrics. Now the same thing extended between.04:17ACI to ACI, you can also connect to ACI fabrics and any time we say fabric, it's basically leaf spine and epic cluster. So if you want them to talk to each other, you can also use a border gateway connectivity for two ACI fabrics to talk to each other. Now with that being said, let us zoom into the.04:36Multi pod architecture, and a little later in the session, Ronak will zoom in through the multi site architecture. Let us move to the next slide.04:56Okay, we have another question for you. Now that you know what the different architectures or data center architectures are that are available, which ones do you currently use in your environment? I see only two of you on the call, but ok, three now.05:15Please do, share your thoughts on what data center architecture are you currently using in your environment. Okay, a lot of single pod ACL fabrics. Next followed by multi pod multi site, I see around 24 on the Slido, there are nearly high.05:32Hundred and ten attendees on call. So please join the Slido. Donnak has already shared the Slido code and the password on the chat as well, but you also see the QR on the screen, right? So please.05:52Use that. Okay. 70 % 60 % of single port. Okay, we'll give it another 10 s, but I think it's all, you know, going towards single pod, multi pod, then multi site and very few of you work, you know, have a remote leaf. Okay, let's go on to the next slide. I think around 40.06:04Of you have given us your whole. So let's go a little deeper on the multi pod architecture now. And later we'll also show you some demos and some features that you can use to manage your multi pod environment. So we'll also do a demo. So now coming to the multi pod fundamentals, let's move.06:25Move on to the next screen.06:45Okay, so now as we were talking earlier, right? Multi pod is basically your active active data center deployment and it gives you some amount of fault domain isolation across the boards. And what do we mean by this fault domain isolation? Let me again see if I get.06:50And get the.07:10You know, an annotating tool here, ok, I do. Okay, so we have some level of fault domain isolation between the ports. As we were discussing, ACI is a VXLAN fabric, right? It has both overlay and underlay. Overlay is your VXLAN, and underlay is where you have a protocol called as ISIS. Now.07:12So in addition to that, of course you have the control plane and data plane.07:31For control plane, which is for you to advertise your endpoint reachability information, right? It can be IP or Mac endpoint. So for that, we use a protocol called as hoop in ACI. Now these protocols are only contained within a pod.07:35ISIS scope. So this scope is within the port, not extended to the other port. So that gives you some amount of fault domain isolation. But what if you want to extend this endpoint reachability information and maybe some external prefixes that you have learned, right? What if you want to extend it to your other.07:52Support.08:12Then in that case, we use MP BGP for this. So BGP EVPN is used to, you know, extend this reachability or control plane information to the other port. Now there is also data plane connectivity, of course, which means you have end to end VXLAN.08:13Connectivity, between one tunnel endpoint, right? So between one leaf, let's say to another leaf, so you will also have a tunnel using which your data plane traffic would be sent through. And this entire environment is managed by a single epic cluster. So which means it is a single.08:33Fabric. It's a single fabric single epic cluster, but in a multi pod architecture.08:53Now with that, let us look at what are the supported topologies for your multi pod deployment. So the 1st thing that we spoke about is this IPN network, right? So your IPN network can be any layer three network.08:592nd, but the only requirement is the spines on one side should be able to reach the spines on the other side. So through the IPN I should be able to find a spine I should be able to reach all the other spines through the IPN, but IPN can be any layer three connectivity. Now usually what we have.09:15We have two IPN devices on one port and the other on the other data center we have two IPN devices on the other side and then we do a criss cross connectivity for redundancy. So that is how we usually have this IPN connected. But.09:35There is also an other option. Let me just erase the annotations for the time being. Now, let, there is also another option, but this option comes with some constraints. Let's say you have two pods, only up to two pods, NO more than two parts. You have two pods in your multi pod environment and you.09:53Are not using the features which need this IPN connectivity. Just give me one sec.10:13Just make sure this light detects me, apologies for that. So as I was saying, so let's say you have two pods, NO more than two pods, and you are not using any feature that requires an IPN. Then in that case, you can also.10:26To connect the spines back to back. So this spine can be connected to this spine, it can also be connected to the other spine and then similarly from here. Now, again, like I said, this is only feasible if you have NO more than two ports and you do not have features that use.10:43That need this IPN connectivity. Now, what are those features that really need the IPN connectivity, right? To name a few. If you have a multi site environment, you will need an IPN or called an ISN, right? Or if you have a multi pod environment, which is a part of a multi site architecture, that is also feasible. Then in that.11:03That case also, you will need an IPN. Or if you have a remote leaf, then also you will need an IPN.11:23Or if your epic cluster, if your epic, is not connected directly to the leaf, but you're learning it via an L three out, then in that case also, you will necessarily need an IPN. These are just some features there are others as well.11:30But if you do not need these features and you have NO more than two parts, then you can connect your spines as well back to back. Now, can I have a back to back connectivity plus an IPN connectivity? That is only like allowed if you are in the face of migration, which means you.11:45Are either migrating from an IPN based connectivity to a back to back connectivity or vice versa, right? Back to back to IPN based connectivity. Now only in those conditions, can you have both otherwise?12:05You can only have one of it, either IPN based connectivity or back to back. Now even coming to back to back, usually you would have a full mesh connectivity, which means this spine will be connected to both the spines on the right hand side. Of course, within a pod you would not put back to back connectivity between the spine, so whatever back to back I'm referring to is.12:18From one port to the other port.12:38So you could have this sort of full mesh connectivity, but there is also another option where you can have a partial mesh. So which means this, let's say spine one is connected to, e.g., spine two in port two and spine two and part one is connected to let's say spine one and part two. So what this means is.12:41In your data center, you have only two links going from one DC to another. This is one link, this is another link. So this is supported, but not recommended because this might lead to higher convergence times when you bring up a pod, when you bring up a device and so on. So this is not recommended, but it is supported.13:01So with that, let us move to the next slide. Let me just clear my annotation in the meantime yeah we can move to the next slide.13:21Okay. So the next important topic that we'll be discussing is the epic cluster deployment considerations. So of course you know right cluster is a group of epic controllers. Now, even before we go to the next slide, let's understand some fundamentals. In ACI.13:48The whole fabric database that you have, right? Let's divide it into small small data chunks. So let's divide the whole ACI fabric into small small chunks. Each chunk will have three replicas. Those are called as shards in the concept of, ACI, you know, distributed processing or any distributed processing.14:06Thing right? So you'll have three copies of every bit of your ACI fabric database and.14:26Each of these three copies will reside on a different epic controller for redundancy. Now, how does this distributed processing work? Of course, why are we doing this is one for redundancy, we are dividing into three, but why are we.14:32Dividing into chunks and doing this because in distributed processing there is this data quorum, which means if a majority of the shard is available, then you can write to that object. But if a majority is not available, then you can only read that object. E.g., of course a lot of.14:48You may already know this, but just brushing the concept a little bit for those who may not because there's another concept along the way. So, let's say you have.15:08Three epic controllers, right? A standard deployment. You have three shards of a specific, you know, part of your database on each of the epic controllers. So if one epic goes down, you still have two copies of the shard, which means you can do both read and write. But if one more epic goes down, you will have only one copy. One out of three is a minority.15:18Which means you will not be able to write into the fabric, but you can do read operations. If you want to do a write to the fabric, you will have to bring up at least one of the two epic controllers that have gone down. But the point that you have to realize is just by increasing the number of epic plus epic controllers, will I get more redundancy, which means.15:38If I have five epic controllers there, does that mean that you will give me five copies of the database to put into five epic controllers? The answer is NO. We will still have only three copies. That is a standard. So by increasing the number of epic controllers, we are not increasing the redundancy. So.15:58Up to 200 leaf switches if you have, just having four epic controller should be sufficient for most of the environment. But there are use cases. Of course, the number of epic controllers also depend on the number of leaf switches that are there in your fabric, right? So the more scale, more epic controllers, but there are some other special use cases as well which we will see now.16:17Let's move on to the next slide.16:37We can load up the whole slide. Okay, let me go back to my annotation here. So now let's look at the scenario on the left side. So you have two parts pod one and part two, and 1st let's think of the scena.16:41Scenario where there are only three epic controllers, ok? We'll build on top of it, but 1st let us assume there are only three epic controllers. Now there are two scenarios. 1st is what happens if your IPN connectivity goes down? In which case from these two epic controllers on the left hand side, we will be able to do both read and write because.17:00There are only three epic controllers in total. Two of them are available, so we can do a read write. But from the epic on the right hand side, this only has one of the shots, so even if you take the color blue, e.g., the light blue, you have one two on the right hand side you only have one of the shard, so with this, you'll only be able to do a read operation into the.17:20Fabric. So how do you fix this? You'll of course have to bring up the IP and troubleshoot the IP and bring up the IP. Synchronization will happen between the two parts and then even this part, this epic three right will be able to make both read and write operations into the fabric.17:41Now, what if there's another scenario where instead of the IPN going down, this whole part has gone down, which means your pod is down, epic one is down, Epic two is also down. Then in this case, this epic three, it only has one copy of the shard, so for it to be able to do right off.17:57We bring in the concept of a standby epic. So this is what you called a standby epic. So it will be in cold standby, which means it will not have any of the configuration, but you can bring it up and make it replace either epic one or epic two. So once it replaces your epic, then here you will see that there are two epics.18:17Controllers majority, so I'll be able to do both read and write. So this is how a standby controller comes into the picture.18:37Now imagine what happens if you have five epic controllers, so now let's look at the screen on the right side. So now you have five epic controllers. Let's look at the scenario where the IPN goes down. So if the IPN goes down, then on one side you have three shards on the other three epic controllers on the other side you.18:45Two epic controllers. Now, it is very important to realize that we only have three copies of a shard. So if you see the light blue one, you have one here, two here, and the 3rd one is over here. So which means for this object, your pod two will only have read operations, but pod one will be able to have right operations.19:05Questions, but for another object, say, you know, blue, e.g., here dark blue, you have two of that over here, one of it over here. So again, in this scenario, you will have inconsistent read write. From some controllers, you can write some objects from some another controller, you will be able to only read that object and vice versa. To fix this.19:25Again fix the IPN 1st and restore the connectivity your cluster will synchronize and all is well.19:45But there is the last scenario which is the most problematic scenario we will go through that now. So now this scenario is exactly what is seen on the screen. What if your whole pod goes down? Now pod one goes down, which means you don't have access to any of this. Now, look at the green shot. Your green.19:50Chart because again there are only three copies, right? Let's remember that again. So we have only three copies. So you have the three copies all in the same pod. So which means you have lost your data, right? You don't have a replica of that in your other pod. Now in this case, we will have to get the tag or BU involved so that they can do a recovery.20:11Recovery is possible, they will, you know, help us do that, but there is another way to avoid this, which is what we'll discuss in the next slide. But before that I think it's important to understand usually rate I've seen a lot of customers have just maybe four epic controllers, but distributed in such a way.20:31That at least one, you know, one of the pod has a majority and is able to do the read write operations.20:50Now, how do you prevent this? There is this concept called as warm standby epic. Let's move on to the next slide. Clear my annotations in the meantime. Yeah, we can move on to the next slide.20:58So from 612, we released this feature called as warm standby epic. So how is this different from the cold standby? So cold standby is where I told you epic does not have any configuration. It can be used to replace one of the faulty epic. But warm standby is where this standby epic always.21:18Already has all the configuration and it is not playing an active role, but it still has all the configuration. So if you end up in a scenario like where I was just mentioning, right, you have five epic controllers, three of them have gone down, you're losing a shot, then you're losing some data. Then in that case, this warm standby epic, it already has all.21:38All your replicas, all your shards. So what you do is you build a cluster making this warm standby epic as your epic one. So that is your epic. Your cluster size is just one. You bring it up. So you have all the data intact. Then you extend this cluster with size one, you extend it to two and three by bringing in the.21:58Other two epic controllers, so in the previous window, if you remember, we had two epic controllers on the right hand side which were still operational, right? Two epics on the right. Part two were still operational, three epic controllers on part one were down.22:18So here you promote, you bring up something called as a warm standby epic. You make this as the only epic in your epic cluster and let it come up. It has all the data already. And then later you can wipe the configuration of the other two functioning epics and then add it to the cluster and extend the cluster. So this is a feature that was introdu.22:32Produced in 612 and you can have up to three warm standby epics. You can distribute it across the pods to make sure that none of your pods end up in this scenario. We can move to the next slide.22:52Yeah, this is how you configure your warm standby settings. Let me just clear my annotations and of course from six.two.one all your standby epics are by default now warm standby epic. Let's move to the next slide.23:08Okay, now we'll talk about the IPN requirements. One more slide please Romit, sorry. Yeah, now let's talk about your IPN requirements. So 1st thing is, of course, this is the most famous, right? Between your spine and your IPN, you have sub interfaces with the VLAN tag of four. Now, why is this needed when.23:28The spine boots up for the 1st time, it does not have any configuration, right? Or mainly let's say if this was your primary port and this is your part when your seed port, and then this spine boots up fine, boots up 1st time, it doesn't have any configuration, so it will send a DHCP discovered packet on VLAN four. Now that should be passed to the epic.23:48Via DHCP relay which you configure on the IPN?24:08If you have either an SVI or a VLAN based interface, then there would be issues with passing this DHCP discover through your DHCP relay because SVI or VLAN, your spines, your the spines that are just booting up, right? They'll be sharing the same MAC address. So you would have an issue in terms of.24:12Yeah, in terms of the packet moving to the other side when the spine initially comes up. So that is why you have spine, you have sub interfaces configured with how we learn tag of four DHCP relay configured. Apart from that, of course you need bidirectional PIM configured. This is for your broadcast unknown unicasts and multicast traffic. So multi.24:32Destination traffic essentially, so for that we will have to configure bi directional PIM. Then for reachability to be exchanged, you can either have OSPF or BGP. And the last thing to take into consideration is whenever there is a VXLAN encapsulation needed, obviously we will have to account for the MTU. Now.24:52This is a little more detail which we'll talk about in the next slide. If let's move on to the next slide, please.25:12Yeah, ok. So now the mtus, so we have two different mtus, which is your data plane MTU and your control plane MTU. So data plane MTU is when your servers or endpoints are connected to the leaf, right? So this sends some packets, that is your that is essential.25:23For your data plane. This gets encapsulated in a VXLAN encapsulation when going to the other port. So for that encapsulation around 5054 bytes of overhead we will have to account for. Then you also have your control plane MTU, so which is basically we spoke about BGP, right? So there is a TCP connection that will be.25:43Be established for BGP. Now that by default is around.26:039000 bytes. So this is the default value. Now, what if you have an IPN that does not support 9000 bytes? We've also come into scenarios like that. If your IPN does not support 9000 bytes, then there is an option on the fabric where you can set the.26:07The MTU value to a value that is supported by your IPN. So that is possible. I'll show I'll show where you can find that in the demo, but that is an option that is possible. Now this option is only for your control plane.26:27Traffic which is that TCP for this BGP, right? It's only for that communication, which means there'll be a handshake, right? So let's say from an epic over here, a leaf is being discovered over here. You will see there is a SYN going synac coming and your TCP packet going. If that TCP packet goes with, let's say a thousand 500 MTU, but your IPN supports anything.26:40Less than thousand 500 MTU, then the don't fragment bit will be set and the packet will get dropped off. So to resolve this, there is an option on the fabric where you can tweak the MTU.27:00But also, what about your data plane MTU, right? Just give me a SEC, what about your data plane MTU right? So that is 2nd.27:11Second thing. Now the data plane is for your VXLAN communication. So for this as well, starting from releasing 03F, there is an option where you can adjust the DCP MSs adjust option on the ACI on the epic. I'll show you this as well in the demo, but just to remember that these are options, these options are possible.27:31Let's move to the next slide.27:51Okay. I think I just saw a question on the chat, of course I'll let Manoj and Ronak answer it, but for the back to back spine connectivity, the spines are discovered using LLDP, so you'll have to ensure that you know the spine LLDP packets are flowing between the two spines. But of course if Ronak or Manoj want to add on top of it, please feel free to. I just saw that question.28:01Pop up. Ok. Now let us go to QoS. Again, if there is a situation where you suspect any congestion in your IPN environment, right? Then there is an option. So again, between costs and QoS, by default your QoS takes place in term.28:21Because of the cost bits. So cost is your layer two priority tag, but your layer three priority tag. Now when the traffic goes between the two ports via.28:41Costs, based on the cost values. There is a chance that your IPN, your IPN is not managed by the epic, right? This is any layer three network. So there is an option, there is a possibility that it might change your cost values. And so when you receive the packet on the, you know, on the other port.28:55The priority that you wanted it to take may not happen, so you might see maybe latency while the traffic is passing or some drops. Now to avoid this, there is a place on the ACI where you can map the cost to DSCP. So which means cost value is present in the VXLAN outer header, you map it to a specific.29:14DSCP value based on whatever mapping you do, you can have different DSCP values to prioritize different traffic. E.g., epic traffic needs some priority, span traffic needs another priority or you can even say there's a specific EPG and that carries very important traffic, so I want to prioritize that in a priority queue. So.29:34So based on the mapping that you.29:54Provide the cost gets mapped to a DSCP value and that will be maintained throughout the IPN, and then once it comes to the other part, it'll again decapsulated together the actual cost value that was your intent. So there is an option to again tweak this value. More on this will be covered of course in the deep dive.29:57Session but here we are just telling you that these options are feasible. Let's move on to the next slide.30:17Okay, so now coming to the interpod MPBGP EVPN control plane. Yeah, let's load up the whole slide on it, thank you. Okay, so now in a normal BGP, a VXLAN EVPN fabric, right? When an endpoint is being learned on a leaf.30:27It will propagate that information to the other leaves using the BGP updates. But where is the ACI different is ACI uses Coup as this control plane protocol, right? And if you have configured bridge domains on the epic, right, you would remember this hardware proxy setting that we usually call out. So what is this hardware proxy.30:46See, your spine is basically something that is serving as a proxy. So.31:06Now let's say you have an endpoint one, which is being learned, which is sending some packet, right? So the 1st time it comes up, it sends some packets, so the leaf learns that, ok, this is where my endpoint is present, or maybe there's an ART request coming in and hence the leaf learns that this is the endpoint that is present. Now this leaf will not update all the.31:11Other leaves in the port. It would instead use the protocol of coop to update the spine that this endpoint is learned over here. And the spine maintains a database called as coop database or otherwise called as the proxy database. And this database is synchronized across.31:31All the spines, then there is also a synchronization across the boards. So which means all the spines know where the leaves are present or at least see now if you see EP three, it knows that I have to send it to proxy B And then proxy B will know EP three is connected on leaf four. So which means the spines will know.31:51Where the endpoints are connected leaves need not know where the endpoints are connected. And for this database, the information and we already discussed that Coup is limited to a specific port. So for the Coup information to go to the other port, we have MBBGP EVPN, so it's almost like you can call it.32:10Some sort of redistribution that's happening to advertise this information to the, to the other part as well. Now this is in terms of your control plane.32:30I think I I think ok let's talk about the data plane as well. I don't think i've made a separate side for it, but think about the data plane as well. Now, imagine you have, let me just clear this annotation. We'll talk about the data plane as well using the same example. So let's say you have EP one and it wants to talk.32:40To EP three and there is a contract between these two endpoints allowing this communication. So EP one will send the packet to the leaf. Leaf will know that EP one is present on this interface, and then, but it will not know where EP three is present, right? Only the spines know that Leaf does not know it. But when you configure hardware proper.33:00Proxy, the leaf will send the traffic to the spine proxy, which means to the spine. The spine will know that EP three is basically present somewhere in proxy B behind proxy B So this will encapsulate the packet and send it through the VXLAN tunnel.33:20Two proxy B Proxy B knows that EP three is behind leaf four, so it will send the packet to leaf four. Leaf four will now learn EP one because in that packet you will have the source details as well, right? So leaf four will learn the address of EP one. It will also check is the policy enforced already.33:38Because we had a contract, right? Now leaf over here wouldn't be able to enforce the policy because it doesn't know where the other endpoint is present. Because of that, it would add a bit to say that policy is not enforced. So when leaf four on this side sees that the policy is not enforce.33:571st it will check the contract, ok, it's allowed and it will enforce the policy and communication will happen. Now for the return traffic from EP three, now this leaf knows that the other endpoint is present behind leaf one saying port one. I already have that information because I learned it from the previous packet. Now because of that, it will.34:17Check its policy, it knows that there's a contract and wherever recommended, right? The policy should be applied on the local port, which means this is where the packet is coming from, right? So if I know this is where the other endpoint is present and I can get the policy and I know the contract that is configured, I will apply the policy here.34:37And then now I will send the traffic to the leaf on the other port. Now leaf on the other port will also learn where this endpoint is present and any subsequent communication will happen directly from the, on the tunnel from the leaf to the leaf. It NO longer has to go via the spine proxy. With that.34:55Being said, I think I'll just take another moment to talk about the remote leaf architecture as well. Let's go to the next slide.35:15Before we go to the demo. Okay, next slide please. Okay, so now talking about the remote leaf architecture, so as we were talking a little earlier, this is exactly like a multi part, but the only difference is that.35:23And I don't need a spine. So my remote leaf will directly connect to IPN and the IPN will connect to the port on the other side. And when I bring up a remote leaf for the 1st time, I still have to mention which port will it belong to. So it's more like an extension of a primary port. So e.g., if when this remote.35:42What leaf is coming up, I'm saying, hey, it belongs to port A then what that means is for the data plane communication, it can directly happen from the leaf, ok? Let's say even between remote leaf in this site to remote leaf in another site or pod, right? The data plane communication, as you can see in the yellow line can happen directly between.36:02The leaf to leaf. It doesn't have to go to the spine. But for your control plane communication, the traffic will go to the spine of the.36:22Port that you have mentioned that, ok, this remote leaf belongs to this port. So e.g., if you say a remote leaf one belongs to port A, then for the control plane communication, it will go via the spine in the port A but there is also one difference between this control plane, you know, communication.36:32We discussed earlier that **** only remains within the port, right? It doesn't go to the other port. But in the case of remote leaf, this control plane has both the protocols BGP as well as your cooper, which means your endpoint reachability information. Also, if you have any external networks that are being advertised, that information also.36:51That control plane or that information is also exchanged via this control plane connectivity. For data plane connectivity directly leaf to leaf that is possible. Now, there is also one feature which will help ensure, I think there's another slide, right? Let's move to the next slide.37:11Let me just clear this. And one more thing is the RTT rate for remote leaf should be 300 milliseconds to the port, so that is the RTT. And now this is what we were referring to, right? So if you have pod one, and this is also part of pod one. So we mention which port the remote leaf should be.37:31Reaching out to for the control plane communication. Now there is another feature which means it's called this port redundancy feature, which means even though it is now part of pod one, let's say your connectivity with port one goes down, then you can enable this redundancy feature on the epic. I'll show you how that.37:53Will help you move the control plane to port two. If pod one is down, then your next control plane traffic can go via the port two's, you know, spine because spine main maintains the database, right? But again, between your leaf to leave your data plane traffic that can directly go between the leaves. Whatever we are referring to is only for the.38:11Control plane for that control plane redundancy if one of your control plane goes down, it can switch to the other control plane. Leaf happens directly from leaf to leaf.38:32I think that is all. Let's move to the next slide. I'll quickly switch to the demo as well. Ok, so this is about the control plane, the, you know, the, how the data plane and control plane works, we will share this with you. Now let me share my screen and we'll take you.38:42Through a few things that you know you can use in your environment if you have a multi part or a remote leaf architecture. Rona, you may stop sharing I'll share my screen. Yes.39:01Thank you. Ok, so firstly, this is an epic. Let me go to, this is a specific site or a fabric which has two pods. In one of the port we have two leaf switches, two spines and two remote leafs, and on the other pod we have two leaf switches and one spine.39:20So this is what the topology looks like. Of course you can see the tep pools that we have assigned. The con the epic clusters will use the tep pool of your seed poll, seed port, but your other devices on port two will use this other tep pool. You can also see this from the CLA, right? So just.39:40Read.40:02This should give you the IP addresses that are assigned from the pool. This is one that you can use or if you have, any issues when the pod is coming up or when the cluster discovery is happening, AV read is a common command that you can use. Before I switch back to the GUI, again, if you want.40:03To, you know, see where an endpoint is learned. You can check this. E.g., this is an endpoint that is learned on my port 112-01-1202. This is part of port one. I can still do show endpoint IP 33.40:23Now this is learned via port two. Show endpoint Mac also works if you want to raise a Mac address, then you could do that as well. Or if you want to do this via the fabric, you can also do this where you have an endpoint search, right, which is your EP tracker.40:40So you could put this and then you can figure out where was this endpoint learned, which tenant, which application profile, when would, when was it learnt unlearned? So all of that information you can see over here. Then you also have system settings. This is where I was talking about your TCP MSs are just policies over here.40:57We also have.41:18The control plane MTU that can be controlled over here and, yeah, your remote leaf redundancy policy right that I was talking about in case the port is down, the control plane will go via the other port. So that can be configured over here. It is also important.41:21That your epic in general connects to the inter site through which you can get some advisories and information if at all your you know ACI infrastructure is being or if your epic is being impacted by any advisories, but Ronak will talk a little bit about it later, but this is just I'm just showing you in the demo. This only needs an op.41:40Outbound connectivity to inter site, you don't need an inbound connectivity at all. So this is one thing in case you've not set up already. I would urge you, we can share some documentation on this. This is something that can be set up for you to gain better visibility into your controller.42:00Now, apart from this, if you go to the infra tenant, you can see the L three out, e.g., right? So you can see the routed sub interfaces and VLAN four that we were talking about and the DSCP policy that we were referring to.42:17So that can be found over here. Now apart from this, in case you have a Nexus dashboard, again, Nexus dashboard, doesn't need a license in itself, so if you have a DCN license for your switches, right? That is sufficient for you to spin up your own nexus dashboard. So if you don't have it all.42:37Already I would encourage you to, you know, spin up a Nexus dashboard because it doesn't need any additional license. Of course, if you want to use some premier features, then you will need a premier license, but some features you could just use as is. E.g., this is a search and explore. So you can even type a command like can endpoint for.42:57Example now I had an endpoint, right? This one talk to endpoint.43:17This one on ICMP. So I can even ask questions like these where it will tell me if the endpoints can talk to each other or not, and it'll give me other information, right? How are they talking? What is the prefix? What is the VRF? All this information can be found in one place. If there's any policy that is being applied.43:23So all of that will show up over here.43:43And also in general, right? You could also look at site one e.g. if you want to get a better understanding of the topology, just give me one sec, yeah, so if you go to manage and fabrics, right, you could also see.43:46The, topology that is there over here. E.g., this is site one. So you can see that.44:05Yeah, so you can see even at a switch level, right? You can get everything at one place. What is the interface? It is, is it up or down? Where is it connected to?44:26So that sort of information you can get over here, and I actually wanted to show you just one more thing before I pass it on to Ronak, just bear with me for one sec. So the topology that you have over here, right? So you can do a.44:45Deep dive e.g. like I was telling you, we have site one and we have site two over here. So if you double click on site one, how does site one look like? You have two pods. If I double click on pod one, what is connected over here? So you can see we have two remote leaves, you have two spine switches, two normal leaves.44:59So all of that you could get from one place epic controllers that are there over here. If you have vCenter integration and an endpoint learned via vCenter, right, you can even see that in this topology diagram. So there are a lot of features. There is also connectivity analysis that you can do from the, you know, nexus dashboard. This is almost like.45:18Like an Elam, this will also give you a good you know graph to check where is your endpoint connected, how is it talking you know across to the other port. So all of these are some features that you can use apart from, you know, generally monitoring the falls and other best practices, right? Like always make sure you have that.45:38Backups configured. There is a remote location where a backup is always being sent to or a config rollback, right? To take a config rollback so that if at all you want to roll back to a specific configuration, you can do that. So those are all general best practices that we would of course recommend, but these are some additional things that you can look at.45:58And in case you would like to explore any of this further one on one with us, right? Towards the end of the session, we'll be sending you an email as well. Do respond to that. We can help you in terms of if you want any automation or if you want to upgrade your fabric to keep it up to date, right? We have some CSS offerings around that as well. We can help.46:16Help you with pre post checks. We can also proactively engage the view in case any issues arise with your upgrade. So all of these are customer success offerings. So if you need any inputs on this, please reach out to us one on one. With that I will now pass it over to Ronak to continue with the remaining.46:36Over to you Ronak. Thank you very much for such a wonderful presentation and give a giving us a glimpse of how multipod works, how we can configure remotely and what are the things that we would require to set it up in our own environment with the demo as well.46:54Okay, so now that we have seen what multipod is and what is remotely and we also had a demonstration of APIC and a little bit of dashboard. Now this is the time to switch gears and go into multi site.47:11So multi site is the ideal architecture for loosely coupled data centers unlike multi pod, multi site consists of separate ACI fabrics. If I make it up here, yeah, so you can see that there are two fabrics here. Just one thing Ronak, I think you're not in presentation mode. You're in the.47:27Speaker mode.47:48Thank you. Okay, so here you can see that we have two fabrics here. If I bring up in, you have one here and another one here. So these two fabrics are independent AP clusters, they have their own AP clusters of their own and they're independent to each other.47:50And there's NO, because they are independent, so there are NO latency latency related limitations between the fabrics, which means you can deploy this across any geographically locations and you do not need to worry about.48:09Meantime of resolution. Now, when you talk about Nexus dashboard or Nexus dashboard orchestrator, depending on which version of Nexus dashboard you're using, so this is the central management component that pushes.48:26The cross configurations for both of your sites if you have in case of two sites that you're configuring and managing through your next dashboard, you can configure that through the nexus dashboard and it will be that single pane of glass for you through which you can configure anything.48:40Regardless of which site it belongs to. So the control plane uses MPBGPP, EVPN between the sites and the data plane uses VXLAN encapsulation across the sites. You still get end to end policy definition and enforcement.48:58The policy mode is autonomous, so each fabric can operate independently. The key components here include inter site network, which consists of, which can which connects the site one and the site two.49:15Via VXLAN and MBBGP EVPN and Nexus dashboard is used for central management through REST API and GUI as well. So you have both the options. You can configure it through API or GUI. The options are open to you.49:31Okay. Now, multi site can also be used as a purely autonomous deployment mode if the fabrics are out operated in independent autonomous sites. NDO.49:48Can still be used as a single point of provisioning without using ISN and VXLAN EVPN for East west communication. The inter site layer three communication is still possible via L three out if you want to do that.50:02Now Ndo can still be used to replicate configuration across sites when you are associating the same autonomous template to up to a hundred fabrics that you have. This is useful when you want a centralized configuration management.50:18But you do not want to stretch your layer two capabilities or the direct East west VXLAN connectivity between your sites. So this is the diagram that highlights the same that you can see over here. Now, if we talk about inter site coming connectivity deployment considerations, so there are some considerations that you need.50:34To ensure while configuring the inter site connectivity, which will be essential for your deployment to work towards best capabilities. Now, if we talk about the requirements here for the requirements, there are a few requirements here that is.50:54It must be independently configured as day zero configuration. The I put the IP topology can be arbitrary, but it is not mandatory to connect all the spine switches to the inter site connection that we have.51:13The main functional requirements are that OSPF or BGP should be enabled to the peers with the spine switches and it should exchange the addresses for reachability. You must also use sub interfaces with VX or VLAN tags for between the spines that you have.51:30No multicast requirement are there for bomb traffic for forwarding between your sites. This is just a s a significant advantage that you have when you compare to multipore. And you also might have to increase your MTU.51:50To at least 50 or 54 bytes more to handle the VXLAN encapsulation that is there as a overhead when you want to have an ISN network of your own. So these are few requirements that we need to ensure while we go ahead and configure the ISN network.52:06Now just like a multipod, there are two different MTU meanings in multi site as well. One is the data plane MTU and one is the control plane MTU, this was also something that we saw for multipod when was covering it. So the data plane MTU is the MTU of traffic generated by.52:25The endpoints that we have, it could be the servers, it could be the routers or the service nodes that are connected.52:45To the ACI leaf node. You need to account for 50 bytes of overhead for your encapsulation for the inter site communication. And you also have the control plane MTU, so this is traffic like MBBGP sessions across the sites, the control plane traffic.52:51Is not encapsulated the different the default value is 9000 bytes, but it can be tuned on Apic as well to match the maximum MTU value that is supported by your ISN as well.53:11So it is essential that we have, we ensure that both the MTU values are properly configured when we talk about the data plane MTU or the control plane MTU both have to be configured properly so that we don't have any.53:26Drop traffic or even packet fragmentation. Okay, but now what if there is a scenario which is a common question that we often get that what if your inter site network.53:43Only that it only supports 1500 bytes of MTU. This is a challenge because the 50 byte VXN overhead would cause fragmentation for standard sized packets, but the good thing, but the good news is starting from the release of six.zero in ACI, there is a solution for this so.54:00What does the solution look like? So introduced the TCP MSS adjustment.54:20Functionality to handle the situations where your ISN only supports 1500 bytes MTU. This feature is enabled at system level and you can go to your system settings and then.54:27You can go to adjust your TCP MSS settings for IPV four and IPv six as well. So there are three options that you have over here. One is the global option which applies to all flows including the multiport multi site and remote leaves. You also have one for remote lea.54:43And finally you also have one for remote leaf only. So all these configuration options are there to you and you can update it according to your needs if you have a constraint of using only 1500 bytes for your ISN network, ok?55:03Now, this was something that was also talking about during her demonstration that is your fabric currently connected to inter site for telemetry analytics? We just want to know, is it connected?55:20Or not, and in one more question in the future, we will also try to understand if it is not connected then what are the challenges that you're facing that you are not able to connect it to your environment? I see that there are.55:35Okay, we see around 25 people responding and 50 % of them mentioned that they have not connected it yet. Around 30 % have connected it and 21 % are not sure.55:51Okay, we see the numbers changing a little as we go along. Okay, so the general trend that we can notice as people are still responding is that a lot of people have not connected it to their inter site.56:06And we see the numbers still increasing. So the next question will definitely be something that will help us understand what are your challenges in connecting it to inter site. So if you could also answer to this question that what is your biggest challenge when you are connecting inter site.56:24To your ACI.56:44So that we can understand better and try to clarify from our side that how it will help you in your environment when you connect it to inter site as well ok so as people are starting to vote.56:46In we see the majority for now being security concerns about cloud connectivity. We are also seeing some that are saying unsure about licensing complexity, ok, NO strong business use case.57:05And finally operational ownership who manages it. Okay, as we see majority again, moving towards the security concerns about cloud connectivity. Now few things that I want to clarify over here is inter site.57:22Yes, it is a cloud connection to inter site SaaS, but to connect to inter site you do not have to give complete internet access to your fabric. The only access it requires are few URLs, we can share the link for you as well that which are the URLs you need to give access to.57:43And it also.58:03Gets that access through port 443, which is again a secured connection through HTTPS. On top of that, we also have TLS based encryption. So all the data that happens or the communication that happens is through an encrypted channel and everything is secure when you are going ahead and connecting with inter site. 2nd thing.58:04That I would like to highlight is whichever also mentioned that it is all.58:24Outbound connection and it is ACI that sends out the data to intercite. It's not like inter site will itself fetch the data from its own sites. So it's all outbound and nothing is inbound a connection, so it's all that communication that happens from in from your device to intersite.58:30That's how the communications happen. 2nd thing that there is some lack of clear steps as well. Now the steps are very easy to be honest. On your ACI fabric, you will have a.58:49Under the system tab, you will have a option called as device connector under that settings. So once you go to device connector, once you give access to those few URLs that inter site uses, it, there are three or four URLs and there are also documentations referencing what those URLs are used for.59:01So there are some URLs that are used for advisory, there are some URLs that are used for updates, so every URL and what it is used for is also documented in our.59:20Documentation which we can share after the session with all of you. So once you enable those URLs through those ports and you also configure your DNS and NTP on your device connector, those are few settings that you would need to configure on your device connector as well, which you would already have in your environment. So again, nothing major there as well.59:31So once you do that, then you can just go ahead and get a device ID and claim code on the right of your screen. And once you get those, then again, it's a very simple process. You need to go to inter site. If you already have an account, really great. You can onboard your ACI fabric to the same. If you don't have an inter site.59:51You can go to inter site.com, that website, and you can just sign up.01:00:12And create a new account. So creating a new account doesn't require any licenses or anything. You can just go ahead to intersite.com and create your own account and then you need to give the name of the account that you want to use and also.01:00:17Select the region, so if you don't have any data locality related concerns, you can go ahead and select US region. But we also have a support for EU or EU central region where Intersight is also hosted. So Intersight is currently hosted in US and also EU central. You can choose between them, but if you don't have any.01:00:35Data concerns, you can just go ahead and use a US.01:00:55Then once you select that, after that, you can go to system tab on your inter site, then you will have a tab called as targets, then you select claim a new target and after that you can just copy the claim code and device ID that you got from your APIC to the inter site and you can just click claim and the claiming process will be done.01:00:58Yeah, so this is a very simple process.01:01:18Just the prerequisites have to be ensured that you get them done. Those URLs need to have access through port 443 and then just claim using just copying two values. One is the device ID and the claim code and pasting it to inter site and just claiming. So these are the only few steps that are required. So I think we are.01:01:20Are covered with most of the responses. We see one more being that operational ownership who manages it. You do not need to manage anything on inter site to be honest, it will give you advisories alerts through the SAS automatically on your environment. So there is nothing much to manage, and as it is a SaaS platform, there is.01:01:41There are NO need for any upgrade that you need to do.01:02:00Or any maintenance is not required. It is completely hosted by Cisco, so all the new features, all the new information gets populated automatically. So there is NO need to managing it as well. So I think we have covered most of the queries. Hopefully, you will now feel a little more confident in your environment too.01:02:03Onboard your ACF fabrics to your inter site account. But still if you have any other queries regarding any of this option that you chose or any other concerns that you have, you can reach out to us. We will be sharing our mail on which you can reach out to us in the chat and you can send us your queries there and we will be more than happy to help you.01:02:23Okay, now that we were talking about these telemetry and monitoring features, now I think it's the best time to transition into Nexus dashboard. Now Nexus dashboard was covered slightly by as well during her demonstration.01:02:44But I want to cover a little bit about Nexus dashboard in case you are, there are few customers who are not aware of the evolution or about a few information or brief description of Nexus dashboard. So if we talk about the evolution.01:03:01So, at 1st we had, the next dashboard that was, so we had different products that are the next dashboard insights, the next dashboard orchestrator, and next to the dashboard fabric controller for plan or SAN.01:03:17But starting from version three.one and three.two and even four.one above. So we have now unified the next dashboard into a single installation image, either through an OVA or a virtual deployment or ISO for physical appliance. Now this unified appliance.01:03:36Go significantly simplifies the deployment and management for all of your nexus dashboard services. So this is what it used to be there before.01:03:56Version four version three.one and now it is completely unified as a single solution. Now, if you talk about the various form factors that are supporting orchestrator, because in this I am more focusing on the multi site of things. So for orchestrator.01:04:04Based form factors. These are few supported options that you have. One is the three node physical appliance that you have, the physical nodes that you have in your environment. So it supports orchestrator and also telemetry up to hundred switches of G two and 400 switches for the L four.01:04:24With 10000 flows or hundred thousand conversations per minute. So these also support upgrades that you can do in future. You also have six node physical servers or physical nodes.01:04:43Which supports orchestrator telemetry for up to 20 fabrics, 500 switches, 20000 flows and or you will have conversations per minute with upgrade support as well. Then you have the three node in the.01:04:58Node G5S, which is a new hardware supporting all features for 500 switches and thousand flows or one leg conversation per minute. This requires a new installation and upgrade is not supported for this use case.01:05:14Then you have three node virtual apps as well. Now you can deploy nexus dashboard as a virtual appliance as well on your environment on your own servers if you want. It supports orchestrator with upgrade capabilities as well. So it is mainly suitable for smaller deployments. If you have smaller deployments and you want to test out Nexus dashboard, you can go ahead.01:05:31With a virtual.01:05:51App nodes as well, which is in a three node fashion. Then you also have three node virtual data nodes. This is new and supporting all features for up to hundred switches, 2500 flows, 5000 conversations per minute, but this again requires a new installation and upgrading.01:05:53Is not supported for it. Now finally, if we move on to the 6th node virtual with three node apps and three node data. So this is again a new option supporting all the features for two, 200 switches, 5000 flows and 10000 conversations per.01:06:12Minute. So these are few options. Again, this is a new solution and upgrade is not supported. There is one more option that is a single node, but that is again only support only recommended for lab use and only for testing purposes. It is not at all recommended for.01:06:32Your production environment. And there is a very nice tool that is the capacity planning tool which is available on Cisco website. The URL is present on, on the slide. We can also paste it in the chat and you can go to this capacity planning tool and understand what are the requirements that you would be.01:06:52Having if you want to deploy maybe a virtual ND, what are the things that you need to ensure and even if you want to go ahead with the physical ND, how many nds would you require for your use case?01:07:11So you can do the capacity planning through this tool, ok? Now if you move to the next slide, so to upgrade to next to dashboard three.two, if you are in our older way version, then.01:07:23From MSO which was the multi site orchestrator and all the versions of MSO or if you want to or if you have ND version four.two or earlier, upgrading from MSO requires a backup and import step to the next dashboard orchestrator.01:07:41To, four.two. From there you can follow the upgrade procedures and this involves upgrading our backup and importing it and then you can roll back the configuration to the backup file and all of your configurations will be there.01:07:59Now if you want to upgrade NDO up to three.seven.one upgrading from next to dashboard three.seven.one or earlier can be done through a multi step upgrade, backup and restore is also an option but you can only restore up to NDO 4.2.01:08:15Again, you can export backup and import backup and roll back configuration here as well. Now from Ndy versions three.seven.two or later, upgrading from these versions can be done following multi step upgrades or you can also have three.seven.two backup that can be imported directly into nexus dash.01:08:33So the documentation is there in this.01:08:53Age as well. So we will probably give all these links to you at the end of the session so that you can refer to all these all these details and decide what upgrade path you need to do depending on which version of MSO or NDO you are using in your environment.01:08:56Now if you want to upgrade to Nexus dashboard four.one, these have some specific requirements you must be on Nexus dashboard three.224.one any one of the versions and this will be a direct upgrade path and.01:09:16Only upgrade is supported in this case, NO backup or import is supported for from the older versions that you have. If you're changing form factors, e.g., if you are moving from a virtual app to, from a virtual app node to a virtual data node, you need to upgrade to four.one 1st, then.01:09:31Can back up the cluster and restore the backup onto the new cluster with the different form factor, and this is the diagram where you can see there are two upgrade paths, the direct upgrade from three.two to four.one and also you can back up and restore.01:09:50Using this upgrade path as well. Okay, now let us talk about the ND high availability. So in this case, at least two ND active or primary nodes are needed to keep the cluster operational. In case one primary.01:10:10Fails, then the cluster will be operational but in a degraded state. So there are few use cases that are few scenarios that we have covered here. I will quickly go through it. So for next dashboard orchestrator, high availability, this approach is.01:10:30The approach, the recommended approach is to centralize the next dashboard cluster deployment. This provides low latency communication solicitating quick data exchange between the sites that you have. Services require RTT between the nodes that ranges between 50.01:10:48Milliseconds to maximum of 500 milliseconds. So the diagram shows that site A fabric A with a cluster of primary one and primary two and primary three. So all these are present over here and there is one standby.01:11:08And you also have standby nodes with RTT of five or 50, and there is a separate DR cluster on standby as well in this configuration. Okay, in case of a two side deployment it is not recommended to deploy a distributed cluster, so.01:11:28It is not recommended to separate your primaries.01:11:48Across the two sites that you have. It is always recommended to have the primaries on one site if you have a two site deployment, ok? But in case you have a three or more sites, the distributed nexus dashboard deployments become viable, you can go ahead and have your primaries distributed across your sites.01:11:51If you have three or more sites in your environment.01:12:11Okay, let us move quickly. I think we are running a little short on time. Okay so here are a few recommendations or few things that you need to ensure when you are going ahead with.01:12:15With your multi site deployment. So one very critical gotcha to understand is you should never mix the ISN and the layer three parts for the east west routed communications between your ACF fabrics.01:12:35The routed communications between the different fabrics can be established in two ways. 1st, you could or or this is the actually the recommended way is to use VXLAN tunnels established via the inter site network, and second, you can have it through the layer three communications deployed in each fabric leveraging.01:12:48Connectivity services off the external layer three network domain that you have. But mixing these two is highly discussed because it may lead to unexpected dropping of traffic. So this is one thing that you should ensure. Now, ok, this is something what you see when you deploy.01:13:08Template. So this is a graphical view that you can see from here of the template that you're going to deploy. But what happens is in the back end you can also see the JSON view with the payload that gets sent.01:13:26To your apex for the configurations to happen. So you can also see this in terms of a JSON payload as well and that option is supported on the UI as well. Okay, now what happens if you change NDO objects?01:13:41On epic. So e.g., you are managing your APIC through NDO and there are multiple sites that you have on maybe site one, you change, you make a change on your epic, and it is not done through the NDO, you do it directly on the epic. So what happens in that case? So there is a drift detection that happens on NDO, so any changes that you.01:13:58Make on your Apic directly and which is not there on your NDO will show up as an out of sync and it will highlight you that this is something that has been changed directly on the endpoints and you can compare also that what are the changes that have been made and then you can.01:14:18Ensure that if this is the change that you want on NDO as well, then you can sync the status and the configuration across your APIC and your NDO as well. Now, after that what do you do is the drift reconciliation. Once the drift is detected, NDO provides a reconciliation workflow at the template level, it displays the.01:14:36Different different view showing exactly what is different that has been done and you can compare and see whether you want that change to happen or not and accordingly, you can ensure that the change happens in your environment. I will quickly stop sharing my screen and share my.01:14:56So that I can share a brief walkthrough of the Nexus dashboard that we have in our lab. Okay, I hope it is visible. So this is the same lab that was also showing up.01:15:13I'll just quickly go through a few tabs here. So once you go on to your nexus dashboard, this is some, this is the main view that you see on top. In this you get to see all of your fabrics according to the geographical locations what you have set while you are onboarding the fabrics. You can also see the topology.01:15:33And you can also create dashboards of your own and you can so you can see these are all dashboards that are created for our use case. E.g., if I just open one dashboard, we can see various widgets according to our needs. So we have the fabric widget, devices widgets, and a lot of different.01:15:52Visualization that we can see on this portal. You can add widget, you can edit this dashboard, whatever you want, you can have and this will help you in your day to day operations and have a clear visibility of what is happening in your fabric from a top down approach. So this is the 1st thing that you can see on your.01:16:12Tabs. Now this is the managed tab that we will focus on mainly. So in manage you can see the fabrics that are onboarded on this nexus dashboard currently. So we have a few fabrics here, few ASA fabrics, few classic LAN as well.01:16:32All those are there. So if I just open a fabric, you get to see a view like this. So what is the license data that you are using, the software version, the BGP configurations, there is also an interesting tab of energy management of how much energy you're.01:16:50Utilizing and how you can save your energy related consumption, you can reduce your CO two usage. All those are also here. You can get to see your inventory, the switches that are there, you have three AP controllers, there are VPC pairs, all these things you get to see.01:17:09You have other tabs as well to see the the different configurations and inventory that you have. Similarly, you have other tabs as well. Now, one more thing that I wanted to show is this orchestration tab. This will be the main tab when you want to orchestrate between the sites that you have.01:17:27So you have few tabs over here, you have the tenant tabs and you also have few templates that you can create. So you can go ahead and create your own template and then you can go ahead and deploy those into policies and those policies can be used across your sites and you can use it.01:17:46To both of your sites if you need to use it there. Now, if we just open the tenants part, if it loads up, ok, so by the time this loads up, we will go to.01:18:05This as well ok now you can also manage your various images and up the fabric softwares, the upgrade files, all those could be stored on next to the dashboard as well and you can push your upgrades through this.01:18:23Next to dashboard as well and you can configure your upgrade paths, your upgrade groups, and then you can plan your upgrade through the next dashboard as well. Apart from that, you also can create your templates and there is the library of template here. You get to see a lot of templates that are present over here and these templates can then in turn.01:18:43Be used to create various policies of your own. Now, as we are running a little short on time, one more thing that I wanted to highlight is after you deploy your nexus dashboard, a very important documentation that will be helpful for you is this help center, which you can see in this cross.01:19:02Mark on top. So any questions you have regarding nexus dashboard, any of the options, all of those could be found out in this. Your API references could also be found out. If you want to use API for your use cases, you can go ahead and use it through this. Any kind of questions you have, everything will be covered on your.01:19:21Help center and this is available to you offline. If you have a nexus dashboard appliance in your system as well, this is available to you offline. You do not need to go to any internet site. You can see any kind of information from here. So if you have any info like if you have any queries or information you need, the 1st step.01:19:40Which will be to go to the help center.01:20:00Okay, now I'll stop sharing my screen and hand it over back to conclude the session. Thank you so much, so we still have 2 min. I hope all.01:20:02If you were able to at least address most of your questions, thank you Manoj for being active on the chat. Ok, I see your screen share is back on. But for the others, as I was mentioning at the beginning, we will also have an exclusive 16.01:20:22And it's a Q and A session, which is the 4th session in our series, so we will send you the link to our community page. Keep it bookmarked. You will see all our upcoming sessions there along with the recordings of this session and our previous session as well. So if you wanna forward it to somebody who might benefit from it or if you want to watch it.01:20:42Again, we also have, we will also be sharing you some of these quick links that you can use to go through the details and share our email as well. If not shared already, we'll put that on the chat so you could reach out to us either by email or check out the community page for details or.01:21:02Attend the next session, so the next one is next Thursday. We'll do a deep dive on multi pod and remotely. We have another one the subsequent week and finally the Q and A session. Apart from this, as I was also mentioning, we have some CSS offerings in case you want us to help you with some automation use cases related to your multi pod multi site.01:21:22Or if you, you know, want us to help you with your ACI upgrade, we have some CSS offerings around that as well to do some health checks and status checks before and after upgrade. So just reach out to us. That is all you have to do and we will be happy to continue working with you.01:21:42With that, thank you so much, everyone. Thank you, Ronak and Manoj for joining us today and for everybody else who stayed you know till the end. Thank you so much everyone. Have a good day. Bye.01:21:59