Multi-site deep dive

Oh, this endpoint now belongs to the remote remote spine of the site one. So it will directly encapsulate the packet and send it out to.00:00The spines of the remote site here. So that's how it will flow. So there is now this spine is just acting as a transit router, it won't encapsulate now anything going forward for this communication and like both the leaves will now.00:20To have their own VXLAN tables updated and it will send it out to the remote site spines directly. So this leaf will send the packet directly to this spine anycast IP address of the remote spine. Similarly, inside to the.00:40Will directly send out the packet to spines of the site one.01:00And that's how it will the whole communication works. Okay, and yeah, that's how the this the reverse path is also complete, assuming the contracts are in place.01:05Okay, so yeah so yeah this is the end state once the broadcast is done, the tables are updated, the leaf will have entry of the spines of the other site in case of inter site traffic.01:20Yes, so yeah ok are there any questions which you want me to answer? Yeah NO approval yeah ok ok yeah so yeah so we will also now see how this.01:40Objects are getting created in the background and what all, what all are the behavioral nuances of the ACI objects across the multi site. So if we see here like we are extending one single tenant and a VRF and.02:00The BD and EPG from one site to another site. So both these sites have.02:20Their the same tenant VRF and PG configured, ok? And we have two BD subnets which and with their own EPGs in them and then we want to have a contract to allow communication between them. So we have a.02:27Contract in which one will act as a provider and other will act as a consumer of the contract. So, if this is like how a layer two extension across the fabrics would look like. So both of your VRF.02:47BDs and EPGs are configured using the NDO and using a single pane of glass you can configure both the apic clusters simultaneously. And if you don't want to enable bump traffic across the site, that is also supported.03:08Like ACA uses a different mechanism. If you allow the bump traffic, it will send out the broadcast traffic across the site if even without inter site bump traffic support, we can still have aL2 extension across the sites.03:27Using NDO.03:47Okay, so, yeah. So here we are having another scenario that is also supported. So here the BD across the sites are different having different subnets. However, they are part of the same EPG. This is also right supported, but in.03:51This we have disabled the L two stretch in the BD while we create the configuration on NTO. However, like in this the shadow objects won't be created like as Naveen was mentioning when the shadow objects will be created, when the EPGs are.04:11Like different EPGs for two different sites, yeah, in that case the shadow objects will be created. However, in this it is a stretched EPG object which is getting created here, ok? And there is one more scenario in which.04:31Like this this is more common scenario in in case the stretched objects are not present. So we do have tenant and VRF same on both the sites multi site object. However, the BD and EPGs are different.04:51So on site one you have EPG and BD red. On site two you have BD and EPG red green, sorry, my mistake. So in this like the EPG green inside BD green wants to communicate to EPG red, so there is a contract.05:11Which, which is in place to allow communication from EPG green to EPG red. So in this case the shadow object needs to be created and this will be handled by NDO, so NDO will take care of your.05:30Shadow object creation you like we don't have and we don't have control how the shadow objects will be created but yeah NDO will take care of the shadow object creation and the contract will also be using those shadow objects if required.05:49Yeah, so this communication is also supported yeah so yeah moving forward so like yeah continuing from the demo what Naveen had shown, right? So.06:08This is how like we will do the inter fabric connectivity from the NDO which Naveen had already shown. And once the interfabric connectivity and the control plane data plane configurations are ready in place, then we.06:28We will move to our tenant templates and fabric templates as shown in this GUI. So.06:48What is a template? What are what is a template and schema we will look in the next slide as well. So here in this slide we'll just see where it, where the option is available on the NDO, so.06:56Yeah, so if you click on manage and like under the template section you will have a tenant templates and fabric templates. So your tenant templates will have your tenant policies, your L three out, application profiles, all those things.07:12While the fabric template will have the interface policies and monitoring policies.07:32So yeah, just like your ACI object creation, right? So, what are the different types of templates under tenant and fabric level config fabric templates? So yeah. So all your like EPG BD VRF.07:38If and other things will fall under tenant. However, your VLAN pools, the physical domains, your NTP SNMP configuration and like any span sessions and all will come under fabric templates, and you can manage everything within the single pane of glass using NTO.07:58So that is what this slide is about. So, yeah, moving on. So this is application template like this is important part of how we will push the configurations from NTO. So, if you see like.08:19Of.08:39A single template can have some information regarding some BD PG to be created on a, on ACI fabric or a tenant or a VRF and a collection of those templates is called a schema. So the schema is nothing but a combination.08:40Of templates. So we will on NDU the logical construct apart from the ACI logical construct, there are two new things that you need to remember. So, there is a wider container of templates called schema, which will have multiple templates.09:01And it can have a combination of templates. Some templates can are only pushing configuration on one site, some are pushing configuration on multiple sites while creating the template you will you will select that option whether it's multi site or autonomous.09:21Right, and the site APIC will get configured using all the inputs via the different templates. So here site one will have the templates coming from site local template as well as the stretch template, all the configuration will get pushed to the site.09:41So the effective policy will be a combination of all the templates which are to be configured on the site. However, site two is only having stretched template policy. So yeah, that is also supported. So yeah, so this is how like a logic.10:01Really in the background the templates and schema come together. Any questions before we jump onto the demo next?10:21Yeah, ok, ok. So, yeah, I'm I'm bringing up my here.10:40Yeah, so here is the demo which we have created. So, yeah, so basically we go to the templates section under orchestration and.10:56Yeah, just a minute yeah so ok ok yeah before that ok ok I I have stopped sharing. I will share one more thing.11:13Work.11:33So in the demo, we are trying to create this scenario, like we are trying to create a tenant called pseudoco, under two fabrics San Francisco and New York and we are having the VRF configured a common.11:40They have configured across both the fabrics. However, the BDs that are configured and the PGs, they are on their own, like, they are having their unique BDs and EPGs and there is a contract at the end to allow communication between the web and app BDs. So if.12:00You see here this tenant as well as the VRF are stretched across the two sides and the BD are BD and EPG are locally configured and the contract is also like kind of to be configured on both the ends, right? So.12:20So how will we do it using NDO, that is what we are going to see in the next demo. So yeah, I will stop sharing and start sharing my.12:40Quick time player, yeah, so.12:54Yeah so are you able to see my quick template? Yeah yes yes, ok. So we had a question like can we use multiple multi site solution you want to take it?13:09Yes, so it is covered in the next upcoming slides as well after the demo, yeah. So, yeah. So yeah, we can wait for that for those slides. So yeah, so here we are configuring basically three like L three stretch is a schema here.13:28And then we will create multiple templates inside the schema as I was mentioning that schema is a combination of templates. So if you see here there are like we will be basically creating three templates, one which will apply the configuration.13:47Across both the fabric, like the stretched the VRF and tenant configuration.14:07And we'll have two single templates which will push this specific BD and EPG configuration across both the sites like so in total we are going to create three templates inside a schema. So this is the 1st one.14:13Francisco and New York template. So we have given the name so as to justify that this is a stretched template. And we are selecting the deployment mode as multi fabric here if you see.14:33So, once the template is created, we are creating a VRF called VRF one which will come up in your screen.14:50Yeah, so, yeah, so this is the VRF one creation. We will just create it without any advanced options that are needed like in general it will be just.15:06The VRF on a routine basis. And.15:26Till now we have just created the VRF on NDU. It is not yet connected to any of the fabrics. So here we can under the actions we can click add and remove fabrics. And then since this configuration needs to be applied to both the fabrics, we will select.15:31Both the fabrics here.15:51And then, so till this point as well, the configuration is not pushed to the fabric. It will be pushed only when we deploy the template on the right. So we'll save the schema and deploy the template.15:54Oh until now it is out of sync because as per NDO the configurations are not present on the apic. Once we click deploy, it will configure those things.16:13Then it will be in sync onto as well.16:33So the apic, can manage the existing configuration on the apic, as well as this new templates which are getting pushed. It won't overwrite the configuration.16:40So yeah, we'll click close here and then so now as we were discussing this is a configuration on the like stretched template. Now we are going to create a standalone template.16:56Which is only for San Francisco configuration, and we will select the tenant here as, yeah, and mind that the deployment mode is still multi fabric because if you don't select multi fabric, there will be some issues in.17:15Had object creation and other things. So you still need to select it as a multi fabric mode.17:35Oh, and then we will create the BD and application profile and EPGs that are needed to be configured for this.17:43San Francisco site So this is the BD for San Francisco.18:05Okay. And like ok that was the application profile now BD and so there was a question when Naveen was presenting, can we also configure L four to L seven?18:22Service graph and other things using NDO yes you can. So, if you had seen here the create object also gives you L4L7 option.18:44Yeah, so, so this is and we'll select the VRF, we have disabled the L two stretch since this is.18:59Are not going to be L two stretched across the fabrics. So yeah. And then all the other options are as it is we have enabled unicast routing and unknown unicast is hardware proxy. So.19:19And then we will add a EPG inside the BT.19:39So we'll select the BD here and we'll just proceed. Yeah, and then till this point we have not selected which fabric the configuration needs to.20:06Okay, so we will select the San Francisco fabric here. So, yeah, so this is what we were discussing about that for each site it will.20:18Do all the calculations, what needs to be coming from stretched templates, what needs to be coming from standalone templates and it will push the configuration, the NDO will push the configuration to both the apex.20:29So we will save and deploy the template. So yeah, and the similar exercise we will also be doing it for.20:44New York, so we will create the New York template.21:03Yeah, just.21:23Yeah, so, yeah, we are creating the application profile now on the NDO before we push it on the apic.21:30And we are disabling the L two stretch here as well.21:51So, and then we will create the EPG.21:54Yeah, so.22:15Yeah, so we have created the BD EPG application profile for the individual and new York site and we will add the fabric now. So, yeah, and then we will save and deploy the configuration now.22:19Okay so if you are still with me, you will you would have noticed that we are not yet ready with the contract to allow the configuration between the apgs. So.22:36We just to do apply the contract, we need to have the EPGs to be created before the contract can be associated to the EPG, so now like we will.22:56Configure the contracts as well. Going further so yeah so since the contract is going to be configured on both the fabrics so we are again back to the.23:14A stretched template which is San Francisco and New York template. So it will create contract on both the fabrics together. One will act as a consumer to the contract and other will act as a provider to the contract.23:35So the contract is created, we, we don't have any filter entry in the contract as of now, so we are also creating a filter to be applied to the contract and we will also create a entry to be put in the.23:55See.24:14So basically we are allowing all the communication between the EPGs, so we are not putting any specific IP or port restriction or anything yeah so.24:16Okay, quick time check ok and then we.24:36So we now go back, yeah, yeah. So now we go back to the San Francisco template and we modify the EPG after this to have the contract associated. So we are little.24:53Short on time, I will quickly jump once the contract is configured. So we.25:13We are permitting all the traffic here.25:32And then we will go back to individual template and apply the contract. So if we see here yeah so this is in stretched template, now we will.25:46Deploy out of sync. Now we go back to the individual templates and modify the EPG configuration too.26:03Have that contract. So in San Francisco we are selecting it as a provider and.26:24In New York we are adding it as a consumer. So once this is done we'll go to the APIC and we will see whether the contracts have been created along with the BD and EPG and other logical concerns.26:33That's.26:52Yeah, so now if we go back to the individual apic.27:17We'll see the.27:20Contracts present along with the BDs and EPGs. So we have the application profile. So if you see here on the apic as well it shows this object was created using Nexus dashboard as a warning like as a information to the user that.27:22The particular object is being managed by the NDO. So if you see this blue icon everywhere it is showing like this was being configured using NDO. Yeah, so.27:41Yeah, that was the demo. I will go back to the presentation. We have one more thing to cover before we end the presentation. Yeah, so any questions Naveen that I should take up?28:00Yeah, so that is the question that I answered it. You want to add more you can add it like a shadow object concept, a shadow object concept only applicable when the configuration is pushed to be our. So what will happen if there is NO in the environment? Yeah.28:17Correct so if yeah ok so if there is NO NDO like it becomes difficult to configure both the vrfs and EPGs and other constructs across the site and you cannot.28:37Carry the ACI policy construct across the site. So what you can do at the maximum is have a, like communication via the L three out the asset to standalone DCs that will make more operational and.28:54Like use case sense then having the same BDs stretched across the across the ACI, right? So yeah, ok, so coming to the last topic, we need to cover this in next 10 min, so when we have multi site and.29:13And we want to use the L three out what all considerations you need to keep in mind. So generally in a multi site environment, we will have active standby firewall cluster of the each of the site because we are deploying different sites standard.29:33And like they will they should be having their own standby and active firewall clusters. So what happens when an external user wants to communicate to?29:54The endpoint which is in either of the DC. So this is a stretched object the BD is stretched across the both the DCs so yeah so we will look into one scenario. So both the DCs are advertising their own L three out.30:08Advertising the 1010 subnet to the.30:28Remote user. So the remote user may land on site one, but the actual traffic and host maybe present in site two. So how will the traffic flow look like? So you will like ACI will take the traffic from here and it NO.30:31Knows that it's a endpoint located in the remote site, so it will send out the traffic to the remote site. But when the remote site.30:51Sends the reverse traffic back, it will use its own L three out, right? And that firewall will drop the traffic because it does not have the state in present in the firewall state table, right? So to solve this issue, right? So, what.31:00We recommend is like we will have a.31:20Host route advertisement enabled in the L in the L three out. Okay, so that will send out each specific route to the user so the user will perform the longest prefix match and land on the correct DC and it won't have any issues. And you can do this.31:24Host route configuration on a per BD basis. So, yeah, like you don't need to advertise all the host routes, only certain routes which are stretched across both the sites we will advertise using host routes, so.31:44So then now the user will land on the perfect DC and it will send the traffic back the the traffic will not get dropped on the firewall. And there is one more concept called inter site L three out. So, yeah, so going back.32:03Back in time like before ACI four.two, there was only one kind of supported design like each of the fabric should have its own L three out like configured via NDO or or or on the Apic directly. But it will have their own.32:23L three out and the user will.32:43Access the DC servers using the corresponding L three out. However, this was not a supported design in four.21 before four.21 like if one fabric wants to use the L three out of the other fabric. Yeah.32:47Right, so this was not supported till four.two starting four.two the other design is also supported. So if you see here both of them will have their own L three out and they will like this DC in case of a failure, you can use the L three out directly from.33:06The site one also if your inter site network is in place. So this is how like the current support looks like.33:26And it will also perform the like inter VRF and inter VRF transit routing as well, right? The L three out can perform. Okay, and yeah so yeah as I was mentioning starting four.21 this is supported.33:37And it can, it will also be supported if you are using SRMPLS L three out so it will advertise all your.33:57And host routes into VPN V four and VPN V six of the MPLS network, ok? And coming to one last scenario before we end the session, if you are using NDO, like there is one use case which is.34:07Like you don't want to use both the sites as a multi site option. However, you want to manage multiple apic clusters using the NDO. You can still do that and that is supported, so in this case, the NDO will be used more as a configuration management tool.34:26Which will push all the configurations to the corresponding APIC without the multi site extensions needed. So you don't need an inter site network you can just make the NDO reachable to the apic clusters and you can still manage all the configuration using templates, schemas, and.34:46Other logical constructs on NDO and if you see here we will when we onboard the site and when we create the templates, we will use this autonomous option if we are going to use this.35:06Yeah. So, yeah, so this will use its own L three out and we will have the routes coming from the van for the remote DC, yeah. Yeah, ok. So, yeah, and.35:20So what kind of firewall integration are supported in multi site? So since we are talking about multi site, we expect the DCs to have their own firewall. So there are limited support options if you want to keep the active firewall on one DC.35:40And standby on another DC and also both active is also not supported unless you only have all the L two like you don't have gateway on your ACI fabric, then this is kind of supported and still it has some other issues. The most recommend.36:00Practices to have the like both the DCs should have the active standby firewall on their own. And we use the inter site network to send the traffic across the.36:22Sites. Yeah. So, yeah, so these are some of the sources available on internet for ACA multi site and yeah, that's pretty much the end of the session. Yeah so any question.36:34I hope the L three out question for the viewer got answered.36:55Yeah, yeah, I think yeah it's done so thank you everyone. Thank you for joining thank you all.37:03Yeah, thank you all. You can reach us on our email address which I have shared you on the for any one on one consultation or if you have any queries. Thank you, thank you all for joining us.37:19