Cisco Crosswork Assurance within Cisco's portfolio​

Oh. Thank you everybody for joining. Today we're gonna cover the CNC and Crosswork assurance integration. So we're gonna be looking at a simplified service assurance for L three VPN.00:00And services. Today for your host and co host, we'll have myself Robert Hoffman, Rajesh and Walter, and let's go ahead and get started with today's agenda.00:20So this presentation is structured like a business story. We don't want to do our product manual here. So 1st we're gonna define our two products and then we're gonna explain the value of bringing them together. After that we wanna walk through the steps in the CNC using as I like to call it plain language so that we all understand the.00:31The Role of each object.00:51And then we're gonna connect the setup to the real service example, so you're gonna see an L three VPN with assurance probes. And finally, we'd like to explain the monitoring logic, the metrics that matter and how this is gonna support a closed loop automation. You can see right here from our agenda slide, we've got what is CNC, what is CWM?00:53What is called for assurance, the integration story summary, the L three VPN overview, the probe functionality, and then we'll have some time for questions and answers.01:13So let's get started with a CNC introduction. Your crossword network controller or CNC as I just called it, is going to be your all in one network automation platform and is designed for your modern IP networks. So I want you to think of this as the central brain for the network.01:21It's gonna combine your real time network visibility, your smart traffic engineering, and your automated service provisioning into a single interface. So instead of us as engineers jumping between five different tools to find a problem, fix it, error or even like provision a service.01:41We're just gonna go to CNC because CNC is gonna handle the entire loop automatically, and this is gonna be even across your multi vendor networks. Now this slide here summarizes the core values of CNC and when we talk to operators.01:59The biggest headache that we found is that managing separate management tools in silos, so basically having to jump to individual tools to try to get a job done. With CNC, it completely eliminates that fragmentation by unifying the visibility, the insights and the actions into a true closed loop life cycle.02:19So I want you to think of this continuous life cycle. 1st.02:39We provide the total visibility and the way this happens is CNC gives you a multi vendor real time view of your topology and your traffic patterns while continuously monitoring service health against the active SLAs that you've set.02:44Second, we turn that raw data into insights. So the platform analyzes the collected telemetry to predict congestion points and to feed our optimization engines. Then 3rd, we're gonna take action. So once an optimization or a provision decision is gonna be made, CNC is gonna.03:01To execute it. So it handles path optimization for SR, SRVC or SRV six, and it leverages NSO.03:21For the multi vendor provisioning across up to like a hundred and 70 device types, and it's also gonna use low code low code playbooks to tie these workflows together into a single automated response, which you'll see later in our demo. Now.03:29Moving to the architectural piece of the crossword network controller, if you're looking at this kind of this blue outline box here, this whole thing is going to build your crossword network controller, but it's got a couple of kind of internal pieces and external pieces. The gray box is going to be your internal pieces where.03:46These boxes down here are gonna be your external pieces and we're gonna talk about each one of these.04:05But when you install your crossword network controller, you gotta remember this is gonna be a microservices based application suite. So depending on your license structure or what your use cases are, you're going to start installing different applications, and these applications are gonna be direct directly linked to.04:10Your use cases. So we've got provisioning, we've got visualization, optimization, service health, closed loop optimization automation, and you're gonna start installing these app application suites to get these use cases. Along with that, to fully integrate and create the crossword network controller you need.04:27You need an SRPCE, you'll need a data collection layer, which will be your data gateway, and you'll need your configuration engine, which would be your NSO.04:46So let's break down each one of these. The 1st one is the Crosswork Data gateway, or as you'll hear me call it the CDG. And in modern networks.04:55You have a massive variety of devices that are all gonna be producing this inconsistent data streams, and with that, you're also gonna have multiple applications that are trying to consume it. CDG solved this mismatch.05:10With this simple philosophy here, we're gonna collect once and then distribute to many destination. So what it does is it entirely offsloads the heavy lifting of data collection from your application layer. Now, on the southbound side, what it does is it collects telemetry using whatever the hardware supports and that.05:25Could be things like model, model driven telemetry in GNMI or things like your legacy SNMP, syslogs or CLI.05:45And then on the northbound side, it's gonna groom that data and it streams it precisely where it needs to go. It's like sending real time link statistics directly to a crosswork optimization engine, which you'll see as one of our use cases. Now operationally, your cdgs are.05:54Just lightweight Vmiss So if you need more capacity, you can simply scale out horizontally and CNC orchestrates them into these things called CDG pools, which gives you this built in automatic failover, so your real time data pipeline never really drops.06:11Our next external kind of component here is your Cisco NSO. And this is gonna be our core provisioning engine. Now with this, we've got a couple of different boxes that I wanna explain here, and this is just a high level of overview of NSO, but we've got our service manager, our device manager, our CDB, and our package manager, and our TSDN function packs.06:30I want you to think of our service manager as our high level intent. So this is where we manage and.06:49Review the full life cycle of things like your L three VPNs or your L two VPNs. Underneath that we have our device manager, and this you can think about as like the execution layer. It provides a single network wide API and it maintains a local live synced copy of all your device configs so that in a.06:56So always knows the exact state of the network.07:16Moving down we've got our CDB. And this is our in memory Yang database, and it serves as the single source of truth for NSO. It holds both your service intent and your device configs, and this allows NSO to perform a fast pre deployment validation and dry runs.07:20Next up you've got our package manager, and I want you to think of this as like the organizer that manages the life cycle of all the service models and translations and code. Now, the part that's gonna mean the most to the audience here because we're talking about CNC is the CSD.07:40And function packs. These are pre built Cisco engineered packages.07:58That are specifically made for the CNC provisioning of transport SDN services out of the box. So with these you'll be able to provision services on day one using your CNC. The last bit I wanted to explain here is the NED. So this is your network element driver, and you can think of this as a translator between.08:04Between our NSO and CNC and the southbound device. So if you have a Juniper device, you'll need a translator for.08:24For NSO or CNC to talk to us who you need a juniper ned. If you have an iOS 60 iOS six R Huawei, anything else, you need a separate NED for each one. So just know that the neds are the translators. Now, the final piece is the SRPC and a SRPC is a central.08:31By stateful path computation element, so a PCE and they are built natively into your Cisco iOS XR, and what it does is it acts as like the brain for the traffic engineering across your segment routing, so your SRMPLS and your SRV six networks, and instead of relying on individual routers to calculate you know the.08:51It's complex end to end pass using only the local topology data.09:11The SRPC aggregates the entire network topology and status to compute an optimal constraint based path. Now the way this works and integrates with CNC is that in the CNC architecture.09:16The SRPC serves as the critical southbound controller that's gonna bridge the abstract intent of the API down to the physical network. Now the way that you actually gonna collect this topology is the SRPC is going to dynamically learn the network topologies linked.09:33States and segment routing identifiers so your seeds by using the BGP link state so your BGP LS.09:53And you're gonna see more of this in a, in a couple of slides when we actually talk about the SRPC with the COE. With that, we're gonna move into five use cases for the CNC. The 1st one is going to be this end to end provisioning, and we're gonna walk through the.10:02MDM provisioning.10:20Of a VPN service that's gonna be initiated by crosswork. So whether you are provisioning a simple transport transport VPN or like a complex one layered or complex layered one, that's gonna have you know some traffic engineering constraints, your execution pipeline.10:21Are going to follow three clear paths, so NO matter what type of service you're, you're initiating, it's gonna have three distinct kind of phases.10:421st, we define our service intent. So an operator can either fill out like a standardized template in the CNC web UI or you can trigger it automatically via like an external API. Once received, the CNC hands this high level intent off to NSO using an API.10:502nd is the actual configuration translation, and this is where NSO uses its service packaging and the multi vendor neds to render a raw device configuration block and a critical point.11:10That I want to emphasize here is that NSO relies on accurate network state. So what this means is that devices must always be tightly synchronized with nso's internal configuration database to ensure a clean fail safe transaction when the code is pushed. So what we're saying here is we need to make sure that the code that is in.11:26Installed or currently configured on your box is the same code that NSO knows about. So we do want to do a sync from or a sync to using NSO to make sure that those devices.11:46Configuration on the box and on the NSOCDB match. Then finally, the last phase is once the underlying routers accept the configuration changes, a success notification loops back up to your CNC, so we're basically just letting CNC know that, hey, we accepted these config.11:55Integrations and they are working.12:15Moving on to our active topology. Now the active topology is gonna be another one of those applications that you can download and install. And this is gonna be the central UI hub where your operators get this real time visibility across the entire infrastructure. So what we're doing here is.12:19Instead of jumping between, a static network diagram and CLI outputs, your active topology gives you a dynamic living view of the network. So you're gonna be able to toggle between a graphical map or like a logical diagram depending on like your, your actual preference. Now this URL.12:37Is gonna be completely interactive. So if you click on a node, it's gonna drill down into that device specific parameters and you'll be able to see things like router IDs and IP addresses. Now, if you click on a link, it's gonna reveal reveal the exact interface defect details, like your IP addresses or your IP allocations and your live traffic util.12:57More importantly than both of those is it's not just gonna show you the physical boxes. It actually maps the logical service layer directly over the physical layer. So us as operators can see exactly where you know our SRTE policies and our RSVP.13:21The TE tunnels are routed and they can actually visualize this end to end VPN services alongside the participating PE routers and the bound LSPs. So basically it bridges the the gap between the raw infrastructure and the customer facing services into the single pane of glass.13:38Moving on to our crosswork optimization engine, and this is one where I mentioned this SRPC will handle and work in conjunction with. So this SRPC is gonna partner with this crosswork optimization engine or the COE, and it's there to handle the real time bandwidth optimization. So I want you to think of COE.13:57As our like our dedicated brain for anything.14:17Related to bandwidth allocation or traffic engineering. And when you deploy a bandwidth on demand service, let's use an example of like an SRT policy that's gonna require a dynamic path with a, let's say a strict 10 mbps of guaranteed.14:21Bandwidth. COE and your SRPC are gonna work in tandem, and they're gonna analyze the current network to network network utilization. They're gonna compute a path that guarantees the bandwidth and then they're gonna assign it to the policy.14:41Now, if the network conditions change and that path can NO longer assure the requested bandwidth, COE automatically recalculates a new compliant path in real time. And to make this magic happen.14:56The controller is gonna need absolute visibility, which is what we're achieving through our two data streams. Remember we've got our CDG and our SRPCE. So we're gonna be viewing our topology in our link state, and this is gonna be delivered via the IGP and the BGPLS to map the physical network.15:12We are also gonna need our traffic insights. So we're gonna gain the real time interface statistics, and these are gonna be collected via your crosswork data gateway, and we're gonna use that to monitor the actual utilization. So if we look here on the left, I'm gonna walk through this actual kind of workflow of.15:32This kind of like provisioning a service and then recalculating what would happen if a path changes. So starting with step one here, this is where we're gonna define the intent. So us as the operator, we've got a question here. Is it possible to integrate CNC topology and crosswork assurance topology? I missed that.15:53I'm gonna let one of you guys handle that while I finish up this slide if that's ok. Appreciate it. Yep, ok. So step one, the operator creates an SRTE policy with a specific bandwidth constraint.16:13Step two, NSO pushes this policy configuration down to the head end router. Step three PCE delegation. Because this policy specifies a dynamic path, the head end delegates the path computation to the SRPCE.16:29Then we want to move on to step four. Because we recognize that a bandwidth constraint or that there is a bandwidth constraint constraint, your SRPC is going to consult our COE, our crosswork optimization engine, which is going to calculate a traffic engineer path that fits the requirement.16:46Step five, the SRPC will deliver this computed path to the head end and you know traffic will be begin flowing. Now six is where we'll throw a loop into it. Let's say that link fails or a student traffic spike occurs, the network is gonna alert the SRPC, which is instantly.17:04Signals the COE, which is gonna recompute this path that it made.17:24Step seven, the SRPC will then push this newly calculated healthy path to the head end router to maximize downtime. Okay moving on to the EMF network inventory.17:29So, the EMF or the element management function is going to be a new module inside the crossword network controller, and this is what allows CNC to act as the traditional fcaps compliant element manager alongside its role as your SDN transport controller. So.17:44While components like NSO handle this high level service orchestration, your EMF handles the day to day device life cycle management in this unified multi vendor environment.18:04You see, are you all able to see those questions that are coming in? No, when when you finish yours, I'll read out the question and then you can both of you can answer it. Okay, that sounds good. All right.18:19So the core functions of this EMF is we've got our fault management. So this centralized, this centralizes the real time device alarms, syslogs, and network events into a single topology map view. And this is going to drastically reduce the, that time to detect your infrastructure failures. We've got configuration and software management, so if you think about.18:36That's your swim. So this manages golden configurations, it takes routine configuration backups and it automates your software image management.18:55Or SMU patches across your devices. You've got your accounting inventory and management, so basically with this module, you'll be able to discover your deep device hardware details, so things like line cards, optics, serial numbers, chassis state, this is gonna be both for Cisco and 3rd party.19:05Boxes. We also got your performance management, so it collects device level health, telemetry, things like CPU memory, interface status, and this uses to establish this trend and will trigger alerts when operational thresholds are being breached.19:23The final use case I'm gonna cover today is the service health, and service health in CNC is how we shift from reactive troubleshooting to proactive assurance. So, instead of just checking if an interface is up or down.19:41We're gonna use the IETF RFC 9417 to map the entire context of a service. So CNC combines this multi layer infrastructure metrics with a active probe testing to give operators like a, a true service centric health score.19:57So, what do you think about like this? If a service degrades, operators don't hunt through logs anymore. They're going to use this assurance graph to visually trace dependentries straight to the root cause and then they can use the the timeline view to see exactly when this degradation started or if it's a reoccurring pattern.20:16So if you look in here, if we got like an example of an L two VPN, we're gonna break down what makes an L two VPN path reachability, path SLA, device health, interface health, IP, and we're actually gonna monitor these individual pieces and parts. When you monitor these, you'll actually see isolation.20:36Issues, you'll see where the issue occurs in isolation. It might not be anything over here. It might be just this portion which causes that causes the actual degradation in your L two VPN. And we'll be able to set up SLAs or basically a way to monitor and you'll have different phases like maybe you'll have a gold standard and a silver standard, and you'll be able to monitor that in real time.20:53All right, so that wraps up the CNC. I'll be handed off to Rajesh here soon. Yeah, so let me let me read out the question that has asked and Angie wants to know, is it possible to integrate CNC topology and crosswork assurance topology for end to end topology view?21:15So if the issue is at PC or transport or compute et cetera, I'll take that yes it is possible and I will be showing that later on when we're going through the topology, you'll be able to see the the SLA and the probes being displayed as.21:35But as part of the topology. Right, so let me go ahead and get into the There is one more Rajesh is from Tony. Where is the Q and A being handled? I guess that's what the CNC.21:55Yeah, so Tony, you can just place it in chat or if you've got a question you'd like to ask, got loud, we could probably unmute you and have you ask that if you've got one. So I'll go over a brief overview of the workflow manager and.22:13Like Robert mentioned, we'll go over the workflow manager and then Walter will go over a use of of that and along with crosswork assurance. So crosswork workflow.22:31Manager really is.22:51Is a tool available as part of CNC or or even standalone, but with this tool, you will be able to create workflows and and integrate with multiple OSSBS system. This is for planning.22:55And executing change or automation within network. You know, all of us who have worked with service providers, they have mops and then mops have multiple steps requiring are basically workflows. So.23:14And, 1st, rather than having to gather information separately in advance, you can just build it right into your workflow, and then a workflow can be defined to provide dialogues and notifications and operator guide and.23:30Along with.23:52User control. So, as you can see on on this picture, it, the workflow, we, you know, one way to think of it is, is that it it provides you with East west integration, you know, like how we, we, we do.23:55North south integration as in like a controller is talking to devices that it controls as north south and east west integration is where you're, you're moving a particular task along different stages at that level.24:13Right, that's and.24:33So workflow manager allows you to do that. And then the idea for workflow manager is to do to be able to pause and resume workflows right that's one of the big ones. And then.24:36What CWM has with crosswork workflow Manager has created is a library of pre built use cases that you can use. We'll, we'll get into each of this as we go on, and go and.24:55Further on this. So hold on, why is this not ok there you go. So a way to look at the hang on I don't want to go into this as a little extra information this is details of what, what I just mentioned.25:14I'd rather look at this picture is is better. So here, you know, this is how, if, if you haven't worked with workflows a lot, then this is this is what a workflow is, is, you know, a scheduled prebuilt mop.25:34You collect inputs, run prechecks, then execute on either the devices that or or the your execution might involve sending.25:51Messages to different systems, right? And then, and then take the take the output of those those actions and then take.26:07Appropriate action, right? And then human interaction is as needed, you know, approval of a approver in case of a mock and then finally verify. So that that's like an overall, that that's how.26:22A workflow is supposed to, you know, you know, you can abstract it. We, we also had had another tool called BPA if you have come across it, it it did the same thing provided.26:40A workflow automation and with this crosswork workflow manager functionality, we have now combined it within this crosswork platform.26:58So, you know, as Robert was explaining, the crosswork platform has multiple multiple functionalities within it, right? You have the element manager function, right? Then there is, we didn't go over the crosswork planning, but it also.27:13So part of it. So in the same way they've.27:33They'll build the workflow manager inside it also so now this this this one's like a more detailed.27:38Components of of the workflow manager and it lists the features and and various parts and what I would like to focus your attention to the center here. And as you see.27:54That, how the workflow manager is the, is at the center of integration and orchestration of other orchestrators like the NSO or or you, you receive are you receiving.28:10Events from other controllers or you know CNC or you could be even be talking to HCO. So, so that's.28:30That that's basically where CWM is helping out. Now this is this is where this picture we can look at that crosswork workflow manager has really.28:45Three parts. You can look at it. One, there's this left left hand side is the UI to which with which we the users interact with it, right? That's part one. Now we have the access to create a workflow, look at the status et cetera, right? It also has the event hand.29:02And then, then another important part is the workflow catalog, and then what's in the catalog? Catalog has pre built workflows, and any workflow that you create, you can save in your catalog to.29:22Use later later on, and create create your custom workflows you know, so, so as to enhance your particular function you know use case. And then 3rd part is the adapter catalog.29:38And because.29:58As part of, you know, getting out your workflow activities, you're gonna need to talk to different systems. Like in case of your if you're managing a bunch of devices, you're do you're gonna go do that through NSO.30:01So, so that's why you need a NSO adapter.30:20Which is which is already available in in workflow manager. And similar, in similar ways, there are other adapters available and also CWM creates has given a way to create new adapters.30:24Yeah, you know, very quickly and in a very simplified manner. You know, that that's the other additional value with you know that that we brought. So here in this one, I want to show there's a, that's a little illustration where you.30:42We'll see how, before I start the illustration, I'm just gonna set the stage.31:02So, to create a workflow, you have a, you know, you can you can you know there's a little start button and then from start, you can have a decision block block and in decision block you're over here you're doing dry run.31:09And then, then you're taking in taking a decision based on that, right? And then, then, then you are doing a bunch of other things. And now let's let's go into the illustration. See how it'll go to the two box.31:26Find a operation called the parallel with with with which you you're now going to be doing two parallel actions together like one of them is sending an email.31:44And, and another would be sending a Webex. And this is a, a super simple example of what you can, what you can do. And.32:01So if we if we if we look at some details of each, each workflow, though, the workflow builder provides you with a view where you can.32:19Your graphical workflow which is like here on the right hand side side by side with the, with the code, right? And the code maybe, you know, sometimes if if you are, if you're not.32:38And if you don't like it, then don't let this, you know, bother you or overalm you.32:58Because the idea of, Cisco work for crosswork workflow manager is to minimize the use of any coding and we can.33:06All accomplished almost everything through these through the buttons and decision blocks etc. And if even if that wasn't enough.33:23There is this other now, we have a oh before I start this illustration, I'll set the stage. So now, now we have with, we can provide.33:37A prompt with details. Now it's the the prompt string maybe maybe it's a little too small for you to see, but, but it's it's saying that you know CLI net to run CLI command on XR device.33:53With following NSO Restcon, and then what's the sample run should look like is and you and you you describe it, this thing and GitHub Copilot will create a complete workflow.34:13And, and that is right now available. So, so it's basically any of this that earlier a lot of this was like a, a limiting factor, like people would say that, hey, you know, yeah, I understand that there is you know this.34:28Everybody exists or.34:48But we don't want to, we don't have developers in our team or developers Well, you don't need them anymore. You can do use use use this. So and then just just some recent change.34:50Just for today's session, I wanted to highlight the what what has happened is that in the last, last couple of updates is the.35:10Availability of crosswork assurance will go in details of of this. But this is now available as as a, as the entity that provides events that can be managed through.35:18The workflow manager. And, you know, we talked about the pre built work use cases and everything. I wanna keep some time for the actual actual workflow manager on the, so that's why I'm just gonna going through this fast.35:38But so there's, this, this feature of tagging, workflow tagging is, you know, you created your workflow and then you provide a tag and that's like then then it becomes easier to, to search, easier to.35:57Find the execution status. Also, you can then refer that into your new workflows that you create.36:15Where you want to recreate or where you want to reuse for the right so this and I mentioned BPA earlier. BPA used to have this feature also. So, and also if you are, if you are a AWS familiar with that, they also have.36:30Tagging extensively used in various of that. So, and that job tracking on workflow manager is how you can you are going to be able to see the status of your of your workflow j.36:51Job. As soon as you create a job, it goes into the, the.37:09The list of jobs and as as soon as you like click run on a, on your workflow job, and then then you can see where you know the status of, you know, whether it completed or it, it it failed or how far it progressed, right? Also.37:14If you have scheduled a job, right? Like, because you could you have the ability to, to run a job in in future, right? Or or or it on a cadence, like cron job would.37:32Right? So though those those are available features available to you and then the wanted to highlight the pre built use cases that what the big one is the fleet fleet upgrade.37:48Now what fleet upgrade, is, is where it because this is a common need across customers, right? Like you have your, let's say your XR devices and then they need to go to the next, yeah.38:07And it it it becomes such a massive.38:27Task if you have a large service provider as your customer. So you have a pre built workflow for free fleet upgrade, all of your XR device you can upgrade using this.38:31With a single action that you can monitor. So that and then then golden config is is a very important use case. Every customer has their need for this device migration, and then customizing the the mop builder custom, mop builder.38:48Is another facility that that is very useful.39:08Which is, part of the workflow manager. Here the fleet upgrade performance, some numbers like they had on a single VM.39:15They've done hundred parallel device upgrades and they achieved 200 upgrades in 4 h. And there's some other, this is just an example of a data that I picked up from some of.39:30Some of data published by BU, so, and, and the devices supported for prebuilt are these. You have the XR and XE and and the iOS.39:49So, the availability of memory, you know, for a standalone or a cluster deployment depending on the requirement. So I want to.40:06Go and quickly show a live hang on a second. I have to Yeah I want to show.40:24So I'm logged into the CNC here.40:57We will look at the workflow manager within it. So here's workflow automation and here's workflows, right? And, here, here are my test workflows. I want to quickly show you this.41:00Show config workflow. See how it's it's this the simplest possible workflow, you know, you start and then decision and then there's one action you're taking and the action you're taking.41:21Is on, on, the action is show config and you're taking that on a device XR 9KR1, ok? That's, and that's being managed by.41:36NSO controller. So that's why this resource NSO and then and as you can see on the right hand side here NSO. So, so and then it all it's doing is doing a show config.41:57Okay, and, the split that I talked about is it kind of looks like this, the code it you know kind of look like this, it's got the, the show config in here which we wish to do.42:11The name of the device and the resource as in the adapter, ok? It's on here. So now I'm gonna quickly run this. I just do run job and then I go to job details.42:28And then, it's gonna your 1043, this is the one that just finished and here's the results I got. I got the show config of the device. Rajesh, how did you go ahead mention a time check, we need to move on to the next topic. All right, ok, cool. All right. So, so that's the.42:45And I'll now give it to Walter to get into the crosswork. Alright, so Crosswork assurance, we're gonna start quickly and I'm gonna I'm gonna be playing with some different things in here.43:05Seeing some of the the real CNC interacting with the network the customer consurance platform that we have, the analytics. So definitions, I'm not gonna go through definitions right now. The majority of you know that.43:23Crosswork assurance is network assurance platform. It uses sensors or probes to measure service performance crosswork assurance places measurements points in the network to check how the service is performing, and it measures the quality of the service path himself.43:43It can also send results to CNC, which is what we're gonna see soon. Okay, so.44:01Why bring them together? This is we're gonna start with. Think of it as by himself CNC can create a service and track his operational state and crosswalk assurance can measure network performance. But when we integrate both of them together.44:16Willing service intent with measured reality and that is the value. This is the foundation for automation based on real performance. After you create, the service.44:36CNC can receive health evidence from password insurance and an operator can set, it cannot it can only see the LTVPN that exists but also whether it is meeting the expectations. Let's get into the real story.44:52If you need to remember one slide, I want you to remember this one. This is very important. And I go I'm gonna go through the steps very quick. And so step one is connect CNC to Crosswork assurance. CNC.45:17Needs a trusted connection to crosswork assurance.45:29Step two, create or manage the service in CNC. CNC creates or track a service such as L three VPN, and this is very important because you might not need to create an L three VPN to the customer to add the assurance process, which is step three.45:34You might have a service already established, so we'll see that option also in a moment. So step three, you attach the probes to measure the service. The props you know are designated to do a service test and is continuously 2047.45:51Step number four, stream probe metrics back to CNC, so we need to bring all that flow back to CNC. Apply monitoring to logic and thresholds. That means CNC is gonna use that logic that it has and we're gonna also take a look at it in a moment, and see.46:09Six and seven is used to results for dashboard seven alarms and automation. So CNC is gonna show the health using the UI and it also can trigger alarms or actions which are gonna be picked up by what we're just we're just explaining you, the trustword workflow.46:28So the story is simple. Create the service, measure the service, understand the service, and improve it. Now, this is a very quick.46:49The slide and I want to bring it in because this is a provisional life. Onboarding, integration, activation, and assurance. These are the four steps, and the last one that I have at the bottom is, is circular float. And they're all involved onboarding is you gotta have your credentials to connect to Crosswalk assurance. Integrations. There's all sort of circu.47:06Certificates that you need and credentials, activation is having your LTVPN and putting them together with the probes, assurances start doing the measurements, ok? And then that's the flow.47:26Okay, so let's get into something a little bit more tangible now. We have, the following picture shows all the components that need to work together in the cross work assurance solution.47:40And this is independent to CNC and this is basically how Crosswork Assurance works. It's not a very complicated solution to understand, so please bear with me step by step, and if you come to a situation as I've been there that they bring you in to explain something about we're all exceedings and something about an exceeding.47:56You need to understand the situation, so I'm giving you the the simple picture. The model I'm showing represents a service provided network where T one sessions are established between P1P2P3 and P four. These sessions indicated in indicated in the blue dotted line.48:18Right, which I'm gonna start doing the motion in here.48:37Okay, is utilizing container based software sensors. So these are the software based sensors. And by the way, we also have the flexibility to deploy hardware sensors.48:42You know the sfps modules which are part of the Crosswork assurance sensor portfolio. Okay, so so far so good, it's all clear. Now there is a management connectivity line that go into what is referred as the sensor collector. So let me bring it in here.49:02The collector is a proxy that allows agents to communicate back to the main control element which is the crosswalk analytics platform. And this is done over a TCP port 443, so you can punch through firewalls.49:22And run on prem and in the cloud if you need to. And you'll see it in the far right. The analytics platform uses SSL tunnel to allow users to reach the system via.49:36Web interface or be a rest rest interface.49:56I will be using the Advantage analytics license, and I will show you that in a second. Here's the last part, ok? I will be using the advantage analytics license package, and I'll show you in a second, and they're both on prem.50:01It can be used to store the the advantage can be used to store all the data and all kind of analytics and also visualize or the visualization and reports. But in some cases you might wanna use the lighted version of Crosswork assurance, with the essential license package and, and the.50:20Can be used with the CNC integration. So I have, I have both of them.50:39In a moment I'm gonna show them to you. The CNC uses the restconf interface to create the sessions and consume telemetry through Genemi to get the status of the T one sessions. In terms of how we configure the sessions, we're going to drive everything to the.50:44LTVPN payload that CNC gets from the, from the VPN.51:04Okay, so I'm gonna go quick to this because I need to show you some, some live demo so what does CNC needs to do before we start the integration? Okay, so.51:08It's gotta have, ok, it's gonna have all the certificates and credentials that are provide to access the system. This certificates and credentials.51:24They're used to configure the objects, ok? And, and these are gonna give you This certificates will allow you to answer the questions, who is the provider? How do I trust it? How do I authenticate to it? And what service do I want to monitor?51:41These are the four questions that it's gonna answer by doing all these certifications that we're gonna do in a moment. They look a bit complex, but as you do one and you do it only once, you'll see that it's very simple. Okay.51:57So you get your certificate and I'm gonna go to a live system now and navigate through this. Let me log into the CNC.52:12Okay, so we're gonna certificate management and this is the 1st thing you gotta do and all these pieces of the puzzle that I'm giving you are gonna come from the team of Crosswork assurance exceeding team when you get your, your analytics.52:32Platform. So I'm gonna go to.52:52The 1st part, which is administration and I'm gonna go to certification management, which is right here. And you're gonna see a PCA. I built this a while back, so I called it a PCA in here, ok? And you see your expiration dates and all the details in here. So quickly I click on the update certificates and you're gonna put this one time.52:55Only one time, you're gonna get your certificate name, which is PCA, whatever you wanna call it, your certification role which is already in there for you to select. And the team from Crosswork Assurance will give you your CA certificate, ok? And your CA certificate is basically.53:14The CNC can trust you. It knows who you are, ok? And this is this is given to you by the provider's certificates, a provider certificate provider. Then you have your client and your key, which are pretty much your, if you're familiar with the this is the collector keys, you know, the, the.53:34We used to call it the runner, so you're gonna have those kids and they're gonna give it to you and you just gonna drag them in here. You're gonna select save and.53:55Your certificates are good. Now if it doesn't work, there's a lot of ways to troubleshoot this. As I said, this is pretty straightforward. I've done it a few times and I haven't had any issues. Okay, we move into.54:03Once you have your certification, you move into the credential profile. So in simple language, this is the login identity. This is what you log in. CNC needs a reusable credential object so you can authenticate to Crosswork assurance, for both.54:18API style access and metric streaming. So he needs this. So he trust is handled by the certificates. Login is handled by the credential profiles. Let me show you quickly the credential profiles. Again, this is very simple.54:38Credential profiles, and I'm gonna select the PCA that I have already added in here. And again, you're gonna have all this information coming. Now, my login in here is pretty much the login that I used on my.54:59Advantage. This is my advantage as you know analytics. Okay, and if I log out from here, I'm gonna do it quickly, ok? And I try to log in back in. Here's my user. This is the user that I'm using right now.55:14And the same password that I'm putting in here, ok? Alright.55:32Back in here. Username password, the same thing for Gemini, ok? Once this is done, so you got your certificates, now you got your login and here comes the 3rd part. You have to have a.55:49Provider, access and I'll show you in a second the provider access CNC, this is where it defines the external assurance system. You tell CNC the provider name, which is.56:08The PCA or whatever you call it, which credentials profile, which certificate profile, the HTTPS and the Geminian connection that we just I just saw you and in plain English, this, this setup answers the question.56:24Which insurance system I am talking to and what do I need in order to talk to it security? So, last part that I'll show you, and then we try to move into a demo, your provider.56:39I select the provider. Obviously if I'm not if i've creating a new one I'm gonna click on plus, but since I already built it, I'm just gonna edit it. And you can see in here the provider name was PCA credential profiles is called PCA. There is the family exedium proxy, which is by default.56:55Fault you will select it in here. The credit the credential profile, it doesn't show it in here, but you have it. And.57:15The IP address of my, of my analytics. Now, if I look in my other crosswork, ok, I have another one here and this is connected to.57:22Analytics essential. The reason I have both of them is because I do some demos and some testing, so I'd like to work with both of them. Okay, now.57:39I want to show you the difference in here, not because this is essential, it's just a different access to my analytics. I have a DNS in here. So if I go to administration and I go to provider, you're gonna see in here.57:59That I'm using an FQDNA. And over here, I'm using just the IP address, ok? I wanted to show you the difference. So this is pCA.exceeding.net analytics.something, ok? And the information.58:18You have in here is pretty standard for 43 and for the Gemini you need to use the encoding tool. So this is very much a standard, there is nothing complicated, but you need to do these steps. All right.58:36Let's move to the next 1L3 VPN, so we're gonna talk about the LTVPN and I'm gonna build the LTVPN. Okay, so this is exactly what assurance matters. An LTVPN maybe configured correctly and still.58:50Will performs poorly.59:09L three VPN, you know what it means, you know, layer three virtual private network, it securely connects customers sites over a sure provided network. So to the customer, it feels like one private network and it is used normally for branch to branch branch to a data center and cloud.59:11Auto connectivity. Users my experience delay, jitter or packet loss CNC with Crosswork Assurance helped operators.59:31Prove the health of the L three VPN and again I'm using that word health because health is gonna come a lot when I do the demo. Alright, so let's build one. Alright, so I'm looking at the.59:41This CNC, which has got the call, the workflow automation. I'm gonna go straight and I don't wanna, you know, there's a lot of menus in here and a bit of confusion when you're not comfortable with the CNC. I'm gonna go straight to provisioning and the NSO, ok? So there's nothing provisioned here as you can see.59:59So we're gonna provision an L three VP, an L three VPN called Rannie.01:00:18I'll let it finish in here. This is the only thing and when you go live on demos, you know, there are some delays that come in, so there you go. So we're gonna go on provision and I'm gonna do this as quick as possible.01:00:23Okay, so we select the workflow. I have already built the workflow that I want to use, ok? Let me bring it on my screen.01:00:39Okay, I put it down here and ok there you go. This is the one I deployed today. As Rajesh was explaining you, you know, this is this is the beauty of this of the workflow automation. You can provision this and you'll.01:00:59See it in a few, in about a minute, a minute and a half. If you were gonna do this manually as i've done it before, it will take you up to 45 min and you need to understand all the network. So the 1st part of this code that I selected in here is just gonna load all the modules that I need to make this call to tell them, look, I'm building an LTVPN.01:01:19This is the name of it and I'm gonna have some more equipment coming in so make sure that all the workflow IDs that I need are provided for this.01:01:39So quickly, I'm gonna take the ones I need, which are here, and I'll show you what this is.01:01:47Okay, so we're ready to run and what I'm going to run is an L three VPN that has got the following components. It's got router one, router two, router three, and there is a PC in there. We're not gonna concern about this. We're concerned about this three.01:02:03In green, you see your agents, software agents. There could be a switch in here, ok? I just put a direct connection to the router. And by the way, these are real routers, virtual, but real routers, ok? Each one of them has an IP address.01:02:23This agents and the router has his management IP address and the interface, so I wanna I want you to concentrate for the moment on this. This is the interface that is facing the customer. So this could be a switching here and all this traffic is coming to this customer, it needs to be routed to this network. So let's go in here.01:02:43And push the 1st thing. So I'm gonna send this let me do a quick refresh. As Robert mentioned to you, it's important that the routers.01:03:00I synchronize with the system. So here are the four routers I just show you. I'm just gonna do a sync from. So i'm taking all the information the routers have now bringing it here to synchronize it, ok?01:03:17The reason I'm doing it also is because we do a few changes sometimes. So I just want to make sure that everything is up to date, and I believe we're good on time now. So I'm gonna I'm gonna slow down a bit.01:03:36So I am boarded all these routers through NSO, and then I can see them here on the network devices. Obviously there's other ways to onboard this router that you don't have to bring one by one.01:03:51You just bring the main, let's call it the main router or the the controller router and it will just import everything in here. Okay, so they're all being synchronized. I just wanted to make sure that everything is ok in there. I'll go back to my workflow.01:04:10And I had my, this one. Those are the workflows that are by default created and some of them are created by us. Okay, I'm looking at the code again.01:04:27And I'm gonna send this information. So, the workflow is gonna ask me, ok, so what do you want to input? You just call all these modules. I'm just gonna paste here what I wanna input, ok? Which is the router information.01:04:43Put it in here quick. So this is what I'm gonna import. Each router, and what is important for me now is what interface is facing the customer, ok? And in my case I'm gonna be adding the pros later on on this.01:05:02I'll show you how to do it. If everything could be added in one shot. So I'm gonna run this job.01:05:22Might have an error, so that's ok. There's some changes that i've done before and we see that it's deployed, ok? So the L three VPN has been deployed, so I need to go and see it now and we'll see that it's probably in provisioning right now.01:05:29It's called Rannie, it's in there. So this is the point when you come and see a customer or or or someone from Cisco bring you says, hey, you're the Cisco, you're the Axian expert. Can you just tell us how we put, you know, crosswork assurance to do some performance measurements on this L three VPN that we have in there?01:05:47So either you build it or you're coming in to add this service, ok? I'll be synchronized in a moment. Once we have it green, I'll proceed to do the the probes, and I'm gonna do this manually, so let me put the picture in here.01:06:06Very quickly. So I'm gonna add this pros in here.01:06:25And the router knows that the connect the connection between these two is via this interface Cause that is being provisioned already. Okay, live network. There you go, just turn green.01:06:30So this is actually gone through NSO, ok? Send those information that I provisioned, put it on the router, and now it's ready.01:06:51That's what it is, that's what he's done. So now we're gonna add the proofs. Obviously we're not adding the proofs to the routers, we are in the proofs to the CNC and we're gonna operate them from here and they're gonna take that path that we selected. Okay, so let's go into edit.01:07:04And this is, this is the part that I was mentioning to you. You can build all this manually, go one by one and create everything, ok? Or just use, as I show you the and and Rajesh explained you, the.01:07:21Workflow manager. It's easier, simpler. So i'm coming here to the proofs. This is the exceeding part of this is the crosswalk assurance part. I'm gonna create this one right here. I'm gonna call it AG one.01:07:37So we can identify it, ok? AG one has to be at, sorry. AG one. There you go. Continue.01:07:52So what do I need from here? It's attached to router one, ok? So when by the time I do AG three it's gonna be attached to router three, which is all the LTVPN configuration I just sent. Okay, there is a network access that I built for that.01:08:10And it recognizes right away because I'm working with only router one. There is also the interface, very important. What is the interface that I'm using that is facing this L three VPN or or the, let's call it VRF Rannie, if you know what VRS it is. What was the IP address? Five.five.five?01:08:28That 20 is NO longer the interface, is the actually IP address of the probe, and this probe can access.01:08:48This device and this device here that can do aT1. Remember we did a little T one bit of graph. Okay, and I'm using VLAN one because it's a native VLAN. I'm not complicating myself with the VLANs today. Okay, so the 1st one is done. Let's do another one.01:08:58Quickly. AG two. Continue. Okay, so this is attached to what? Router two. Okay, I refresh the screen so I can get my VPN access that I built to it and I refresh again so I can get my.01:09:13Interface so I know they're all facing interface one and this is gonna be six.six.6.twenty. Okay.01:09:33Native VLAN. One more AG three. Router three.01:09:42Refresh. Here is the attachment to the router refresh one more time, and here is my interface, ok? And again, seven.seven.7.twenty. Obviously, I have connectivity on this network and not building the connectivity. I don't want you to imagine that, oh, you can ping this without.01:10:00Building the LTVPN and NO NO NO NO NO. I could ping this system with NO issue. So now my diagram is this. What am I missing in here? I'm missing building everything on my analytics platform.01:10:19I already built it, but you have to keep that in mind. This is your staging. You need to come and says, ok, I'm using this probes, one, two, and three, ok? And they're running on a Linux machine, which is user docker compose. And that's the those are the ones that I.01:10:38Gonna generate the T one, so here they are already built one, two, and three. What is the connection with the CNC? CNC is gonna talk to this.01:10:58Analytics through something that is called and I don't know if it's called mystru. Okay, it's a module in there that use restconf, communication, and it's gonna talk to him and it's gonna tell him, build me this sessions. So I'm gonna proceed.01:11:08We have nothing on session one, we have nothing on three and we have nothing on two. Okay, so I'm gonna proceed and says, ok, I'd like this. I'm happy with this configuration. I got my three probes. I now need to tell them is it point to point? Is it a hot spoke or is a mesh? I'm gonna use a mesh because it's quicker for me.01:11:26Need to do it. Point to point I'm gonna have to select AG one goes to AG two and so on. So I'm not gonna do that, ok? I'm gonna make them go mesh.01:11:46And it's quicker. Okay, so dry run, you know what dry run means? It's gonna show me what I built, and this is what I built in here. Okay, I can save it, I can use this to create a template as Rajesh was explaining you, send it to circuit and tell.01:11:55Then give me a.01:12:15You know, JSON file for this. Okay, so I'm gonna commit the changes and the changes are now committed. Now it's quick. So what do I have now here? I'm going to go, and I'm looking at the time so I'm gonna go quicker, ok, to my.01:12:18VPN service and I'm gonna start this. The service is there now. It wasn't there before, now it's there, ok? And I'm gonna start it right away and I'm gonna, in a moment I'm gonna come back here where the service starts, take some time, start monitoring.01:12:37I'm going to use and i'm coming back to this screen in a moment, advanced monitoring, and I will explain you why. And there is silver and gold. I'm gonna use silver. And I'm gonna start the monitoring right away.01:12:58Okay, because this is gonna take some time. I want to start it now and I'm gonna come back to the presentation. We'll let this go for the moment. It's initiating. So it's gonna take some time. Let's go back to the presentation quickly. I'm going to the next.01:13:12Slide which is understanding prove functionality. Okay, the word probe can sound complex, but the idea is simple. A probe is just a sensor used to test the service path. One side.01:13:29Says the test the test stream and the other receives or reflects it. The system calculates performance values, so this is what we're all talking about. The probes continuously test the service and send measurements back into assurance workflow.01:13:44What does the advanced monitoring mean? Okay, so here we go. I'm gonna go back to that little screen that I told you before. To put it in simple terms advanced monitoring evaluates service health by gathering a much larger pool of edits.01:14:08And applying far more intelligent logic to the data. E.g., advanced monitor can incorporate probessions, and I'll show you the results, richer service health rules, which means operators get more meaningful answers than just red.01:14:22Or green.01:14:41They get a better understanding of why a service is degraded. So let's go quickly on the demo and we're gonna go to this one. Well, I'm not gonna disturb this one is running right now. I'm going to go to VPN service. This is the other CNC I have, ok?01:14:43Now I'm gonna show you this one too. This one has got, 3L3 VPN services. One is called Bell Quebec, the other one's called Videotron. I'm gonna take this one that is not doing anything and I'm gonna start the monitoring. And here is what I were talking about. If I use the basic monitor, this is.01:15:02The information that I get, ok, the KPIs, they call the simple KPIs CPU threshold memory, and I'm not even I don't have the probes in there. So I need to go into advanced. Advanced will give me probe delay, probe variance and packet loss. So I'm looking at delay, jitter or you can call variance too, and.01:15:22Packet loss and this is bi directional. I'm using gold. Gold is higher on when it comes to, to the metrics. So gold saying 1 % packet loss is a problem in here.01:15:42Okay. And then you have the other metrics that comes with the routers. Packet errors, the threshold, packets that have been discarded, CPU threshold, memory, SRPM, segment routing performance metrics, delayed threshold, and the same thing for variance, so you guys.01:15:53All this information in advance. If I put it in silver, you'll see that.01:16:13It's a little bit different. Now packet loss instead of being 1 % is 2 %. So these are templates that we have. Can you change these templates? Yes, you can. You can go into the system and create your own template and call it gold verizon, let's say L three VPN and put the values that.01:16:18You wanna place it there for your customer and you will start the monitoring obviously. Okay, so let's take a quick look and how is this doing? It's still it's integrated state right now, but you know what? Let's take a look.01:16:34At the sessions. So now we have two sessions. Two sessions are being provisioned and it was done all through CNC, ok? Let's go to the other sensor agent. We have one session running, so I believe we have three sessions running.01:16:50Correct. So only three. And if I go to inventory, the sessions are here. They probably have not started, but I'm interested in seeing the metadata. So I do have the name IDs, which is important, ok? And this is how the service is tied from.01:17:06Analytics to the probe and the router, ok? So this is how we tie all this service and is going from end to end. So we tested end to end service. Remember that?01:17:25Okay, same thing goes here end to end service and same thing goes here, so that's three sessions we got. Okay, if I wanted to show you the analytics that for me is.01:17:43Essential, this is the analytics essential. Okay, it's tied to this CNC that has got the bell, ok? I can see here, let me see, I got ten more minutes. Okay I can see in here the graph that somebody asked before. Can you, can you put all this together? Yes, you could.01:18:02And I'm showing it to you right now. This is the topology graph that we have, and you have that L two VPN called Bell Quebec. And at the moment I'm only showing you.01:18:22The part SLA. If I click on it, you'll see that there is three sessions running in here, C, B and D, ok? And there's telling me there's some degradation. I know what the degradation is. We'll see it in another, in another one better in a moment, but I will show you this and I'm not collecting this.01:18:33Information. Unfortunately, I didn't put this because I didn't want to use it on this demo where you get the router information. This is router D, this is router C and this is router B And each one has information about the BRF health, interface health, BGP health, and the PC the PCPC health computation.01:18:53Protocol, health. You have everything in here. Right now you got it in blue $3 is because it's not collecting anything. This is telling me it is working and it's ok.01:19:13Going back in here, I'll take this one from Videotron, and as you would see, I'm gonna click on it so you can see. It's only two routers. That's all. This one you have four routers, ok, you only saw three because one is a PC.01:19:24Okay, so two routers in here. I'm going back into.01:19:43Assurance graph. And again, I get the whole picture. I'm interested on seeing my videotronot VPN, my assurance graph, and I see it's been there's a degradation in there and I'll show you how to see it and how to understand it. Again, my routers are not giving me any information. I have two routers, router one and router two, I can again click on the.01:19:48Plus side and if it was collected with SNMP and all the telemetry information for the router, I will just do plus plus in here and I can drill in on the health of the router. So an operator can see.01:20:07Your probes performance or your SLA and you can see also how the routers are behaving from this graph. And usually people go quickly to this picture. Now there's more to do in here. I can go straight to the measurements.01:20:22View details, processions, and I can actually open it from here and see my historical data. So I'm seeing today, let me enlarge this. I'm seeing today all the data that has been running, you know, I got the threshold for those of you that like to see these graphs like this.01:20:37This is what the the the dashboard comes from, from CNC. And remember this one is showing degradation so look at the packet loss. So.01:20:56Sorry, prove lost forward, prove loss reverse, so uplink, downlink, we got 2 % a threshold and we're going up to 5 %. So this system is always showing that it's degraded.01:21:07Okay, so let's go back to the one you just set up. We don't have a lot of time. Yeah, we I mean if you if you could stay a bit longer I will stay a bit longer, five more minutes if it's necessary. Let me close this with, if you.01:21:24This is the one we just built. There's three processes. I know I'm gonna get an error right now. This is because they're still processing, but I should be ok once I reactivate the probes. Okay, so the probes are being reactivated. One more second and let's see if we got any info right now.01:21:40Let's go to the assurance graph, three routers, your SLA path, and they're all clean, they're all good. You have a degrade because this is not getting any information, ok? And.01:22:02Just to close this up, let's see if we can get some view details. So you have two options. I'm gonna click again here so the error doesn't come in. You can click just right here and you'll see quickly.01:22:21Your KPIs, your information, your measurements or you can just bring it up there again into view details and historical data and you're gonna see, the problem active, ok? So close it up. I should be able to see in here something. We're gonna open it.01:22:39And we're gonna put it in 1 h, we still haven't got anything yet. So information will be coming in a few minutes. Okay, there you go. You said, it started to come in and it will come in for all the three different probes. Alright, so.01:22:57I would say I'm pretty much done. I want to close this up with telling you something. We were supposed to present today, and that was our intent, ok? The closed loop automation, ok? This is what we want to present today. Unfortunately, we don't have, we don't, we didn't have the time, we have previous engage.01:23:17With other customers and we couldn't finish it. It's almost done. I didn't want to present the the team decided not to present something that we don't have complete that we wanted to show you the path and all those little details, you know, that you can see on the graph, not just show you commands that it says oh there you go. You change your route like we did on the NSO that we show you the route.01:23:37About it and I will show you how we actually changed the path when the valuation the.01:23:57When, when an SLA violation came. So, we will do yeah and that would be for the end of the somewhere the end of before the end of August, we'll have it presented, we'll send you an invitation. Sorry Rajesh. Okay yeah so so the.01:24:02Questions do you want me to bring them online or should I just read them?01:24:22Yeah, let's go see the line the the whichever is the best. Yeah, so if you if you Pooja asks if if it's mandatory to deploy SFP or sensor agents for integration with CNC, if the customer already has a setup.01:24:27Where we have each circle establishes the session directly with the routers and collects performance met matrix, can CNC and assurance leverage these existing TVAMP data or do they specifically require sfps for, for probe configuration and data collection?01:24:43Okay, so if I understand the question correctly, you already have, is that Axian devices doing the T one? If you do, if you do already, you just have to integrate it with the analytics and the proper analytics, as I said before, the.01:25:02And talk to the CNC if the customer already has this, ok?01:25:22If not, I mean, you still have to use the, the correct version of analytics that has got the, the, what we call the maestroke, which is the interface that talks to the CNC. Okay, so then the, you mentioned that they do have a legacy orchestrator.01:25:28And sensor control. Okay, so definitely you will have to do an upgrade, alright and an upgrade the whole system. Okay, Amen's question about what is the granularity?01:25:48Here in CSCNC for probes. I believe the granularity is pretty much the same as we have in in in our analytics. Let me just quick, we can take a quick look. I I have not looked at that, but if we are going to zoom in.01:26:05Let's say from here to here. Yeah, can you share share 2340, but I see about the could be a 10 s in there. I I think I have to come back to you to be more precise on this.01:26:23On this answer, but the probes are acting as they act when it comes to, the performance measurements. So if I just go into the same thing in here, we do have the 1 s in here. You're not sharing Walter. Oh I'm sorry. I apologize. I am.01:26:44Sorry. Yeah, I'm looking right now in the Crosswork assurance and I'm trying to see a granularity here. So I'm zooming in and I think I'm zooming the maximum I can do in here.01:27:04So it seems to be about 5 s that I see in here. If I can zoom in more, I think this is the granularity that you're gonna have in here, but I can come back to you with a better answer and same thing is gonna be here. So here we do a.01:27:191 s.01:27:40Okay, what's the next question? Then how many KPIs that are are supported today? Okay, right now CNC and Crosswork can only do three, and it's T one, that's the protocol that we can use and as you saw is delay.01:27:43Delay variance or some people call it jitter, ok, and packet lost and they're all bi directional, so you're looking at six.01:28:03Again, to give you a better answer, if you are familiar with the, if you have access to a CNC, you can go to administration and heuristic packets, and if you look at the metrics and you put probe in here.01:28:11These are all the exceeding probes. So you have the six of them, ok? As the system, I guess grows and the demand grows, the use cases, they will probably can put more in there, ok?01:28:31Can we, ok, we can even ingest 3rd party data. Well, I mean.01:28:50You need to have a connection with the, with the analytics, you know, you have to have a system connected to the analytics. So if you are having reflectors, maybe you can integrate it with with the CNC if there are reflectors, but to generate the T one, it will have.01:29:04Have to be using on our our devices.01:29:24The Cisco devices. Yeah, and and one one question we missed from Steven was that what is the probe metric are sent in what form? Is it SNMP or? No, it's.01:29:28It's the we're using the Genemi which is streaming the data so there is NO there is NO gaps on SNMP or anything like that. Okay. As a matter of fact, as you saw, the, the graph on the CNC came faster.01:29:45Then what we have on the analytics. Okay. I guess we're right on time. Okay, well thank you everyone for attending and last words I can tell you is there is there's if you need.01:30:03To do something like this. If you have an LTVPN, you can bring those probes. Remember. If you're an SC.01:30:23The customer has a CNC, you have an L two VPN, you can bring those probes in there. You just have to stage it in the in the analytic size, ok? And if he doesn't have one and you're doing a new connection, you can build the L two VPN and put the probes in there too. Assistance, you got the whole team in here, you know, with the CSS in North America.01:30:29Robert Rajesh, myself, and there's other members of this team that have participated in this presentation, but anybody can help you from us. CNC, crosswork, workflow, anything you need to do related also to Crosswork assurance. So you're gonna get an email with some.01:30:50What do you call that survey? Survey. So well appreciate if we can answer so we can continue creating this kind of value of videos or or or presentations for you. So please just take some time and let us know how we do.01:31:12And that's it. Thank you so much. You have a plan for this, does CNC support multiple VRF for probe probing? Yes, it does. And again, before I let you go.01:31:29Let me show you one. Okay, so this is I'm going to an L three VPN here and it's got three different L three VPNs, Videotron, Quebec, about Quebec and R two.01:31:44This one is not functional and I left it on functional on purpose, but these two are operationals as you saw before. I think we're good. All right. Thank you. Okay, thank you everyone. Bye bye.01:32:03