Cisco NSO kickers in action​

Oh, everybody. Thank you for joining today. Today we're gonna talk about kickers in action. This is going to be the expert insights series for NSO. For today, I will be.00:00Your presenter, Robert Hoffman, and I am a customer success specialist in the Internet and mass Scale infrastructure group.00:20So, for today, a little bit of housekeeping keeping here. If you have any questions, please go ahead and raise your hand. We'll try to unmute you or get to your question. Also, we've got a Q and A panel, so if you want to ask there instead, just go ahead and click on your question answer down here in the bottom right and we've got a panel.00:28Panelist of folks who will help answer those questions and also just don't forget to raise your hand.00:48So for today, if you spent any time in NSO ecosystem, you wanna know that it's a powerhouse for managing state data, but a static database is really only half the battle, and today we're going to talk about how we want to make NSO reactive. So how do we get the system to wake up and do something when the.00:55World around it changes. Well, to do that, we're gonna use this trifecta here that you see on the screen. We're gonna use notifications, kickers, and actions. And I find the best way to understand this is through a real world scenario. So throughout this presentation, I want you to try and put this back on this real world scenario I'm about to get you give you. And this scenario is gonna be.01:16Health healing services. So I want you to imagine that you're deploying a complex L three VPN service. The configuration has been pushed and NSO says success, everything looks great. But about 5 min later, there's a technician on site who accidentally bumps a cable and now your interface goes down.01:35Well, what I want to do is I want to use notifications kickers and actions. I want to use this trifecta to kind of remedy the action. So our notification is gonna be where the device sends a netconf link down notification to NSO. Then we're gonna have this kicker, and the kicker is gonna be configured to watch for a link down event.01:55Specifically on interfaces tagged with something like managed service. The kicker is gonna see the specific notification and it's gonna say, hey.02:15Wait this interface affects a live customer service. Then we're gonna have our action, and the action comes into play because the kicker immediately triggers a verification action. And this script checks the operational data is gonna realize that this link can down this this link is down, and then it can either do things like bounce the interface, trigger service readable.02:22Forward or, you know, at the very least shoot off an email notification or alert like a slack or pager duty that, you know, there there has been an issue here.02:42So with that, let's jump into this 1st portion, the notifications. Now, if kickers are the brain of our automation, notifications are gonna be the ears, but not every device speaks the same language, and in a real world brownfield network, you're going to hear mainly two.02:54Two distinct voices. You're gonna hear this old school shout of SNMP, and then you're gonna hear this modern structured conversation of netconf. So let's cover both of these.03:14Your SNMP traps, this is gonna be that legacy shout. So I want you to think of this as like a one way shout from across a crowded room, like someone's just haulering at you that something's happened. And the way it works is this device experiences an event like a link failure, and it sends an unacknowledged UDP packet to NSO.03:25Then we've got our netconf notification, and this is gonna be that structured update. And I want you to think of this as like a formal digital memo delivered over an established connection. So it's almost like someone handing you a note direct like address directly to you.03:44And the way this one works is it's gonna be an XML based message sent over an existing SSH session, and because it's tied to a Yang model, NSO already knows exactly what this data structure should look like. So let's do a deep dive on each one of these.04:00Starting with the SNMP notification receiver, this is gonna be that southbound ear that is specifically designed to listen for SNMP traps and informs on your network. And as you know, SNMP is going to be a push protocol.04:20So the device just blasts a message to the NSO IP address and then we've got this SNMP notification receiver which catches those blasts and then translates them into something that NSO can understand. So let's talk about how this is going to work.04:34Well, if you've ever worked with SNMP, you know that these F SNMP traps are gonna be very difficult to read because they use these oids, which are just like these long strings of numbers like 13614142. And the NSO SNMP receiver is gonna act as a high level translator, so it's gonna look at that number.04:51Where it's gonna tell you what the issue is, what it's saying. So.05:10The way it works is NSO listens on specific UDP port usually like 1602, and this listening for these incoming traps, and then it wants to identify, so it looks at the source IP address and if the IP address matches a device already in nso's device list, NSO accepts it.05:16But if it's not in there, NSO is just gonna drop that. So let's talk about the process. NSO uses this notification handler that's usually written in Java to decode these oids and then map them to a meaningful NSO data. And I say this notification handler. This is something that you would write has typically written in Java just because it.05:34Translates easier to these oids.05:54But, thankfully in NSO under the examples folder, there's actually a SNMP notification receiver already written as an example so that you can pull it into your running config, do a package reload, get it up and running, and you can start playing with it and modify it to fit your needs, but it's basically a skeleton already.05:56That already written that runs and looks for a link down event. And I'll cover that a little bit further. Okay, so when configuring these SNMP notifications or this notification, there's a receiver, there's a couple things you need to do.06:161st you have to enable it in your system. And unlike netconf, which is a usually always owned for most neds, your SNMP receiver often requires a manual kind of flip of the switch. So to enable it, you're going to, if you look in this top corner right here, you'll do a show running config SNMP notification receiver, and you'll see if it's enabled or not.06:35But to enable it, you'll just go into config config mode and just type SNMP notification receiver enabled. And then you just need to identify what port you want to listen to. So this is the port that NSO is gonna be listening to. So this is actually.06:55The southbound IP address of my NSO, and then of course the port number. Now, if you plan on using this notification handler from the examples folders that I mentioned, you'll need to move that over and then do a package reload which is what we've done here. So you can see we've done a show packages package operational status and you can.07:08See that my SNMP notification receiver.07:28Is in the operational up status. Once all of this has been completed, you'll actually be able to show alarms alarm list and see alarms that the SNMP notification receiver was configured to alert or translate an OID into an alarm for NSO.07:31Now, what you're seeing here is that example from the examples folder, and it's the Java implementation of the SNMP notification handler. Now, the primary purpose is for this to listen for incoming SNMP traps from network devices, he wants to parse them and then confirm.07:52Work them into a native NSO alarm that's gonna appear in your NSO alarms manager.08:12So let's break this down. This 1st box here, this is gonna be the heart of the class. It's gonna be triggered automatically whenever the NSO SNMP receiver receives a PDU, you know, your protocol data unit. And then the code is gonna loop through the variable bindings inside the SNMPG SNMP trap.08:20And it's going to specifically look for the OID 1 361-631-1410, which you can see right here. And this is the standard OID for identifying what type of trap has been received. Next, if the value matches 136-163-1153, this is going to be identified.08:38But as a link down event, ok? Here's that one. So this is the OID for a link down. Moving on to identifying the source. Here in this box, you can see the event context tells NSO which device sent the trap.08:58And then the managed object identifies the service or the specific path in the NSO configuration tree that this alarm associated with. Next, we want to create the alarm. So the alarm type.09:16Is where we categorize it into what type of alarm is, so this one will be connection failure. We'll set the severity so you can say that we're setting this to the level of major, so here's our connection failure, here's our perceived severity major, and then the alarm objective. This is where it constructs a formal NSO alarm contain the.09:31Device name, the object path, the severity, and the description.09:51Lastly, we have the alarm sync, and the alarm sync is the API used to push the alarm into NSO operational database. So once this is called, the alarm will be visible in your NSO CLI or web UI, so basically what we're saying here is once.09:56We do this part and finish up this configuration, you'll actually see this in your alarms list. So in your top right corner of your web UI, you will get an alarm notification just like you would if something else in NSO was wrong. And then the lastly, we've got this return continue, and this tells the NSO SNMP receiver that this trap has been processed by this handler.10:11But that other handlers are allowed to process it as well.10:31With that, let's move into that second type of notification that we mentioned, which is the netconf notification. And these notifications are a standard way for devices to speak to NSO asynchronously. So instead of NSO constantly asking a router, hey, is this interface down? You know, this is like you're polling, hey is this interface.10:38Is down, is this interface down?10:58Instead, the router simply tells NSO the moment that something happens. And to understand this, you really need to look at it from two different types of perspectives. You need to look at from the southbound, which is the device to NSO and the northbound, which is NSO to you.11:00So diving into this southbound, you know, this device to, to NSO, this is the process of NSO listening to the network. So when you configure a device in NSO, you're gonna be able to be able to set up what is called a subscription.11:19A subscription has three kind of major parts. It's the connection, the request and the result. The connection is where NSO opens a dedicated long lived netconf session to the device. The request is where NSO sends a RPC. So this tells the device, hey, send me every event that has.11:36Happens on your netconf straint, and then the result.11:55When an event occurs, something like a BGP file are failing or a config committed, the device is gonna send an XML notification block to NSO. Now, the northbound is gonna be NSO to yourself or external systems. So once NSO receive that notifica notification.11:59One device, it doesn't just keep it, and so organizes these events into notification streams.12:18And these streams are gonna have two parts, the categorization and the external consumption. So for the con the categorization, NSO sorts events into, you can think of these as like radio stations or streams. So, you know, you've got the the blues on one side, you've got.12:25Country on another, you've got brap on another. Each radio station has different streams. And based off the type of event that you get, it'll go into these streams. The external consumption is where your monitoring database or your OSS.12:43Can connect to NSO via netconf and subscribe to these streams. This way your dashboard only has to talk to one NSO instance instead of thousands of individual routers. So let's break down this notification stream a little bit. As I just.12:59Try to explain NSO does not send every event over a single pipe. Instead, it's going to use these multiple streams, and this allows clients to subscribe only to what they care about. So by default, NSO provides a couple built in streams. You've got things like n NCS alarms, NCS.13:16This events, device notifications, service state changes, and your netconf stream. And your netconf stream is gonna be the default RC 5277 stream.13:36Now, if I mentioned I want you to think of these NSO notification streams like a radio station. So you're gonna have your broadcast, you're gonna have your content, you're gonna have a listener, and you're gonna have a replay. Your broadcast, this defines the station, so like I said, that's like your oldies or your, your country or whatever your classical. These are gonna be your different states.13:48Then you have your content. So your content is if an event happens like a link goes down, and so packages this into an XML and plays it on the station. So think of that as like the song that's playing on the station. So if you're listening to the oldies or you're listening to classical, it's the song that's actually playing on that station.14:08And then the listener, you know, that's gonna be your northbound system that sends a create subscription request for that specific station. So that's us listening in, that's us listening to that channel and hearing this content. And then the replay.14:25That's basically a support, and not every stream has this, but most streams do have replay support, so if you log in or if you miss something, you can catch up on all those songs that played from a specific tar start time.14:41Now, I know that's kind of tough to understand, so I tried to build this flow chart to help describe it. So looking at this top box here, we've got our device notifications, which is the raw data from the network. That's your southbound to NSO. That's just the devices talking.14:56Then we've got the internal logic. So this is your notification capability. This is the logic that allows NSO to package data into events. So we receive some type of notification. We're packaging it, we're kind of figuring out, hey, where does it go? What does it fit into? And then we've got our streams, the categorization on the broadcast of those events to the world, that's the NSO to northbound. So we receive.15:11Receive a notification, we package it, figure out where it goes and then we put it into the appropriate stream.15:31At the bottom here, we've got what I tried to build as an example for real life. So, starting on the left, we've got a Cisco router sends a netconf notification saying, hey, interface gig zero one is down.15:38The NSO is gonna process so NSO receives this via the device manager and an NSO Python application or case star strip sees the event. So this is where we're getting into this internal logic here. We receive the event, we're getting into the internal logic. The NSO application generates a custom high level service degraded notification and sends it northbound notification stream.15:53Thing called alerts. So we see we received the alert. We kind of figured out where we go. We're going, hey, this belong in the stream called alerts. Now an external monitoring tool subscribed to the alerts via the netconf receives the message and.16:13Page is an engineer. So here someone said, hey, I'm interested in alerts, please describe me to this alert stream. Now it receives the alert that was sent by the device. Okay, let's look at a, at a notification configuration.16:27So, at the top here, when you run the show devices PE two notification stream, you're asking NSO, what stream or what streams does this specific router support? And NSO retrieves this information from the device's netconf hello capabilities or by querying the device's netconf monitoring gang model.16:44And for this one, you can see we've got.17:04A stream called netconf, a stream called trail audit and a stream called interface. They're highlighted in blue here. The netconf stream is gonna be the RFC standard stream, and most devices have this. The tail F audit, this is gonna be a specific stream that sends logs whenever a configuration commit happens.17:08And then you have an interface stream. This is gonna be a custom stream defined by the device. Now you need to run this.17:28Devices ICE notification stream to see what your device is capable of before you start subscribing to it. Then the bottom, we've got 2nd part of this. This is where you're actually going to be establishing the ear. This is where you're actually going to start listening. So.17:36You're gonna have to build a subscription. So you can see we went to devices, devices, and this is WW 032 notifications. And then we've got the magic here. This is where it happens. This is us saying, hey, I'm creating a subscription.17:56And I'm gonna call the subscription myself, so I want to listen to something and I want to listen to it by myself. What do I want to listen to? I'm interested in interface notifications, so I want to subscribe to the stream called interfaces. So now any information that comes in.18:11On that stream will be notified for us. Just saw a hand raised, is there a question I need to answer? Yeah, they just put a question, Robert. They said, can we use the note.18:28Notification event to trigger another NSO workflow. E.g., a router config has been manually changed and NSO received a notification and then trigger NSO to sync from device.18:48Absolutely you can and that's what we're about to talk about. I will show you how to do that literally in the next slide. So let me go ahead and move to that. So what they were asking about is going to be using the kickers. So.19:01We successfully set up our ears. We have SNMP traps shouting at us and netconf notification sends these structured memos. But if we stop right there, we've basically just built this expensive log collector. So, you know, in a massive network, you don't want to just wake up an engineer every.19:19A Single time you get a notification, you want the system to look at this data and then compare it against your basic logic and then decide if it's worth worth acting on. And that's gonna bring us to the brains of it, which is what I'm gonna call or the kicker is what I would call the brains.19:34So your notifications are your ears, the kickers are your brain, and this is what's going to help you build that, that next step that you just asked about. So in NSO kickers are gonna be this declarative watchdog mechanism, and there will allow you to monitor specific data in NSO and our.19:50Automatically trigger an action when that data.20:09Changes or some type of event occurs. So I want you to think of a kicker as like a trip wire. What you're gonna do is you're gonna set this trip wire on a specific part of a network model. And then the moment moment that something touches that data, the kicker is triggered. So the kicker is gonna kick a script or a service in.20:13To motion.20:32Now, with NSO, we've provided two types of kickers. We've got the data kicker and we have the notifica notification kicker. The data kicker is gonna be your most common type. It watches your CDB configuration database for changes.20:34To configuration or operational data. And the way it works is basically you're gonna point it at a yang path, and if any data in that sub tree is created, deleted or modified, your kicker fires. Your second type of notification.20:53Alright, your second type of kicker is gonna be your notification kicker, and this re reacts to asynchronous events, basically your netconf notifications that are coming from your southbound devices. And the way this one works is it listens to a specific notification stream, like the interface netconf stream that we just discussed, and when the notification arrive.21:10The kicker is triggered.21:30And a great use case for that one is like a self healing workflow, so if a router sends a notification that, you know, a BGP neighbor is down, the kicker can immediately launch a Python script to investigate or restart that session. Now, why do we want to use kickers? Well kickers are gonna be preferred over.21:32Traditional polling for a couple of reasons here. Your efficiency, your speed, and your simplicity.21:52The efficiency, they are event driven, so they're gonna use zero CPU until the exact moment that a change occurs, their speed, their resection time for these are near instant, and then simplicity. So you define these kickers in Yang and XML.21:58And you're gonna find them as configuration rather than writing these complex multi threaded listeners in like Python or Java code. So to get started with these kickers, the 1st thing you've got to do is you've got to move to your top layer, move to your nCS.config and make sure that you've in.22:17Inserted or put in this debug, so by default they're gonna be hidden, so you'd have to go to the hide group and put in debug. Once there, if you move to your CLI session and do move into your config mode and just do unhide debug, then you'll have access to your kickers. So by default, if you come here and type in kicker.22:37Question mark, nothing comes up. So you just gotta make sure you make these two changes.22:57Okay, with that let's get into data kickers. So we're gonna break down data kickers and we're gonna break down notification kickers and the data kicker, as I mentioned, this is this automation tripwire, and it's a specialized mechanism used to bridge that gap between the change in data and the.23:03Execution of a action.23:22So, once you think of this while a service handles how a device should be configured, a kicker handles when a specific script or action should run based off the events happening in the system. So why use this kicker? We're gonna use the data kicker when you need reactive automation.23:25And here's a couple of scenarios. You've got self healing post provisioning and external integration. So self healing. So when a notifi notification arrives saying an interface is down, the is gonna see this operational data and trigger a s a script to kick the interface or reroute the traffic.23:44Your post provisioning. A service is finished deploying, the kicker watches the service state, and once it hits the ready, it's gonna trigger an external system to update an inventory database. Your external integration scenario is when a specific leaf in the NSO config is changed by user. The kicker triggers an action to send a rest API.24:04A call to a 3rd party monitoring tool like Splunk or Slack.24:23Now, there are a couple pieces to configuring a kicker. You've got the monitor, the trigger expression, the kicknode, and the action. Now remember, these are for data kickers, notification kickers will be slightly different. But for the data kicker, the monitor, this is gonna be the specific lead for container and NSO.24:30That NSO watches for right. So this is what we're viewing. This is what we're looking at, ok? This is the part we're monitoring, the trigger expression, this is gonna be that filter. This is gonna be what we're seeing. Like this has to happen on whatever we're monitoring for us to kick the notes. So this is that x X path expression that must evaluate your true.24:48The kicknobe. This defines the context of where, where is this action at? Where should we do it? And then the action name is, hey, this is what we want to do. So this can be some type of executable script, ok?25:07Let's break down this trigger expression a little bit. The trigger expression is gonna be this boolean X path expression that serves as the filter or the constraint for your data kicker. And while the monitor path tells NSO where to look for changes, the trigger expression tells NSO what.25:26What specific conditions must be met for that action to actually fire?25:46So, how does this work? Well, that trigger expression is gonna be evaluated twice by NSO before the transaction is committed and after the transmit the transaction is applied. And then NSO compares these two results to determine if triggering event occurred. And this behavior is gonna be controlled by the triggering.25:50Type.26:11So you can see here as this example, we want to trigger on a specific threshold, so you want to run a script only a device's CPU. It crosses 90 %. What are we monitoring? Device to device lifestyle CPU usage, so we're monitoring the CPU CPU usage of the devices and device.26:14Trigger expression only trigger if it's greater than 90 degrees, and then the trigger type enter, so only do it enter, we're not doing it entering leaving, we're just doing it once. Now, this is gonna be the data kicker that I use in the demo.26:33So this is gonna be an S I'm gonna call it SNMP sync and what I'm looking for is I'm monitoring the alarms from an SNMP notification receiver. So the SNMP notice notification receiver is gonna receive an alarm, it's gonna put it in the NSO alarms. I'm monitoring this alarm path for a connection failure.26:52For a specific problem on device PE two. So I'm looking for a very specific alert. My trigger expression, this is gonna be what causes it to trigger. So I'm looking for a status change. So does this change? Then I've got my take note, what am I.27:11Going to run my action on, ok? Or where is my action located? The action that I want to run is going to be on PE two and the action name is gonna be a sync from. So if I see this alarm come in and it has a status change, I want to run sync from, from PE two.27:28Now, I'm gonna talk about actions here in a second, but just know that sync from is a built in action that does something. Of course, you're all probably very familiar with sync from. Okay, with that, let's move into notification kickers.27:48And to understand a notification kicker, I want you to imagine that you have a security system for a large building. So with this large building, you're gonna need a notification, you're gonna need a notification stream, and you're gonna need a notification kicker. A notification stream and notifications are something we just talked about in the in the previous slides.28:04But the notification, this is gonna be your sensor, like a motion detector on a specific door. It only sends a signal when something happens.28:23Like the door opens. You've got your notification stream. This is gonna be that radio frequency or the wire that carries the signal from the door to the security desk. Then you have this notification kicker. This is the automatic rule you programmed into the system. So you could say if the back door sensor sends a signal, immediately turn on the flow.28:32Sites and call the manager. Now, in NSO, a notification kicker does exactly this for your network. It bridges the gap between your device event and your automatic response. So let's talk about that flow.28:52Normally your NSO just kind of sits quietly and manages your configurations. But devices are gonna be chatty. They're constantly sending alerts and the kicker is what makes NSO listen and react to these alerts. So you're gonna have an event, you're gonna have a notification to stream in the kicker, the event. A router has a problem like a link failure.29:10The notification comes, the router sends an XML message.29:29To NSO. We've got the stream, the message lands on specific folder or channel in NSO, and then the kicker is where it sees, is we're listening and it sees that message in a stream and the message matches one of our rules so the kicker kicks some type of action off a python script or.29:33Something. Now, this is the construction of a notification kicker slightly different than a data kicker but ultimately the same. So we're gonna have an ex a selector expression and this is gonna be the XPath filter that evaluates the data inside of a notification to determine.29:53If its associated action should be triggered. So before we were looking, for something in the yang model, we're looking for some type of change here we're looking at notifications. So a notification will come in, we'll look for a specific notification name or event in the notification.30:12The kick mode, this is gonna be the specific object or path in the configuration or operational tree that NSO monitors for changes or incoming notifications. The action name, this defines the specific event almost like the sync from. So this is the action, this is what we want to do. And then we got the variable.30:29And this is just a nameplaceholder used to capture data from the monitored path. So let's look at these variables a little bit. Well, this notification variable is used to store the result of an X path expression so they can be reused in your selector expre.30:48Expression. So I want you to think of it as assigning a name to a piece of data that you want to track. So instead of writing this massive complicated X path inside your trigger.31:08You can just define a variable once and then refer to it by name. So let's break down this variable structure. Based off this slide here, we've got a variable that consists of two parts. We've got the name and the value. The name is the identifier that you choose. The value is gonna be your standard XPath expression, so it's the logic in so.31:19Users to find the data you want.31:39And in this notification expressure, you're gonna deal with two flavors of variables. You've got your predefined variables, and these are gonna be your freebies that are gonna be provided by NSO automatically. Things like device name and notification name. Then you have your user defined variables. These are the ones that you create and add to your own business logic. So.31:42Looking here, these dollar signs are gonna indicate that these are variables. So we've got dollar dollar signed notification name. This is going to be one of those freebies of built in variables, alright? Now if we come over here, we've got another dollar signment that says IP. This is gonna be one of those custom variables that you make. If you make a custom variable, you'll need to.32:01Have something assigned to it. That's what we're doing down here. Variable IP, the value, 192168. Now we can call IP multiple times if we had a much more or a longer selector expression, we need to call it multiple times, we could call it. Okay, so let's look at this ap.32:21Example and this specific example shows a notification kicker configured to trigger an action whenever a message arise on a specific.32:41Subscription. So the 1st thing we need to do is we need to establish the listening side of the device. So we can see the device supports a stream called interface. We created a subscription named myself on the devices, and this means that any interface event from these routers are now flowing into.32:51And so under this tab.33:10Called my sub. Now, if we look at this kicker named in one right here, this is gonna be the logic that access the brain for these incoming messages. So we got the selector expression, and this is the filter it tells NSO, I don't care about every notification in the system. Only fire this kick.33:12If the notification came through the subscription called mysub, the subscription name my sub.33:33Then we've got this kicknote, and this defines the object that the kicker is interested in. And in any or in many of these NSO designs, this points to the location where the action lives or the service that needs to be updated. So this is, hey, where are we gonna find.33:38This action. Well, it's gonna be under services actions. All right? Then we have the action name. This is gonna be the specific Python or Java task to run. So basically it's a script called diff check. So, if we see a subscription called mysub, I want you to.33:55Use the action diff check, where is the action located? Services wise actions. Alright, so what would this kicker actually do? Well, let's say there's a cable pool on device WW zero. All right?34:12WW zero sends a netconf notification. NSO receives it, and because our configuration NSO knows that this belongs to a subscription called mysub. The kicker in one sees this new new notification. It checks the selector expression since the subscription name is indeed my sub.34:28That expression is true.34:48Because of this NSO immediately calls the action diff check, it passes the notification data and the kicknode path to that script, and as a result, the diff check script runs. Now, by thinking where else can I use kickers?34:51So just to give you a little insight, nano services use kickers as the engine that drives a service through the different stages. So, if you ever worked with a standard service, the standard service runs once and then finishes. A nano service is designed to pause, wait for something to happen, and then automatically resume.35:11The kicker is gonna be that mechanism that wakes up the service and tells it to move to the next state. So looking at this little snippet here I have of a notification or a nano service. Down here at the bottom, you can see things that.35:31You know, point you that and tell you that this is a kicker. You can see that we've got a monitor. We're monitoring our service, we've got a trigger expression service or server done equals true. So when we monitor this and we see this become true, we're able to fulfill this precondition and move to the next part of this man of service.35:46Alright, so let's move into these actions. So far, our kicker has done the heavy lifting of the decision. It heard the notification, it checked the monitor path, and the trigger expression came true, and at this point we've got alarm bells or ringing alright?36:07But this kicker is just gonna be like a middleman. It doesn't actually know how to fix a BGP session or how to talk to an external system. It just knows that it's time for something to happen. So, to actually turn the wrench.36:24The kicker is gonna call an action. And we're gonna think of an action as the hands of NSO. So notifications were the ears, kickers were the brain, actions are gonna be the hands is what actually does something. And if you haven't worked with actions before, an NSO action is effectively a.36:39An RPC call used by NSO to execute some type of arbitrary code associated with a Yang NSO action. And when you load a custom package into NSO, it extends the application functionality. Just like when you build a service, you can build out the data model for the inputs in your service.36:59You can build out a data model asking for inputs and outputs in Yang. E.g., the common action of sync from, we all know this is a built in NSO action that does not store any data on its own, but when triggered it calls us NSO to execute code behind the scene.37:17To log into a network device, sync the data from that device into NSO CDB. Now the NSO action is going to be a combination of a Yang model defining the constraints and data model of the inputs expected or or outputs expected for the action. And then there's gonna be some code.37:35Linked to it when the action is triggered.37:55So when an NSO action is created, just like any custom package in NSO, the Yang propagates the action to be available across all your NSO APIs. So this means that with just a few lines of Yang and a few lines of Python, you're very quickly able to create an API interface exposed to any user who has access to it.37:57NSR.38:17Now, I know this sounds very much like a service, so I want to clarify actions versus services. And typically when you write a Yang model in NSO, it's connected to a template and you're trying to push configurations. But with actions, the Yang model is in place to provide structure instead of pushing data into a template.38:20And then pushing that information to devices.38:41What we're gonna do is we're taking that input into our Yang model, which will feed our Python code to complete a series of actions. So let's let's hone in on actions and services. Your NSO action focuses on specific tasks or operations within the NSO environment such as data manipulation.38:44And service deployment validation and custom automation automation tasks. And examples of these would be like configuring a device.39:04Deploying a network service, validating configurations. Your NSO service represents a higher level abstraction defined the configuration behavior and management of network services within NSO, which can span across multiple devices and technologies.39:12And examples of this would be things like VPN services, MPLS services, firewall policies, QoS policies. So you can think of NSO or NSO services as like kind of higher level touching more things and actions being specific pinpoints.39:30Now, to create an action, it's very similar to creating a service. You want to use your NCS make package command, and if you use your NCS make package command dash-help, you'll get all of the kind of examples you see here in this white box of what you can do next. But what we want to do is we want to use the NCS make package dash-service skeleton.39:49Python and then after python, you can use exact the actions example and.40:09The action example does the same thing that it does with your service skeleton for templates, Python, nanos, all this. It builds the file structure and go ahead and puts in your yang, your Python and goes ahead and gives you your service points and builds out kind of those top configurations that.40:15Attach everything together. So all you have to do is go in and write your code.40:35So, let's look at this Yang model. Each action is described by the action yang statement that specifies the inputs and outputs. The inputs are gonna allow a user of the action to provide additional information to the action while the outputs provide information to the caller.40:39And the 1st thing here that I want you to notice in this code is that just like services use a service point actions use an action point as donuted by the teleph action point statement and this tells NSO to execute a callback registered to this name. So you can see it right here.40:58Now, let's look at the python side of this. With the Python, we're gonna cover these four topics. We've got the registration, the action class class, we got a method, arguments, and then the decorator. So the registration.41:20Well, to make NSO aware of your code, you must use the register action. This links the action point defined in your Yang model into the Python class you just wrote. It's right here. Then we've got the action class. So your logic lives inside a Python class, so two things are gonna be mandatory. You're in.41:37Inheritance so your class must inherit from your NCS.dp.action and then the callback. So you must define a method called CDB.41:57Or CB_action, and this is gonna be the actual brain of your command. Next we got the method arguments. And when the CDB action runs, NSO is gonna hand you basically several useful tools or arguments. And those are gonna be here.42:05And to understand those I want to cover them. You've got your U info. This is who is running the command. You've got your name, which means which action was called, you have your KP, so where is this happening? This is basically the key path.42:24You have your input, what did the user provide? You have the output, what should we tell the user, and then you have the trans, which is the transaction. This allows your code to read other parts of your NSO database during the action.42:41Now the 4th part is this decorator, and you're gonna see this at symbol action.action sitting right above our method here, right here. And the reason it's there is in NSO, data is gonna be handled by the magic API, and this decorator ensures that the input and output.42:57Variables act like easy to use Python objects rather than these raw complex data structures.43:17So let's move on here. Here we've got an example of two Python modules. The module on the right is the main.pie that was created by the ECS make package command. And with it we've got services actions and other features that all rely on this callback registration. And in.43:25Python code, this class is responsible for the registration that drives from the NCS NCS.application.applications as you can see on line five. And this allows NSO to manage the application as code as appropriate such as starting and stopping.43:43In response to NSO events, and these events include things like package load unload and NSO start and stop events. And inside the setup method, we call the registration action with the line self.register action count devices action, count devices action. And this line instructs NSO to you.44:02Use the count devices action to handle invocations of the account devices action action point, and also the account devices action class from the account devices action module. Now, the module on the left is a separate piece of code used for the action, and you know these don't have to be separate, this is just done for a demonstration purpose. But basically what.44:22This is, is this is an example of a custom action that we're gonna see run in the demo.44:42And this custom action is called count devices and what we're looking for this to do is we're gonna go into our custom actions, we're gonna say, hey, I want to run this action called custom action, and it's gonna ask me for an IP address or an IP range. Once I give you that range, it wants to tell me every IP or every device or how many devices are in that IP range.44:47And the reason I brought this one up is.45:07You know there's not really a use case for when you want to do that, but I want to show the versatility of actions because you know there's a built in action called sync from, so we know we can sync from. Now, this is the far other end. We're even able to write a Python script to count how many devices are in a subnet. So this just shows the versatility of actions. So when you.45:10Pair this 1st utility with your kickers and with your notification, whether it be SNMP or netconf, you're gonna get a very versatile tool to do what you want.45:30Because we could build an action that just said, hey, sync from, hey count these devices. Hey, deploy or redeploy a service that we already have built. So the possibilities tend to be endless at this. Now this last point is the gang model, and the gang model.45:40Looks very similar to your Yang models that you have in, your services. You've got your module, your yang version, your name space, your prefix, your imports, and then you have your container down here at the bottom, and here you can see where we're taking in two different things. We've got an input and output. So the output.46:00Is, you know, what's going to be displayed on the screen. The input is, hey, we're only taking IPV four prefixes.46:19Okay, with that, let's move to the demo. So 1st thing I want to show you, if you're interested in learning more about this, if you go to our user guide, connected top is SNMP notification receiver. There's a whole page on the notification receiver with examples.46:29And links to GitHub with more examples so you can read more on your SNMP notification receiver.46:49Same thing with your notifications capabilities, you've got your built in notification streams, define the streams, automatic replay. This is under development NSO netconf server. So you can do more here, and this also has a link to a GitHub where an example has been run.46:56With that, we also have kickers. This is a whole page dedicated to kickers and different concepts with kickers, notification kickers, data kickers, NSO service with, with kickers debugging kickers. So you can go through here and get a lot more information than you were able to at least hour and a half.47:14Okay, so with that, let's jump into the actual demo portion. So I'm gonna cover a couple things, but what I want you to remember is that this is going to be our.47:34That we're that we're monitoring. I'm going to be specifically monitoring gig zero zero to a customer on PE two. To start this off, let's go ahead and get logged in.47:49Okay, you can see gig zero zero is currently up on VRF red. So we know this is up, this is what we're gonna be monitoring. Now we're gonna talk about the SNMP notifications, netconf notifications kickers, and then at the end you're gonna see us run all this and bring it all together. So.48:051st thing we want to do is we want to take a look at our S&P notification receiver. Okay.48:25Just like in.48:45The slides that were showing, if I do show running config snmps notification receiver, you can see that I have it enabled and I am listening on 192168 one.two oh eight, which is my only port out on this or interface out on this NSO so this the interface NSO interface that I have to listen on.48:47If you are planning on using this built in package, let me show you where it's located.49:07So I moved to OBT NCS, my running NCS.six.six and then examples. Under examples, you're gonna find a lot of different examples that you can look for. You got triple AAA common device management, getting started, high availability, so there's a lot of stuff you can look at. We're interested in the device management.49:27Under device management, you're gonna see much more packages and examples that you can run. Of course, we are interested in the SNTP notification receiver located here, and this is the demo part.49:45Work, but if you go ahead and dropdown into packages, you can go ahead and move this straight to your.50:05Running folder into a packages reload and you will actually have this package. Now you will need to go in there and make a couple modifications because this is an example, but it gives you a good skeleton to work off of. So I took that made a couple modifications and stored it.50:12And here and then did let's do a I'm not gonna do a package reload but show and you can see that it is up.50:29Now, that was our SNMP one. Let's take a look at our netconf notifications. So if I get logged in here, this is my local installation. We can do a show.50:47Devices, device is zero, notification stream. So this PCA zero is a net sim that I'm using just for this example, but it is a netconf capable device, and so I will actually be able to see what possibilities and what streams.51:04Become pre configured with this device. So if I come in here and do a notification streams, I can see that I have the built in RFC 5277 notification stream. It does not have rest replace support, so if I missed the notification, it's gone, and it also has restconf.51:21If I was interested in subscribing to that, what do you or if I was interested in seeing more notifications, we'll go ahead and just hit notifications. You can see we're getting a lot of notifications, right? The top part is right here, shows you the streams, and then it shows you the notifications you're getting.51:41On top of this, you can actually see that I have a subscription already built called My subscription. I showed you the the commands for that during the presentation.52:00But this is a great way to check and see if it is running. In the GUI, you'll see something very similar. I come into PCA and down here notifications, we can click on notifications. We.52:08Get very little information, so we want to look at the operational data.52:28Operation, the operational data shows us all the events, the streams and the subscription. Now, once we received an alert on this, there would be another place that we could come in here and we click on my subscriptions and we would be able to click on receive notifications. Your receive notification is gonna give you that good path back to help build.52:33Your kickers. So if I needed you know your XPath expression to where this was located, this alert was located for our kicker, we come here and click on here and you can start seeing this. Now you might want to change this a little bit, but it gives you a good idea of where this.52:52Notification path would be. All right, so let's jump back and take a look at our kickers here. I don't need that one anymore, bring this one up.53:10So it gives a little space to see. Let's go into config mode and do a show configurations kicker. You can see I'm tabbing out nothing's happening. If I do a question mark, nothing enter element does not exist. Remember, this is because we have to.53:29Upon hide. So unhide.53:48Debug. Now, I tab that out, it finishes. It enter, and you can see this is the exact notification of the note, the not notification, this is a data kicker that I showed in the presentation where called SNMP sync. This is our monitoring, we're monitoring our.53:51Arms.54:11We have our trigger expression and where's our sync from located? PE device, we want to run a sync from. Remember sync from is a built in action. So if we come here.54:12PE2.54:33Actions, you can now see a sync from, right? So that is the location of this device advice be two, we're doing a sync from from there. Now, I don't think this is gonna work right now because I don't have any alarms, but if you're looking.54:35Looking for information on like where these alarms would be if you're planning on using the SNMP notification, you can do a show, alarms, alarm list.54:54Display xpath. Now, this is going to get you close to this right here. It'll give you a good starting point. It won't give you the exact path, but it will give you a very good starting point. I don't have any notifications right now or alerts or alarms right now, so we're getting a very small list, but.55:05Once I have it, I'll try and come back in here and run this and you'll see something much more.55:24Similar to what we're here that we're monitoring. Okay, last thing I want to show you before we get this actual self kind of healing transaction to work is our account devices or our custom action.55:30So, when you're using a built in action, you'll just go through wherever that built in action is located, like we did when we went to the devices device PE two and actions, ok?55:46When you build a custom device, custom action, they will actually be under config mode, question mark, you can see custom actions here. So.56:03Actions, do a question mark, you can see I have the package called account devices, devices question mark in subnet subnet. Now, this was where we needed IPV four prefix, so 192.168.1.0 slash 24. Now when I run the.56:21Is what it's gonna do is it'll count how many devices.56:41I have in this subnet that NSO knows about. My NSO from my CML routers should know about eight devices. So we've got 678, because I only have.56:45Loaded this internal network into NSO. Let's get back to it. When I hit enter, it counts it and I get a result of eight. So he was able to go in there and count through the devices. Now, I know this is kind of a.57:02Awkward kind of action here, but if you built this action to do something else like redeploy a service in your kicker, you could come in here and change sync from to deploy service or whatever.57:18Other option you decide to put. All right, so now let's put all of this together. So the 1st thing I want to show you is PE two. Last in sync was yesterday, 24 of March at 1013, and we currently have zero alarms.57:35Now this is gonna go fast, so don't look away.57:55We currently have our interface up, so I'll go into config mode, skip 0000. Let's do a shut. Now, but again, nothing has happened yet. I have not committed, but as soon as I commit it, you're gonna see an alarm and this device go off.57:58Commit, jump back over, there's our alarm and you can see that we automatically did our sync from. So our sync from happened. So basically when we receive this notification alarm here with a major.58:17For managed object with the alarm linked down our kicker, which is managing connection failure and seeing if it changed on PE two, we're gonna run a sync from. So the moment that happened we get a sync from.58:33So, that's the way that you can use, your notifications, your kickers and your actions together. So if you wanted to do a more in depth one, maybe you had MPLS tunnels and you had a tunnel go down that you monitored.58:51And you received a notification for it and then you switched in another tunnel up or if you had another uplink and you could use a service to and to you know spin up another uplink, you could have certainly action. So when you got your notification from a device, your kicker saw it, use the action.59:09Let's say redeploy service A or service uplink B.59:29Okay, so that wraps up the demo for our next steps, you may receive an invite. If you could help us out, that'd be great. And if you've been with us, we had our latest and greatest NSO six.six. That was on 18 March. We will be able to show.59:33The recording here before long and we do the nso's kickers in action. That was today. Our next one is gonna be the NSO observability and focus, and that'll be on one April at 1430 GMT.59:53All right, any questions? So there's one on the chat. Okay. And it was about the, so I answered it, but do you wanna if you wanna make any ball asks, can we use the.01:00:08Account notification event to trigger another.01:00:28NSO workflow or task. E.g., if a router config has been manually changed, can NSO receive notification and trigger NSO to sync sync from device? Yes, so as long as you are receiving that notification you can figure, you can configure.01:00:32Or a kicker to run an action of sync from like we did here. Now you would just have to set up.01:00:51Your notifications in a way that you received any type of change. So probably want to use a possibly a netconf notification or a, and a data or a data kicker would I suggested because it's config CDB change, right? Because when they when.01:00:57It changes, it would be a CDB change, right? Exactly perfect. So data cake here we can.01:01:17Example will be, you know, where Robert showed all the, all the, all the resources where you can look at examples. A data kicker example is is present there.01:01:30Alright, are there any other questions? I see the chat is empty, I don't know if there is more questions. Let me check again.01:01:52Yeah, I think we are all covered Robert. Let's give it might be one, 1 min more just for them in case they are shy to answer.01:02:03Please, we have Robert, if you have any questions, feel free to to put them in the chat, ok? Okay, apparently we don't have more questions. Okay, so.01:02:12Let's grab up the the session here. Robert, thank you very much for your time and well we have the next session next week. So if you have the link registration, please feel free to to share it with the the rest of your team and your organization, and we'll be very happy to answer questions and to have you here.01:02:31So I hope you enjoyed the session. Thank you very much and see you the next week.01:02:51