Introduction to Telemetry in IOS XR Fundamentals and Use Cases

Good morning, good evening, good afternoon, and welcome to today's session, introduction to telemetry in iOS XR fundamentals and use cases. My name is Brooke hammer.00:09I am the English community manager at community.cisco.com and we are very excited to present to you this community live session today.00:20Perfect. Today we have presenters and that would be Eduardo Ramirez and Hector Karava. Sorry, can you, there we go. It is not staying on there so I will go ahead and give you their intros here.00:30While you're doing that, I will introduce you. Eduardo is a technical leader for service provider access at Cisco. He holds a degree from the national Polytechnic Institute in Mexico and brings over eleven years of industry experience in his role. Throughout his nine year tenure at Cisco, Eduardo has contributed across various domains, including professional.00:48Services and the TAC supporting service providers worldwide. He is recognized for his expertise and holds prestigious certifications such as ccie service provider, CCNP enterprise, and devnet. Hector is a technical leader of service provider XR.01:08He graduated from the Monterate Institute of Technology and higher education, state of Mexico campus with a degree in computer systems engineering. He has been working at Cisco for 14 years in the technical support area, assisting internet and cellular service providers around the world with experience in routing and switching technologies.01:26And knowledge of platforms such as CRS, ASR 9000, NCS 5500 and Cisco eight K Additionally, Hector has more than 16 years of work experience and holds the CCIE enterprise RNS devnet and CCNP service provider certification. So these are our fantastic presenters today. We will.01:46Have some slido polls that will come out throughout the session. They will be live in the session, but if you would like to join using a different device, please use the QR code here to go ahead and join those Slido polls. If you have questions during today's presentation, please put them in the Q and A panel and we will have folks on standby to answer.02:06Or you may wait until the end and ask during the Q and A portion of our, presentation today.02:26Today's agenda, we will be doing an introduction to telemetry, a model driven telemetry presentation, coding and transport methods, configuration and deployment, and then of course a lab and demo. And with that, I believe I'm handing it over. Go right ahead. Thank you so much.02:36Thank you very much work and thank you very much for everyone that joins the session. So yeah, today we're going to be talking about telemetry. This is going to be some introductory sessions, so basically we're going to go through initially through basic stuff, let's say basic terms terminology when we talk about telemetry. And from there at the end we're going to have like a small configuration.02:55Session and demo so you can see how this works directly on the devices. So we will be focusing as we mentioned in ISXR devices. So well, let's get started.03:15So 1st of all, we need to talk about how things have been working so far and before, ok? So monitoring is not something new, it's something that it happens since the networks are there. So there's there have been different types of monitoring in the networks. So some of them that you should know, I'm pretty sure that you know or have heard about.03:26Those is e.g. SNMP, or even CLI monitoring, but those technologies, let's say comes with some limitations and complexities, ok? So talking e.g. about SNMP, SNMP is a technology that yeah it helps us to monitor networks in let's say big dimension, but in the.03:46Usually tends to be really slow, ok? So for those of you have that have already worked with SNMP, we can talk about MIPs and NO IDs. If you remember, they are really difficult to read. It's a big number, one.two.3.five.3.ten.I don't know what, but there are really big numbers.04:07Identify each of the components on the device, but that actually gives a little bit complexity in terms of when we want to select why we want to monitor or not, ok? And also it comes with other limitations, e.g., the amount of information that is sent through the network, it may also increase the bandwidth used on the network.04:26And in some situations depending on congestion and other factors, information may arrive incomplete to the servers, to the destination, which means that we were we're not going to have let's say entire information to actually monitor and take decisions of the network. In terms, e.g., with syslog, we know that we in every every single device, it doesn't matter.04:46Which vendor it is or whatever we have syslog messages. But most of the time even between vendors, there could be NO or they cannot be let's say a standard. We can have a format in one device, another format in a device that all that also complicates things in terms of what we when we're sending drafts, e.g., we have syslog message.05:06It depends on the format of the syslog to read it and understand what is happening, ok? And obviously also talking about SNMP assistlock, we are talking about a pool model, which means that we are going to the device and asking for information. Go to the device and ask for for information. So in terms of scalability.05:26That is really complicated when we have big networks as e.g. is an in service provider that we can have 50 hundreds or even thousand devices, ok? And obviously, well, the the last one and it's still used out there, the CLI monitoring is obviously not that good to use because you need to be connected to each of the.05:46Devices to actually monitor them and get the information that you want. So you need to know the specific CLI that you want to run, you need to know the specific platform and all that. So obviously that when we're talking about, again, more than let's say 1020 devices, it's getting more and more complicated, ok? So these basically are some of the limitations that we have right now and.06:06And these limitations are basically what telemetry is trying to let's say attack or try to avoid.06:26So now talking about advantages of telemetry or what we're going to see in telemetry. The 1st thing is that it provides mechanisms or more effective mechanisms to identify which information we want to monitor in this case by subscriptions. So each component of the device is let's say, identified by a model.06:32Low for a component and in that way we can just trigger or just go through that specific model and just.06:52Get information that we want. So each more, let's say specific about what information we want to to communicate. The transmit data is a structure, we're going to see that depending on the protocols that we're using, but basically the transmit data that we're sending is is a structured so that gives us more benefits when we want to read that information.06:59We're using in most of the situation push model that is basically that the device is sending information. So we're not connecting and asking for information, but the device is sending information to the network. So in these terms, it gives us more scalability and obviously a faster response time because we know when an issue is happening, we know it more quickly.07:18So that gives us that advantage war of.07:37React to a change or something on the network, which is actually the last benefit that we're talking about faster detection in failures. We know that right now with the increase on the use of internet, increase on the number of devices connected to the internet, the services that are running there for the service provider and even enterprise networks for whichever network you want to talk about is.07:41Important too.08:01Identify the problems faster and we can do like a quick reaction to that. So we're leaving this as attacked for you so you can have in mind actually all these let's say advantages that telemetry is trying to attack. And from here what we're going to do is we're going to start working on all these steps of that create.08:03It's a telemetry, let's say environment, so we can have our network monitor in a different way from the from CNP and all of all of that, ok?08:22So starting with telemetry, we're going to be talking about this model driven telemetry that is basically this structure or this architecture of telemetry that we're going to be focusing on. So, we're going to see right now like a high model, let's say view of this model. So we have, let me.08:33Put the pointers just 1 s. So we have the entire model, we're going to go layer by layer in in a little bit more in depth in the next slide, but basically this is the general architecture. So we have on the bottom let's say on the devices themselves, the models, these models are going to be based on jank. So.08:55Basically, this is how we're going to describe or let's say know each one of the components on a device. So an interface is going to have a gent model that describes that interface, a port, a power model, a fan, even the protocols are going to have a specific jan model to be able to understand what.09:15What that component is, ok? From there an important part in telemetry is a transport. So we're going to be based more most of the time in TCP or SSH within HTTP, which are basically secure connections or reliable. So what we're talking about in here in telemetry is that we want to be sure that the information that we're sending is actually.09:35Reaching the destination. We don't want to have packet loss or whatever, so we want to use protocols that ensure that the communication is done correctly, so in this case e.g. TCP ensures that comparing right now with SNMP, e.g. SNMP uses UDP. So we know that UDP is not reliable is I'm sending information if it reaches destination on good, if not.09:56I don't care basically in TCP is all the way around, ok? On the next layer of the model, we're going to have the encoding. Encoding is basically how we're going to translate this model into, let's say a protocol that we can share on the network.10:15That we can send information through the network. So for that, we're going to use different encoding mechanisms as HTML HTML, JSON or GPD. We're going to see those a little more more in depth later, but basically this is how we're going to represent this model, we're going to.10:30Encode all this information into a message to be able to send that information on the network, ok? On top of that, we're going to have the protocol, the protocol is basically how we're going to communicate or how is going to be let's say the connection between the the server and the devices or the devices on the server to send the information. So in this case we have two.10:48That are really, let's say important out there that is netconf and GRPC. So these are protocols that are used basically to create a connection.11:08And from there on top of that, let's say on the server, we're going to have all the, let's say the SDK, the applications, everything that we're going to be running on our servers to be able to let's say read all that information, understand it, show it in in graphs, whatever we want. So that's.11:17What we're going to do, let's say with all these SDKs and apps that we're running on the servers to understand what is happening. So this is like a high level design of the, of the model that we are working on telemetry. So now we're going to be going step by step to understand what is each of this, ok?11:361 s, one sec, I think my power phone just crashed. Sorry.11:57There you go. Sorry about that. So let's continue with this. So we talk about the model. Now we're going to talk about some of these operational modes that we have. So there are two modes that we're going to see in telemetry. So, one of them is dial out. Dial dial out is basically that the the device.12:40Let's say the router is going to start the connection to the connector and is going to start sending that. With this, we're going to have two main protocols that is one of those is TCP and the other GRPC. So in here basically the device is going to subscribe or let's say it's going to connect to the server, it's going to initiate a connection and from there it's going to start.13:01Sharing all these statistics.13:20The other model is the part of the dial in Dial in is usually what we call this, let's say pool model in which the collector is going to connect to the server and ask for information, ok? So for this we're going to use only GRPC. So, basically that that that is the the difference, ok? So most of the time.13:23The telemetry, we're going to use this dial out is a more scalable one, ok, but in some situation we can have the the other.13:43Let's say the other operational mode in there just to be able also to collect information. And with that we go to our 1st polling question.13:50Yep, so I'm launching that right now, but in the meantime, what tools do you leverage for telemetry? Right now this is, you know, there's NO wrong answers, but we'd love to see what you are using it for. CLI, Splunk, open source, Cisco Yang suite, or others. So please do.14:05Go ahead and answer that in the Slido poll on your side. Let's see, we've got a bunch of CLI and some open source, couple of Cisco Yang suite, and others, so it's pretty divided.14:21Open source is oh sorry just getting CLI now just took the leave on there. So you have a good leverage it seems of a little bit of everything. Okay.14:39Wonderful, thank you. Yeah, great. So we have a little bit of everything there good. Okay, let's continue with this. So now we're going to start working as I mentioned with each of the models, so we're going to start with the part of the encoding and transport method.14:55So for that we have, let's say, two big components of telemetry that I I'm pretty sure that you hear about a lot, that is Jank and Jank is basically how each of the, let's say, structures on a device is going to be described.15:16So e.g., an interface as I mentioned an interface has a jank model. So that jank model is basically the interface name and all the components. So the model we're going to see that a little bit more in depth and some examples about Jank in the future, but basically the idea of Jank is that is human readable. So if you see a jank model, you can understand it just from reading.15:32So it's quite easy.15:52Quite big to be honest, but easy to understand. For these models what we can do and what we can use this model to is to describe configuration, describe devices, describe administrative actions even. So in terms let's say comparing with the actual use of SNMP, let's say that is a little bit.15:55Similar to the each is what is describing each component of the device. And on the other side, we have netconf. Netconf is how we're going to pass all that information on the network, how we're going to send that information to the collector, how we're going to share all these statistics so they are useful for us, ok? So in terms of comparing this also to.16:13What we know so far is netconf is similar to SNMP in terms of either the protocols used to send information. So SNMP is used to send MIPs ords, in this case netconf is used to send all these jank and all these instructors to the servers, ok? What this is important also to understand is that is.16:33Let's say vendor agnostic, so it doesn't matter which vendor you were talking about or wherever there are janks model and the model itself the telemetry model works the same way even if you have vendor A vendor B vendor or U or whatever or different versions, as long as you have the jank in there and you're able to read it and understand it, with that you can.16:53Can get information on the servers, ok?17:13So now talking about junk models and we put in here a tree because basically a junk model is similar to a tree style description of an of an object, ok? So what do we have in here is just that, as I mentioned, this is a modeling language basically that is going to describe us what is happening on or each feature.17:15Or let's say component of the device. So each component we're going to call it a note, in this case it's going to be like a leaf in there on three and it's going to be represented by a name and a value.17:35Or it could be represented by a name and a child note, ok? So it depends on what we are talking about. We have different node types in this case the leaf note is the last, let's say the last component of three. So in this case it's just a container or it contains a single key and value.17:48So that's it, it's a leaf note. Just one value, one name, and that's it. In terms, e.g., the leaf list note, this is a list of least leaf notes. So in this case let's say it's a list and what we're going to have in the list is only let's say single values. So last e.g..18:06We have this list why why and we have a key and a value and we have another value that another note that is key and value key and value. So a leaf leaf note is only going to have let's say single components listed on a leaf so an entire complete components of keys and values. That's it. Difference from the list note.18:26The least note we can have a child that is another list, e.g.. So we can have a mix between we have a note e.g. in here that is a leaf node only and we can have another child that is another list. So that is the difference between the leaf list node and the least note. Leaf list is only leafs. List node.18:46Could be a mix between leaf and.19:06List and containers or whatever, ok, so we can have like a mix in there in the in the in the childs, and a container is just a grouping of similar components. So e.g., a container could be container interfaces. So inside interfaces we can have a list node that is gigabyte interfaces.19:09Another that is ten gig interfaces, another that is for hundred gig interfaces, ok? So the container is basically we're talking about interfaces in total, but we have.19:29Each of them separate individually. As you can see if we see it from the top to the bottom is more or less like a tree style hierarchy, ok? We have the the main structure and from there we start splitting everything into different branches, ok? So that's basically a junk model.19:39As I mentioned at the beginning, these are basically human readable, so we're going to be able to understand the junctures by looking into it and understand what is describing basically the model that I talk about and the you can extend the capabilities of each model depending on the use, so depending on.19:59So on the vendor they can expand the capabilities if they need more information to be let's say described for a component or whatever, ok? And in terms let's say I was talking also about that we can use these on different vendors, different devices. So this is also a little bit, let's say universal in terms of, e.g., if we talk about jank model.20:19A jamk model can be mapped directly into a JSON if you know Jason. JSON can be mapped into a jam and jam into python. So basically it's quite easy to change, let's say or to modify to to go from a junk model into a Python script just to use the information. So it's quite, let's say easy to do that. There are different tools in the network that can do.20:39Use this automatically, it's basically quite easy to use.20:59If we see a little bit, let's say the dependency or the one on one match between this and between an element is a class, a container could be additionally, a container name is additional key, a leaf name is addictionary, also additional key or the leaf the value is going to be the dictionary value and a list is the list, ok? So with this dependency.21:02We can convert any Jank model into a Python script and from there obviously use it in different SDKs or apps used to read information, ok? This is an example again of the three style model that we have in Jank.21:22And well, just as a quick example of how a junk module is let's say we can see it is one of these, e.g.. As you can see it's plain text is you can understand this, you can understand that where to hear about interfaces. So in here is the list interface that is going to have a key.21:34A name and a description. Another we have another interface that also have a string, a description, we have the IP address, so basically we can go from each of these components of the young model and just understand what it's talking about, ok? So it's human readable. It maybe really big, the container could be really big, but at least we can understand it, ok?21:53And now talking about netconf, as I mentioned, netconf is the protocol that we use to communicate this information, ok? So the idea of the similarity is with SNMP. The difference is that we're going to use SSH in here just to pass information. So what we're talking about is that we're going to have a reliable communication of the information.22:14That's the big let's say advantage of netconf.22:34Inside netconf, we're going to have different, let's say databases to show the information. So we're going to have, e.g., a candidate database for the configurations that we send to the devices. We're going to have a roaming configuration and start. The candidate is similar if you have worked with the iOS XR devices. The candidate is similar when we just start the configuration that we want.22:37And before applying the commit, ok? So it's going to be like what we want to get applied into the device, but it's not still running that is that is actually the running that is actually the operative configuration, ok?22:57From netconf, we can have different mechanisms to create replace, merge or even delete configuration. So from here we can manage the devices entirely and without the need, let's say to connect directly to device. In terms of scalability, we can do this, let's say in a bulk, so we can change configuration in our devices at the same time, we can modify configuration.23:09Lead the configuration event from different devices, so it allows for more, more scalable let's say monitoring and administration of the network, ok?23:29It uses RPC or remote procedure calls to facilitate the communication. So we're also going to see how this connection established, but we're talking about this reliable connection between the server and the device just to establish the connection, ok? In terms of the architecture, we're going to have.23:39Packet, let's say similar to this when we talk about a netconfect packet. So inside this, we're going to have the content. The content is basically the information that we're sending. So in this case the junk model encoded into.24:00XML encoded into JSON encode it into the Google protocol booffer. So basically the content is going to be the information that we're sending and coded in what we, whatever we want, ok? From there we're going to have the operation layer that basically the what we want to do with information. So we want to get the interface rate.24:12We want to get the host name. We want to get the syslogs, whatever, ok? So the operation is what we want to achieve from the communication.24:32RPC is how the communications to be established let's say the protocol to to make the connection, so RPC is basically the connection to the device and the transport protocol as we mentioned is SSH too, so it allows us for security and reliability. So in general, this is basically what we're going to see a little bit on let's say a packet.24:42That is created by netconf. So we're going to have as always the header, the IP header in which it's going to have the net netconf protocol embedded in there and the data that is basically as I mentioned, the junk or the XML encoding information, ok?25:02Some of the operations that we can do with netconf is basically in here you can see the get config basically to get the configuration of the device or a partial portion of the configuration. Edit config config copy, we can merge, replace, create delete. We can do a lot of things in there, ok? So we can actually manage completely the device from net.25:19Comes in from a distance, let's say remotely.25:39Hey while with this, we move to the next polling question. Good, thank you. Yep. Which transport protocol does netconf use? Give me 1 s and I'm gonna start this poll right now. There is a right answer for this one, so we'll, we'll let you know if you guys are guessing right here.25:44Give a moment for it to come up ok we've got one, we've got two and everybody's saying something different, so.26:05I'm interested to see we'll give everybody another minute. Okay, Majority of our votes, I'm gonna let a couple more come in. There we go. Okay.26:20So majority of our votes say SSH. Is that the correct answer? That's the correct answer, yeah. Yeah. We use SSH remember that we want security and reliable communication and that is achieved using SSH. Perfect. Thank you so much for everybody participating.26:35Thank you very much, thank you. So let's continue with this. So now we're going to be talking about this encoding mechanisms. We have different methods. One of them is XML. XML is quite familiar, if you are familiar with HTML and the creation of let's say web pages.26:52Email is quite similar to that is based on tax. So in this case we need to have attack an opening tag, in this case e.g. interface and a closing tag. This/basically or the.27:12The table here is like basically how we open and close the tag. And from there we're going to have also everything in XML is going to be based on these tags that say just to describe each of the components. So if you're familiar with HTML, as I mentioned, this is quite similar to to that to create a page. So XML is.27:25Again it's human readable, if we go through this, you can completely understand that we're talking about interface zero one that is the main number link, the IP address, the mask, so we can have all the information in a human readable way. So this is quite good to to understand that. So basically what we're doing is that jank model is encoded or con.27:44Converted into an XML format and this format is the one that is sent to devices to understand what is happening, ok? We can represent configuration, we can represent metrics, we can represent states, we can represent rates of the interfaces, whatever information we want to include in the XML encoding that includes the model, we can represent it using this. And with this one.28:04They're going to send information obviously to the the servers, the SDK applications and they're going to read that information to be presented and and the structure, ok?28:24Other encoding mechanisms that are also quite used out there are Google protocol buffer. But key value and Jason. I'm going to start 1st with Jason because I think that is more familiar, let's say. So Jason is basically also a way to represent a key.28:34And value.28:53Let's say key value ideas or key value value well in this case. The problem with Jason is that everything the key and the value is going to go in plain text. So for us, it's good because we can understand it just by opening a JSON, we can understand what is happening there. The problem is that the files are going to be big. So if we.28:55Want to represent, I don't know, e.g. an ASR 9910 whatever with that is a really big device, and we want to send the entire JSON file describing all the interfaces and all the protocols and all the power models and all the fines is going to be a really big file, ok?29:15So that's a little bit the disadvantage of the Json because to send this information in the network it's going to be a little bit more intensive. We need TCP with Jason because of the same. We're sending larger values that most of the time we're going to need to fragment the information. So we need a way to ensure that all the fragments are right.29:31To the destination. So that's why we need TCP.29:51If we go a little bit to another, let's say, encoding mechanism that is called protocol buffer key value, this is quite similar to jason. We have a key and a value. The difference is that the value is encoded, ok? So still we're going to have let's say half the size of a file inject.29:54Reason so because all this binary information is going to be, let's say.30:14Oh, well encoded and it uses less information, ok? Or or less space. Still we're going to have larger values or larger files, but smaller than Jason. One of the things that we need with this protocol buffer is that we need these proto files.30:18The protocols is basically the key to decode the binary that we're sending. So the binary is going to have a value in there. We need a way to understand what value is, is that actually. Also because of the size of the files, we're going to be using also TCP as the main transport protocol. And lastly, we have global protocol buffer compact.30:35Difference between this and the rest is that all the data is going to send will be sent in binary format.30:55So in terms of status, this let's say the files are going to be really small, so e.g. a JSON file that can be, I don't know, 05:10 pages long, we can have this in one because it's binary, it's just one and that's it. So it's going to be a really small file.31:01So with this we're going to have a really efficiency in terms of data rate that we can send usually to this device we just send one file and that's it. The difference between this is that we're going to use or need this protocol to understand the keys and another one to understand the values for each of the components, ok? So the protocol need to be a.31:21Little bit more bigger, but even with that the files generated by or when we encode the junk information into a GPB protocol or encoding method is going to be really small, ok? So in this situation.31:41One of the benefits they say is that we can use actually UDP with GPP because we're only sending one file, one small file. So it's easier to send that file and just not to get those in the email address, ok? So these are just some of the encoding mechanisms just to see some examples so you can understand what we're talking about. So it.31:56And here we have an example of Jason.32:16So basically what we have as you can see is plain text. We have interface name, NO zero, that's interface name. We have input data rate zero, input package rate zero, we have output data rate zero. So here is all plain text. If you can see also in here input data rate 812, so everything you can understand. This is a json file, just a snap of the JSON file.32:18I'm not putting the entire one because it's really big. But comparing this to, e.g., GPD key value, as you can see we have well here is zero because the binary is zero, but all these values 812 are actually binary values. We can see this as normal values, but actually.32:38When we are actually encoding this, it's a binary, it's not, let's say a value that we can understand. So we need the protocol to understand what this eight means, what this one means, what the two means and all that. And comparing with the last one that is the GPB normal, we're going to have the same information.32:56Let's say we're describing the same information, the interface, NO zero, but all these, all these 220-00-3000, whatever. All this information is.33:16All these that we have in here. All these name content, input data rate, key values, whatever, all that information is encoded in binary into these files or into these numbers. So basically we need a proto file that is basically that key that is going to allow us to understand what these 220 means, what this is 003 means, what this is 000, whatever means, ok?33:26So we need a way to understand all of this information.33:45But these are basically the differences on the different encoding mechanisms that we have, ok? So with that, now we have let's say the basic concepts that we are going to use in telemetry, that is the jank, the communication netconf, the protocols SSH, the encoding mechanisms that is basically.33:49How we're going to send information and now the important thing, how we configure that into the devices. So for that, I'm going to leave you with my, my friends Eduardo so he can go through the configuration and deployment part.34:09Alright, thank you, Hector for providing us with this particular.34:31Fundamentals pretty pretty good. Now, let's move forward with the configuration and deployment of the things that we already discussed. So.34:381st thing that we will be moving on is about the two main modes that we have, the dial out and the dial in. Broadly speaking for the dial out, what we need is 1st to create the destination group, I mean where we are.34:54Gonna be eh streaming.35:13The information. Second, what upgrade the sensor group, which is which information we are gonna be extracting or or needing to read and 3rd is to have this subscription where it's just binding the.35:16The destination group, the sensor group, but also the frequency about.35:36The information that it's gonna be in stream. Last but not least, as a best practice is just to validate the configuration that we have done so far. This is for the dial out, but in the other hand on the dial in mode.35:41We need 1st to enable GRPC, right? The 3rd, it's pretty similar where we are enabling or creating the same for group, the subscription, and last but not least, the configuration.35:58Right? Here also added some GitHub links where you can refer. This is mainly the young models that we have only for your assumption. Starting with the dial out mode configuration.36:13It's pretty similar. We have three different groups. The 1st one is the destination group. As we mentioned earlier.36:33That we need to go to the telemetry driven model driven context, then I specified the destination group, which particular address family it's the IPV four or IPV six address where we are gonna be streaming, and also if we want to customize with a specific port, it could.36:41Be.37:01Related to encoding section, we can use different encoding methods. It could be GB GPV JSON or self describing GPV. Regarding to the protocol, it could be TCP or UDP depending on encoding and also on the ERP section we can use the.37:02Less host name or NO TLS, just like right? Moving on to the sensor group, also all all these within the context of their model driven. So on the sensor group it's just to define.37:22The sensor path and then add the sensor path that we are gonna be extracting or requiring to have that information, right? On the subscription, it's quite straightforward. It's where we bind all together, we create a subscription.37:40Attaching the social group IDs that we defined earlier or group, and then the frequency that we are gonna be sending that information out or bringing that information for the destination group.37:57That we already defined here. So this is like the template that you can use for implementing dial outcome configuration or dial in, it's a little bit brief.38:14The 1st step, as I mentioned is to enable GRPC. You can use a particular port number, then you need to configure 3rd party application, so in case you have configured this over the global VRF or.38:33Particular VRF, the only thing that you need to do is just to specify the routing context that you are using, but also the source that we are gonna be or the interface that we are gonna be sending out that information, right? Moving on to the sensor group and the subscription, it's for.38:52Quite similar as you may see it's the same as the dial out mode, right? And here we have an example of how we can have that information from a particular model. In this example, it's just the CDP configuration.39:12Or these if you see, we have this timer of 50, all timer of one 80 and advertise only the one it's oh NO it's pretty pretty good just to see the operational status but also.39:31The configuration of a device. Next, we need to define netconf or or configure netconf and obviously it's gonna be over SSH, so.39:51Would we do that? Pretty, pretty basic, 1st we need to have the package required I mean the security package. If we don't have this package or image version from cisco.com, we will not be able to create.40:07Eh, these crypto keys that are required for SSH. So it's important to have these K nine versions within our router. Next thing it's to enable the crypto key J rememberation.40:27Or having SSH server is casefully enabled. Once we have SSH server enabled, we move forward with netconf. And it's pretty straightforward in the way we configure because we are gonna be saying, hey, we are.40:46Gonna be using netconf for SSH server, we will be having this and then if we have some particular tuning in the way we can limit how many sessions and the timeout that we are gonna be defining.41:06Pretty, pretty straightforward. Last but not least, this is optional because we have the default port for a netconf on a 30 port, but we can use this configuration snippet in order to change the default port, right?41:24On the right, we have an example of all our target configuration looks like in XML. If you see, we are here changing the timers.41:45That we have or CDP sorry for that.42:00Then we are gonna be moving to the validation of the configuration and also the brooting in case we have some issues on on that part, right? So the 1st thing that we are gonna be checking is.42:21The sensor group here important thing is that we use all telemetry model driven sensor group. We are gonna be having all the sensor group configured if you have many, we recommend to use the sensor group ID at the end of the command just to have that filtered out.42:42So what is the best practice here? Obviously once you configure, make sure that all are marked as resolved. If some of them all of these are marked not resolved, that means that either it's not supported or it's but.43:01Or or we have a typo or but written on on on the configuration, right? So please make sure that every sensor party state is resolved.43:21Speaking about the destination group, pretty, pretty similar where we are gonna be checking many points. The 1st one obviously it's the destination IP and the port that we are we streaming that information out if we are using the right encoding.43:39The transport and the most important thing here it's the stats, use the pointer, the stats that shows active because if it's not that's active, that means that the.43:57Destination is not receiving that information. So it's gonna be showed here. One other things that are important to review here with this command, which is show model telemetry destination.44:13It's the pending queue size. That means that if we are sending too much information, we should have some, some buffering here. That means that either we are.44:29Slow down by the server, because we are sending too much information and and they are saying to us hey wait a second, I need to process the information that you already sent to me or something is happening within our process that we are generating too much information that is faster than the.44:45Extending rate. So important to review all of this information here. So information to verify one more time, Correct server IP, the port, the transport that we are using, the TLS or the encoding method.45:05Depending queue size. And so one other point to to review or validate it's the subscription per se, right? It's where we all tie together.45:24So we use telemetry model driven subscription and with that, the most important thing here is active. I mentioned this is more related that if our server is confirming.45:40That it's receiving the information that we are streaming. Moving moving on on this, we together all in the subscription that means that we will be seeing the sensor group IDs, the interval that we are sending that information in this case every.45:59Three 5 s and this date.46:19Last but not least the destination group, it could be one, it could be many. This is one thing that it's important to mention here, but we can have a group of sensor groups sent to different destination group but information it's the same just for sake of.46:22With high availability or redundancy, you can have it with telemetry on iOS XR. So make sure that all the parameters showed here are showed as expected.46:42Moving on with these comments, obviously one of the things that are important to troubleshoot is 1st to make sure that the transport is fine. So 1st thing is just to validate the pink.46:59The telemetry server using your source IP forest for telemetry. If this is one thing that it's not working, there are two options, right? We have a transport issue or we have some filtering for ICMP protocol, but this is not too comment.47:16If that is fine, well, we can move forward with the next upper layer which is telemetry. And as I showed earlier, we have this model telemetry model driven.47:37Comments. But here we have an additional one. 1st, the internal subscription, it's gonna be providing us more detail about the the subscription, more statistics and and status related to the subscription.47:52Also the threshold in case we have some issues or process or process issues or hiccups within the system, we will be able to see it with the threshold and not we have the legendary tech support telemetry model, right? So.48:11All this information, it's useful to make sure that our pipeline or at least our streaming is doing good from the router point of view.48:28And one of the things that are important here is how do I know which information is gonna be streamed because.48:48We can configure, specific sensor path, but normally those are to be we can also filtering, but one thing that we can use locally on the router.48:59It's to use this command which is run mbtx then the sensor that we are using and then the frequency that we are gonna be sent out, right? And in this.49:15A example, this script runs endlessly, so it's gonna be showing you the information that it's gonna be streamed this rate, I mean the frequency rate that you just specified.49:31In this example, we are using the sensor path to check the adjacency on the bundle size pulling the bundle, the bundle manager. So the information here important is the note that it's it's coming from.49:47At this point it's router because it's his host name. And why we are streaming the host name, it's pretty, pretty normal because the lemmetry servers are not only receiving information from one particular router. So one thing that to differentiate this information from all these.50:05Different routers or vendors, it's the host. So important to to validate that information. The subscription ID streaming, which is here at this, there is NO NO relevance here, but then the information that it's important is to see the encoding path.50:24That we are using, I mean the the sensor part and the collection start time. Here the thing is that it's gonna be kind of messy because it's in a XML format, so it's not.50:42Easy to read, it can be read, but it's not too easy, so for that we have a second command that it's gonna be helping us in a more visual manner to see what is the information that it's gonna be streamed. And with that is the show telemetry internal JSON.50:57And then the sensor part that you will be using, right? So here important data it's obviously the node IDS three, but also the content that we are gonna be using. Here we are using the sensor part to see what is the optime of the device.51:17And we are gonna be having this the number of seconds.51:37And also the host name. Obviously the content of these JSON files are too big, but at least it's more visually attractive just to know if we are using or or streaming the right information.51:41Yeah, other comment that it's important is the internal command for the subscription ID, right?51:58And as I mentioned before, here are gonna be having more detail about our connection. Many details about the destination IP ports as we saw earlier, but here we have more, more detail, right? So e.g., how many packets have been sent?52:13Since the subscription is being active, when was the last time we sent that information? If we have collection groups which is expected, what is the time that it's taking to collect internally?52:34I mean the minimum, the maximum, the average all the data, how many collections have been done? I mean, if we are streaming every 5 s, that means that we are gonna be having twelve collections per minute.52:50And so on, right? So imagine how how long has been this active, right? More information here obviously if we have different packets or send errors and drops, we are gonna be seeing this. So important to see this information or to validate this data.53:05If we are sure that our destination IP it's fine, there is NO problem in the transport. We are using the correct encoding.53:25The correct VRF in case we are using a specific VRF and then the the last part of this commands are good because it's gonna be also showing you the last collection start and end for every sensor pack.53:34So we have many information there, more, too much detail and and this is only worthy to review when we know that everything on top of that, it's fine. So e.g. here, the information that we have is.53:54Related to the TCP packets so we know how many get leads get get gets leads, data list in data get worked and encode and same packets are over here if we have some errors or some different jobs.54:11So far in this output, everything looks fine and we also can know that it's fine because we are receiving the information from on the telemetry side. So with that, we are reaching our last polling question.54:27Right, let me go ahead and get that up. Which encoding structure does Netconf use? JSON, XML, GRPC, or plain text, and it should be launching right now?54:46Awesome and let's see. So far everyone's getting the right answer. Let's see if they, oh, we got a few that's that's strain, ok? We're between JSON, XML and we had one vote for GRPC.55:02Let's give another minute and get some more votes in here. Okay, and our majority is saying XML. Is that the correct answer?55:18That's great. Perfect, thank you. To stream this information out the router using netconf. Alright, well let's move forward with the funny part of this presentation, which are the labs.55:35We are gonna be showing you two different tools that we have other who consume telemetry using dial out. But the 1st one obviously it's gonna be Cisco young suite. This is a tool that we have that you can use to learn.55:54Or to tune or modify or optimize your telemetry because it helps you to make sure that everything is working fine before you move to a particular telemetry server. So.56:14What the topology is, it's pretty, pretty basic, we are gonna be having a PE two router connected over the cloud, reaching with a TCP connection to the Cisco server.56:33Oh, it's quite straightforward, there is NO more stuff over there, but what we are be doing on the router PE two is just to configure three things as we said earlier. We are gonna be configured two different sensor groups, which are.56:49System resources and routing info, the destination group, it's gonna be the server and this subscription is gonna be named as a Cisco community. What we need to do on the other side on I mean on the Cisco Jan suite.57:07It's just to select the protocols, then the GRPC telemetry, and then we are gonna be listening to a specific part. With that, let's start. So.57:251st thing we move to the section and paste this here just to have a better readability, we move to the telemetry model driven context then we specify the destination group from in a specific VRF.57:41We include the details there about the IPV four, the port that we are gonna be streaming. Remember, this is the IP of the Grafana sorry not the grafana, the and the port that we are, it's.58:00What we are gonna be sending that the the is gonna be reading. So we proceed to commit that information. Then the next step is just to define the other two.58:17Oh, well, before doing the committing the the next sections, we need to make sure everything is fine here. So, as you may notice, we have here the IP, the port, the encoding section, the transport, which is GRPC, and the state.58:33Some collection statistics, but as you may notice, the state is NA because we are not receiving yet from the from the telemetry server. I mean the server.58:54Next section is just to define the sensor groups, as I mentioned earlier, we are gonna be having two sensor groups. The 1st one is resources. We are where we are gonna be sending particular information to uptime memory infrastructure interfaces related info.59:09Information CPU utilization and so on. 2nd one is the routing information. We are gonna be streaming information related to ISIS and.59:29With that, we proceed to commit and validate that all these parts are being resolved and we do this with the show telemetry model driven sensor group. In this case we have only two, and we can use this command without type.59:40Yeah, as you might might see, we have all the sensors resolved related to the information, the same thing for the system resources and server group ID, we have all of them resolved.59:56So far, good. The only thing that it's pending is that it's in NA for the destination, but we are gonna be fixing soon. Last but not least, we need to specify the subscription.01:00:12Where we are binding together the sensor group IDs with the destination IDs. Remember to use the same names with the capital letters if you have. Remember this is key sensitive.01:00:28The intervals important we are gonna be sending this information every 5 s.01:00:47We use the destination group which is server and we proceed to commit. If we happen to verify that everything is fine here, it's not gonna be, let's let's.01:00:52The check with the show model driven subscription. So the sensor groups are resolved, but the subscription is NA in a state NA. The sensor group and the destination groups are fine, but the only thing that it's not ok.01:01:08It's the state, so we we need to activate the in order to listen or to receive that streaming over that particular port. So we go to the.01:01:26Protocol section on the left left side, we click on GRPC telemetry. Since this is a local server, we are gonna be streaming or listening on our zero.zero.zero IP, the port that it's being streamed, which is 50.01:01:42Oh 51. So sometimes we we just click start receiver and information is gonna be starting flowing. Sometimes not, well here it can take some seconds if we see the status is still.01:02:02On N A here I I click again just to start receiving and eventually we are gonna be seeing that information. It's gonna be prompted on the screen. Here we see that the status changed to active and you can see.01:02:21The information that is being streamed.01:02:41Information to match obviously the subscription we see the name of the subscription but also the the parts that are being sent and the numbers of the information related to every part that we define. We can filter, but.01:02:47This is not all the scope of this one. The scope of this one is to show that the note it's streaming every 5 s information that we specify. If you see it's refreshing more or less every 5 s.01:03:02Yep, it's extremely if you see the information can change, here we have the the the timestamp itself very pretty important to have the timestamps because we are gonna be using the timestamp in order to do some graphics or statistics in other tools. So here it.01:03:27It's important to, to also have the the timestamp.01:03:46So with that, we conclude our 1st lab demo for telemetry where we started or we, yeah, we started to to stream information and consume in a particular server. Here.01:03:49We are not storing information. We are just receiving watching and that's time, but in the real life, we normally see this information in a more matter. So with that.01:04:07We have moved to 2nd lab, which is also using the similar approach of dial out, but here we are having an open source stack which is telegraph influxdb and Grafana.01:04:23And, the the topology it's pretty straightforward. The only thing that we are just changing is the IP of the server that it's where it's hosted this particular.01:04:40Which consists in three different containers. The 1st one is a telegraph which is responsible to receive the information on the ports that we are streaming. 2nd it's influxdb.01:05:01Once we have that information received, you need to save it in a time based database, and the 3rd one is which is responsible to display that information in dashboards. So we can create a load.01:05:17Dashboards just to consume this information, to troubleshoot, to analyze what it's having if we have something typical or old of a particular threshold, right? What we are gonna be using on the PE two router, the only thing that we are using is.01:05:36The destination group, but we are gonna be reusing the system resource and the routing need for information for sensor group. The subscription, it's gonna be the same. Important thing here is that we selected word 170-00-0015.01:05:5657 oh oh oh oh in order to receive that information. So let's start with the lab. Oh.01:06:15One thing that I mentioned before, we need to change a little bit of the destination groups that we are using. Let me show you what it is because here it's kind of not readable friendly, so let me do show config. If you see with a specific.01:06:30A new.01:06:50Destination group, which is Grafana over on a particular IP. Same thing as before, we are gonna be using self describing GPV with protocol TCP, and we are gonna be removing the destination group from the subscription, we are just.01:06:52Was changing the destination group to use the new.01:07:12Grafana to be used. So with that, we maintain the streaming information that we have before. So this is also good if you want to change from server to server.01:07:15You can do it in just a few lines. Then the validation is quite straightforward. Here we have active status, this is good. The port and the IP, the description and the sensor groups also.01:07:32Looks fine, so let's see how it looks on the other here I prebuilt.01:07:50Dashboard in Grafana where we are gonna be adding different.01:08:11E.g., we can monitor the the different information that we that we are streaming, but also we can have different timestamps for that. So here we have CPU usage, all time brand usage.01:08:18If we have adjacencies for a particular interfaces.01:08:36And the packets that we have sent on a, on an interface or received and also route count. These are just a few examples of the information that we can visualize in this dashboard or this solution which is open source and.01:08:40One thing that it's good you see how the time goes on. So with that, let's figure some changes on the router, let's configure a or let's enable this bubble internet to 51.01:09:00And with that we are gonna be provoking ISIS neighborships or an ISAS neighborship, but also throughout the change. So if we happen to to validate that we enable ISIS on that interface, there we go, we have.01:09:17An adjacency or neighborship on ISIS to another router, and we received some other few routes, but let's see how it looks on the dashboard. So if you see the road counted also we see some packets.01:09:35Going in and out to that interface. And if you see this information it's being streamed every 5 s so on the adjacency section, we see also that it's true and the packets start flowing.01:09:52Across that, that interface. If we happen to ping one of the learned routes over that, we will be also seeing.01:10:09That it reflects on, on Grafana. But let's do a pink that it's 1000 with an interval of ten milliseconds.01:10:24The the ping is working and you will be seeing dramasic automatically change on the pings or the, all the packets that are going and and returning to the router, and with that you can also notice in the network some events that are unexpected.01:10:42Example, changes on the routing or something expected, so it's good to have this for monitoring purposes because this is just a ping reply or request and reply. We are gonna be seeing.01:11:00Nearly the same number of packets incoming and egressing this interface. Next, we are gonna be disabling.01:11:20This bundle internet in order to trigger the change that we are not gonna be having routes but also we are not having the adjacency over that bundle interface. So just to see how fast this is this.01:11:35Based on on.01:11:55This dashboard. So if you saw, the, the SSC is not lower showed, but also the routes that were learned are zero and the packets that are incoming and ingressing that inference interface remains on the same.01:11:57Number. If we rolled this configuration, very pretty easy just to enable this interface and make all available, we see here that we have the ISIS neighborship, so eventually here we are gonna be seeing in the next nap.01:12:20That information to be streamed, as you may see, we have the the routes back, and also the adjacency back.01:12:39And with that, we conclude all a second laboratory where we have these telemetry information being consumed in a friendly manner where you can monitor and even troubleshoot any trend that you see on your devices.01:12:47Just to have a brief summary, this is the configuration that we use on this lab where we have the sensor group eh configuration, the two different destination groups and the subscriptions that we.01:13:11Used, right? So pretty pretty pretty straightforward. Additional resources, well we added some information here on the.01:13:30Tools that we use today. The 1st one obviously the Cisco server where you can use the developer cisco.com domain in order to consume more information about how to use it, how to tune your sensor paths, so.01:13:46Good to to visit all pages, but also the the open source stack for telegraph influx tv and Grafana, we have some information also from from XR docs.io good to good to review them, but also information related to Grafana and influxdb.01:14:04And with that, we move on with the Q and A section. Wonderful. Thank you so much for those demos and labs and everybody's participation in today's session. We just had a couple questions that came in just about our slides, the PowerPoint and.01:14:24The video recording of today's session will be available in community.cisco.com after the session, so please do check thereafter you should get an email.01:14:43I would ask that you take the survey upon your exit of this session. There's just a few questions. And then we got, you know, a suggestion for another session that's coming, you know, that we could do for the next time, which I think is a wonderful thing. There is space in the survey to add those suggestions as well, so please feel free to add that again in.01:14:52Your survey?01:15:12If there's any questions, please feel free to put it again in the topic or you can unmute you whatever you would like. We'll give you a minute. But if not, once again, thank you so much for joining our session today, again slides and presentation and a recording will be available in community.cisco.01:15:13Dot com and you can find that in a couple of days after the session.01:15:34Again, thank you to our presenters, Hector and Eduardo and all of our question managers. We appreciate you being here. And again, thank you so much for being part of this community live session today. Okay, with that, we'll let you all go. Have a wonderful day.01:15:38Thank you very much for everyone. Thank you. And feel free to follow us on all of community social channels that we have coming up here in the community. Thanks so much and have a great day.01:15:56