Hi, everyone. My name is Mark McBride, and I'm a customer success specialist. In this video, we'll examine how to run AI on Cisco infrastructure and how we can connect an on premises large language model to intersight.00:06Using MCP. We're not only discussing cloud AI, we're looking at deploying intelligent local agents that securely interact with your existing infrastructure, ensuring your data stays under your control while boosting operational efficiency in your environment.00:20Nothing about your infrastructure such as your prompts, your inventory or alarms ever leaves your network, in fact, at all. And because it's an open weight model that's running locally, there is NO per query API build. Let's get into it.00:37Today we will start with a quick refresher on the core value of intersight. Then we will move into the model context protocol, which acts as the glue for our AI agents. And then we'll conclude with a look at a live architecture that shows how we can run these models locally to save on costs and enhance security.00:57Intersight has always been about simplifying the complex, offering a single point of management for global policies and actionable intelligence as a whole. Whether you're running as Intersight SaaS or via a connected appliance, the goal remains the same. Truly, comprehensive automation and intuitive experience.01:18By layering AI on top of this foundation, we can turn that actionable intelligence into autonomous action.01:39Intersight already makes every aspect accessible through one unified, consistent, and thoroughly documented REST API that includes support for O data queries. And that matters enormously for AI so that the model doesn't need to customize any integrations per device.01:47So it gets one clean searchable surface for compute, network profiles, firmware, and even alarms and advisories across many environments today. The richer and more even the API, the better that an AI agent can reason over your environment.02:04So, let us take a moment to discuss how MCP actually works. Think of MCP as the USB C for AI, the standard that lets AI agents connect to tools and data. Just as USB C set a standard for connecting peripherals to hardware, MCP standardizes how AI agents link to tools and data. We run the.02:25MCP client an interface for our agent or LLM and the MCP server, a translator for practical use.02:46This architecture truly enables us to keep the logic local, connecting to tools via standard protocols without needing to expose sensitive data to external 3rd party cloud APIs. So what could Intersight look like when MCP is involved?02:55Here's a practical workflow demonstrated in real life. When a user asks how many servers are powered down, the LLM does not merely guess. It identifies exactly which tools are needed, boots up the MCP server, and issues a comprehensive API call to fetch real time telemetry from intersight.03:14The model analyzes the data locally and then presents a precise, unambiguous answer to the user. The loop is fast, secure, and entirely contained strictly within your environment.03:33So now, let us discuss the overall application architecture of our inter site chat application. 1st, we rely on the streamlit based user interface, a browser rendered chat interface, and on the sidebar, there is the dedicated area where you can enter the inner site key ID and the inter site PM secret file, as previously.03:49Described. These credentials reside only in memory at all times. They are never written to disk and they are repushed from the MCP server.04:09Then we have the orchestrator. This is the tool calling loop. It hands the model, the list of available tools, runs the model, executes any tool calls that the model asks for. It then feeds back the results and repeats this until the model has an answer. Then we have the.04:20CP server, which runs on Node.js, and it exposes 18 type tools that the model can call. E.g., things like get chassis information or get alarms or get physical server information. And then this connects to Cisco Intersight.04:40And this uses your account, which is reached over HTTPS, and the server auto detects whether you're using a version two or version three key within your PM file. And the API key permissions authorize interactions with Intersight. Then Alama runs.04:59It's on a local NVIDIA GPU inside your own personal machine.05:19As a result, the large language model runs here directly on your local hardware via an OpenAI compatible endpoint. The orchestrator talks to it in exactly the same way as it would to a cloud API while the traffic never leaves the box within the device now.05:23By moving the model inference to onsite server resources, we eliminate the per token costs tied to public cloud AI services while also keeping our infrastructure data completely air gapped from any public model providers. Now let us observe step by step how all of this functions together in a practical demonstration.05:41Before we dive into our inter site chat app, let's take a moment together to explore the resources in Intersight. So now at this moment we are logged into our Intersight account.06:04We will navigate over to the server section. Here we can see that we have three servers in total, one that is powered on and two that are powered off. Our intersight chat application is going to run on the server that is powered on, which is the UCS IM pod-one-one.06:16This particular server has access to a GPU that is operating in slot two of the chassis. And we can confirm this by opening inventory and then GPUs now. Here we can see we have an available NVIDIA L 40 S GPU. So let's log into our server now.06:32And thoroughly verify the resources there right away. So we're now logged into our server and we're gonna run the NVIDIA SMI command to verify.06:49The GPU that is available to the server. And we can clearly see that we're now able to observe the Nvidia L 40 S GPU. The server is the host that is running the two docker containers that are currently dedicated to our chat application. We currently have two docker containers.07:06The very 1st one here is the Intersight chat application, and this is what is currently running the streamlid application as well as the MCP server that will be connecting to Intersight. We also have another container that's running our Alama services that will be used to connect the model in use with our L 40 S GPU.07:25So from here, let's go ahead and head over to port 8501 where our inter site chat application is running. So for the moment connect to port 8501 on the powered on server, and this is where we can choose which model.07:45We'll use for our chat application. Here we are using the OpenAI open source model, namely the GPT 20 billion parameter model for our chat application.08:05We'll need an Intersight API key ID and an Intersight Pem file to connect our chat application to Intersight. So let's proceed promptly to create our own API key now. From this point we will return to intersight, and then under API settings and keys, we will generate an API key. Next, we will.08:19We'll provide a description. We will be using the open API schema version three. Finally, we will carefully select an expiration date as well. We'll click generate now, and then this will give us our API key ID as well as the secret key.08:39So we'll save our API key ID. Here is a file intended for use with our chat application for the purpose of this demonstration. Next, we are going to download our secret key as described. It's important to keep in mind that these items function essentially as credentials and you should store them somewhere.08:56Hey.09:16In this demonstration, we will simply use them as local resources without extra steps. In order to use them with our chat application, you will need to save this securely as a.pem file. So we'll go ahead and carefully change the file extension from.TXT to.pem.09:17For our chat application set up, since we are only collecting information from Intersight, we may use a read only API key.09:40Since we are not making any changes nor pushing any configurations, we're only pulling information. So we will proceed right now to say that we have downloaded the secret key. We will now head back over to our chat application and we enter the API key ID indeed.09:47After that, we will upload the site secret key, which we have just saved as a PEM file. Next, we will select the test connection option. Here we can verify that we are now connected to our inner site account.10:12From this point forward, let's consult the chat and ask which tools we have available for our use right now, please. So what is happening here is that we can clearly see that it is using one.10:27Of the tool functions, mainly the one called lists tools. One tool was indeed called, and we can observe that the list tools function was invoked, and this is providing us with all of the tools that are available for use with our chat applications. Therefore, we can use it to test a connection to Intersight.10:46This confirms that list tools is active. We can use a tool invocation to retrieve server profiles, physical servers, chassis information, and alarm summary, and so on. These are all of the different types of collections that we can pull from our chat application.11:04So let's proceed to obtain additional details about the servers we saw in our inter site account for further review. So we will ask it to provide a comprehensive, detailed and organized list of the servers details in full, including the CPU and memory resources.11:20So what's gonna happen now? It is going to utilize the tool for get physical servers, and we can clearly observe the API endpoint that was used to retrieve this information today. And we can observe that the status is currently 200.11:42Then we can see in the chat that it shows us the three servers that belong to our inter site account, and it shows what we had asked for regarding the available CPU information. So we can see each one of them possesses two CPUs with 64 cores, totaling 128 cores, and it also includes the memory information we.12:01So each of these servers has 256 gb of memory, and it even provides us with a bit more information. All of the servers are blade units, and they are housed in the same chassis. So let's go ahead and obtain further details about that chassis.12:22So we'll ask it to list chassis details and what is currently populated exactly right this moment. In every slot, this process will trigger several tool invocations. It will run tool invocations to retrieve the PCI.12:41Information as well as tool invocations to retrieve the compute blades and the chassis information.13:01This sequence shows what hardware is being used in each slot. Specifically, the slots labeled one three and five are occupied by w series servers observed in intersight. The three blade servers, and slots two and four host our GPU PCIe nodes.13:08And then we can clearly see the math it's using to tell us that there are three free slots in this chassis, and we can confirm that by going back to Intersight as you can see right now. Let's take a look at our chassis.13:28And indeed we have three slots available, three slots that are in use by servers, and then the two PCIe nodes that are currently utilized in slot two and slot four at this particular moment.13:43So in this demonstration, we were able to observe how we can fully utilize all local resources using MCP and a locally running large language model to operate AI on Cisco's enterprise infrastructure. Thank you for watching and I hope you will join us again for the next video.13:59
