AI Ready Data Center Infrastructure & Use Cases​: Quick Recap and Live Q&A

Oh, and welcome to today's expert insight series, which is a recap and live Q and A for our recently.00:00Concluded series on AI ready data center and infrastructure. Today we will have several Cisco CSS engineers.00:20That will be presenting and providing demos if needed. We'll go through several of these names that you see on the screen here. We've got Mark mcbread, Dave Cole, Jorge, Ayella.00:35Priyanka Sukija, Brian Morrissey, and more if needed as panelists today that will speak to your questions and engage you in chat. Now our agenda today will cover these five topics here.00:51Which represent previously aired sessions in this series and notice that number six at the end of all of these sessions, we will have a live Q and A you'll see a dedicated slide for that. At that time, you're welcome to post your questions to our.01:10Cisco Slido that we have running will provide the join information in a second. Alternatively, you can post your questions using the Webex Q and A feature, and if you need to.01:29You can always put those in chat, in the Webex chat. We will be including links and other information for you. Now here's our link to the Slido for today's session. If you can navigate to slido.com.01:44And enter our meeting code of April eight QA, April eight QA, and the passcode is the same, April eight QA, which is just today's date and QA.02:03And if it's helpful to you, you can use that QR code on the right hand side to use your mobile device, to scan that QR code and have it produce a URL to take you directly to that Slido event. Now without further ado, I'll turn it over to.02:21Who are our 1st team presenting today and they're going to cover the AI maturity model.02:41In Cisco stack. I'll let that team take it away. Hey, thanks Joe and good morning, everybody. My name's Mark mcbrede, so I covered as the presenter on the 1st session that got us started on our AI journey here. So we started off with a session on what is the AI maturity.02:47The model as well as the Cisco portfolio and, and the AI stack. So today we'll take a few minutes to recap what we went through in the session, what we think were some of the highlights and some of the key takeaways, and then we'll also share some additional some links and some resources for additional followups.03:07So one of the main takeaways from our session was that it's it's not one person that can make an AI project happen. It's gonna be a collaborative effort. It's gonna come from multiple teams. It's gonna come from the leadership to help show you what are the targets that can happen.03:28Most benefit with AI leaderships can help set the goals.03:47And then with your data engineers, these are the ones that are making sure that you have valid and claimed data. They know where the data is stored. They know that the data is gonna provide the appropriate answers for the models and the inferencing that's gonna be used with your AI workloads.03:52There's also data scientists and MO engineers. These are the ones that will be focused on helping to build the model, testing out the interferencing. Then we also have application developers that'll be creating the apps that will be using your new AI models or perhaps you have an existing application.04:12That you're going to integrate AI into that application. And then of course your IT operations folks, these will be the people that are standing up the new hardware supporting the new servers with GPUs, supporting the the back end fabric that's gonna support Rocky V two that GPU to GPUs.04:32Communication so it does take a team of people all working together to to help make it happen.04:52So during our session, we talked about the AI maturity model. So this comes out to five phases that can be viewed from two angles, one being the AI business model, the other one being the AI infrastructure.05:01So starting with the 1st stage, this is the awareness stage. So from a business perspective, you're learning about AI, which many of us have been doing. We understand what AI can do. We're thinking of ideas. How can we make AI work for our business? What are some of the applications that can take into.05:17Manage from AI.05:37From an infrastructure perspective, nothing's changing here. The business runs as normal. You're looking over the hardware you have today and you're starting to think, are there tools or is there available hardware that we could possibly use today to start looking into using AI?05:39Then we go into the active stage. So here's where we start to take those ideas and then we start to test them in a developer area or a staging area or maybe start creating a specific AI test area.05:59Where you can take your ideas and then start putting those into a test environment. Maybe you leverage some of the hardware you have and you buy some additional GPUs and then you start testing.06:15Once you've completed your test, you start to go into an operational phase. So from the business, you now have your 1st AI workload into production. Maybe it's a new app or maybe it's AI integrated into an existing application.06:31On the infrastructure side, you've got your GPUs, you've got your AI clusters, you've got your back end fabrics, you've got storage that's dedicated for AI. And then we go into stage four systemic.06:48You've now have an idea of how to implement AI into the applications you have today. So now as new apps are being developed, you start to think about how you can include AI within the the infrastructure and building AI just becomes the normal process of building new apps.07:03On the infrastructure side, you start to scale. So you've got a few wins, you understand the size of the infrastructure you have today, and you can rinse and repeat and scale out that infrastructure. Then from there that takes you into stage five transformational. So this is where AI is just it's a built.07:24Into the DNA of your business as new apps come out, AI is considered a a high priority. It's the 1st thought.07:44And then on the infrastructure side, now that you've implemented AI, you understand how to scale that out, you know how to add additional GPUs, additional storage, additional network layers, and now you've got a scale unit that you can work with to help expand that.07:52So one other thing we talked about during our session was an example of what happens in a chat prompt. So here we have an example of a chat prompt of where someone is asking a prompt to help organize a meeting. It's asking who should be in the meeting, it's helping or it's asking who can help.08:13Scheduled the meeting who should be a part of it. Maybe there needs to be some documents that need to be reviewed during this prompt. And all of this comes down to what are tokens. So everything in this prompt gets converted into a token that interacts with the model and then we get output tokens or the actual result.08:33That comes back from the prompt. So why this is important because depending on how your users are interacting with this prompt will influence how many tokens are being used and consumed by the models or for inferencing. So if we take this question here, this request, and if we do a one shot prompt.08:53We asked the model to help us organize this meeting, help us to schedule it. It gives us back a direct answer, so we can call that a one shot. But there's additional options here where maybe we're gonna add photos to this prompt or maybe additional documents or files that need to be reviewed.09:13This is going to increase the number of tokens. Maybe we ask it to do deep reasoning, so we asked the model to go through all of the different ways that we can schedule this meeting, who should be included, go through all of the scenarios and then give us back what's the the best way to schedule.09:32These meetings. So with deep reasoning, this is gonna increase that token count by either, you know, possibly ten to 20 times more tokens because we're asking the model to do more.09:52A lot of us have heard what's going on with agentic. So now instead of just using a chat prompt to ask it a question and get a response back, we're now moving into an agentic era where now we're asking the prompt to actually do something. So instead of just giving us back an answer, we're asking the prompt to.10:04Go and perform some type of action. So maybe in the same scenario.10:24We go into agent mode where an agent can then go look up in a directory who needs to be included in this call. It can also go back and maybe review previously your recorded calls, go back through and summarize documents and then actually send out the invite to everybody that needs to be in that meeting.10:28But that's known as agentic and it does come down to managing these tokens, so understanding how the tokens are used, how they can be used differently depending on what type of mode that you're running these prompts will ultimately affect how, how many tokens are gonna be consumed.10:48So from here we also covered a number of of demos during our session and for that, I'm gonna pass it over to Dave who helped cover that. So Dave over to you. Thank you very much. So I covered a number of demos in my session. I'm not going to really get into those.11:10Again today. I'm just gonna go high level.11:30I expect some of you are familiar with our demo cloud or D cloud as we call it, but in case you're not or if you, you know, you're kind of looking for inspiration on what you can get out of it, I just that's what I figured I would just touch on today. So at the very start.11:34It's pretty easy to get to. You just go to dcloud.cisco.com. You are going to need to log in. In some cases possibly in a lot of cases, you may need to get sponsored to be able to run some demos.11:50You'd need to talk to your account team, just ask them, hey, I want to do this demo and they'll spin it spin one up for you. But ultimately, so we, you know, we log into our demo cloud or D cloud and probably the easiest thing is just to go to catalog and just start doing searches.12:11So, this is slowly coming up here oh ok that would normally take you to bring you to this page, our catalog page. The search box is right here.12:29So I started, I did a couple of demos that were based off of intersight. So if we do a search for intersight, we're gonna see a number of really kind of interesting demos here. And by default, it's going to sort by last updated, which is when we're talking especially when we're talking AI, there's gonna be some pretty interesting stuff showing up there at the top.12:48List here. In particular, so the one demo I did was accelerating AI innovation with an AI ready data center. So you're gonna also see here, there's the concept of scheduling.13:09A demo. So this is actually where you schedule it and on our side there's a bunch of provisioning that's automated provisioning that's gonna happen to spin up Vmiss and containers in that dedicated team. But some demos are actually live demos.13:23So they'll just be like a view button and you should just be able to click on view and you can get started. So as an example, this accelerate AI innovation, you can see here it's touching on a lot of different hardware and software, as Cisco AI pause UCS servers, Nexus dashboard.13:40It's touch touching on red hat or OpenShift, Splunk. There's a lot going on with this particular demo.14:00When you go into it, so you you go into it it it gets a little bit deeper into what kind of hardware is available, and what you're what you know hope to learn from the session. And you'll also see here a resources tab. This can often be pretty.14:08Important there's there could be documentation in here that's gonna help you.14:26Just, you know, help you get the most out of the demo. And then ultimately you've got, for instance, the view button here. And if you really enjoyed it and you're gonna go through it again, you can also favorite it. Another one. So and as also as an example, this particular one, this live demo, if you go into it, it's.14:31It's going to have a bunch of different dashboards you can click on and you know with different.14:51Aspects of what this entire demo was about. You know, this particular demo gets right down into playing with an AI database and, building a AI assistance and all of that. Also, another, you know, another interesting one here is simplify AI deployments with AI pods using X.14:56So, you know, if you're if you want to play with a little bit of, you know, hardware before messing with yours, this is a great place to go.15:17Another option would be to search for something as nebulous as AI. So this is going to bring up a lot of stuff in our, our listing these days. Of course with last update, you're gonna see a bunch of you know campus networking and securing your hybrid workforce, that sort of thing. But we also have, for instance.15:27We have a live demo of secure AI factory within video.15:47That gets into, you know, our AI pods and networking underneath of it, observability with Splunk, AI security operations, all of that. Again, this is in this case, this is a live demo. There's lots of security demos on here, lots going on.15:52Another one I want to bring your attention to, and fortunately, it doesn't, it doesn't come to the top of the list, but this is one I think that for especially enterprise sized customers that are concerned about, you know, you're gonna be hosting maybe your own models, and your.16:10You're concerned about, you know, maybe your employees or maybe there's a a public facing.16:29Agent that your customer is gonna be interacting with, and you want to ensure that people aren't trying to break through it and you know make send extract data that you don't want to exposed. AI defense is a very interesting security.16:36Product that Cisco is is bringing up. I encourage you to go take a look at AI defense. And with that that's it for me. Let me stop here.16:56And, back to you ok thanks Dave.17:16So alright everyone, let's do a recap of that. We covered it around data center networking for AI workloads. As a reminder, the core idea here is that the network is not a secondary concern in AI.17:34Infrastructure. It's our 1st class citizen, so GPUs don't work in isolation. They constantly exchange massive amounts of data during training. And if the fabric can keep up your computer starts. So over these slides, we will quickly revisit.17:54The key concepts the networking requirements that AI workloads impose.18:14The protocols that make lostless fabric possible and the tools that give you visibility into what's happening at the fabric levels. So think of this as a fast pass through the highlights, the things you will want to remember when designing or evaluating an AI ready network.18:20Okay, so to support a successful AI ML strategy, we must look beyond standard networking. So as you can see on this slide, the infrastructure requirements are split into three different categories. Training told.18:43Inference and general requirements.19:01When we build this, we have to support four specific network planes remember, the management network, the storage network, the front end inference network, and the high performance backend inter GPU network. So beginning with the training and tuning requirements, I mean the.19:04Backing and storage plants.19:24So for the training and tooling phase, we are dealing with the most demanding workloads, so this is where you enter GPU backend network and storage network live. So as you can see the points at the bottom, the blue points, .1, high bandwidth, training models involves moving massive.19:26Dataset across the fabric, so we need high bandwidth networks to ensure that the data pipeline is never the bottleneck. Moving to .2, non blocking lossless in AI network.19:46Inconsistency directly impact model accuracy and increase training time. So we require a non blocking lossless fabric to ensure that data packets arrive in order and without loss.20:00Well Moving to .3 congestion management because training traffic is bursty. We need intelligent congestion management to detect potential hot spots and redistribute traffic dynamically across the network. So talking about.20:17Inference requirements, I mean the front end plant, this is the the front end network that your user or your application interact with. Talking about the .4 low latency.20:36Online training, which is about throughput, inference is about responsiveness. So real time AI applications require extremely low latency to ensure that when a query is sent, the model responds instantly. Moving on general.20:50Requirements, I mean the management and visibility layer, the general requirements that apply to your entire infrastructure, including your management network. So talking about the point .5 configuration to scale, we cannot configure devices manually. We.21:10We relied on validated template based configuration to ensure consistency and easy of deployment across the data center.21:30And last one, the visibility point. You cannot fix what you cannot see. So we need comprehensive visibility tools that provide real time monitoring allowings allowing us to detect issues and troubleshoot performance degradation before they impact the AI models.21:40So as a conclusion, by aligning aligning or network architecture, I'm talking about management, storage, front end, and backend with these specific requirements.22:00We create a robust environment that balance the raw speeds needed for training with the responsiveness required for inference all while maintaining operational simplicity through automation and visibility.22:13So let's move on here. Now let's pivot to the front end network. While we have discussed the specialized needs of the backend and storage planes, the front end is where you AI infrastructure interfaces with.22:31The rest of your enterprise. So a common question I hear from my customer is do I need to build a completely new network from scratch to support AI? And the answer is not necessarily. E.g., we can leverage existing infrastructure. So as you can see on the slides, you have.22:50Two primary paths for your front end architecture, and both are like familiar to you as our networking engineers. So the option one.23:11On your left, I'm talking about ACI fabrics. So if you are already running a Cisco ACI environment, you don't need to reinvent the wheel. You can repurpose your existing ACI fabric for AI workloads because ACI is built on the Nexus series, it.23:21Provides the the robust high availability and low latency foundation that AI inference requires. And option two of the right VXLAN EVPN fabrics. Alternatively, if your environment is built on a standard VXLAN EVPN architecture.23:41That is equally capable of supporting these workloads. So it provides the same flexibility and high spike high speed connectivity supporting 100 gb and 400 gb links that modern AI application demands. So regardless of which fabric.24:01You choose. The glue that brings this all together is the Nexus dashboard. So Nexus dashboard is your single plane of glass. It sits about both ACI and VXLAN EVPN fabrics, providing the deep visibility and operational consistency you need. So what.24:20There you are building a new greenfield deployment or you are repurposing your current data center, the next dashboard ensures you have the telemetry and troubleshooting tools necessary to keep your AI front end performing at this big. So as a conclusion and the takeaway here is.24:40Is simplicity and reuse. You don't have to build a silo for AI. By leveraging your existing ACI or VXLAN EVPN investment and centralizing your management through the next dashboard, you can deploy a front end that is not only highly performant but is.25:00Also operational efficient.25:20Okay, let's move on here and now let's achieve our focus to the most critical part of the AI infrastructure when talking about the backend network. So this is where you, where the heavy lifting happens, the inter.25:23Their GPU communication. So when designing this fabric using our nexus 9000 fixed series, we generally see two distinct architectural approaches. The 1st one's the classic design. On the left, we have the classic design. This is the traditional way we.25:43We have connected server in the data center for years.26:03Each GPU node is connected to a leaf switch or a pair of switches using VPC for redundancy. The key characteristics is a BGP based fabric that is is strictly non blocking when one to one over subscription radio.26:07Talking about scalability, it supports massive throughput with options for 400 gb or even 800 gb connectivity at the leaf and spine layers. It's a very reliable.26:27Proven model for general purpose, high performance computing, and on your right the the rail optimized design. So whoever the industry is rapidly moving toward the rail optimized design show on the right. So this model we change how.26:42How we map the GPUs to the network, instead of connecting a note to a single leaf, we distribute the connections. So the logic is that if you have a server with eight GPUs, each individual GPUs is mapped to a different leaf switch.27:02And the benefit is, this rail approach significantly simplifies communication between GPUs across the entire cluster. So it ensures that the traffic patterns of your collective communication libraries like the famous NCCO foreign video are perfectly.27:21Be aligned with the physical network topology providing a truly not blocking one to one path for every GPU. So the critical factor here are the communication libraries, so it's important to note that these design choices are not arbitrary. The selection between a classic or.27:41Rail optimized architecture depends heavily on the collective communication library you are using. So these libraries are designed to optimize how all your GPUs communicate simultaneously.28:01To achieve the performance targets of your specific AI model. So choosing the right physical fabric is about ensuring that the network doesn't become a bottleneck for those software level optimization. So as a conclusion, whether you choose the classic or the rail optimized path, but level.28:16Or 400 gb or 800 gb next to 9000 capabilities to provide a high speed.28:36BGP wrote the fabric. So the goal is simple. Ensure your network architecture is a performant, is as performant as the AI models running on top of it. Okay moving on here.28:43So, we, we have established that an AI network needs to be lostless, needs to be non blocking high speeds and low latency. But as network engineers.29:00The next logical question is, how do we actually build that? So this is like bridge the gap between those high level requirements and the specific networking configuration we implement on our switches. So.29:20On on the left, we have our goals. On the right, we have the how. To achieve a lossless high performance fabric, we relied on a specific set of protocols and techniques. E.g., one to one subscriptions, we start by ensuring our fabric is non blocking, providing a one to one.29:37With you so that every port can operate at full line rate without contention. And the famous rocky version two DMA over conversion internet. This is the backbone of our transport. It allows to perform remote direct memory access over standard internet moving data.29:57Directly between GPU memories with minimal CPU overhead, congestion management that is called the QS site suite. This is where we ensure the network remains lossless because we are using techniques like PFC priority flow control to pause traff.30:18Traffic classes greater than dropping packets, preventing buffer overflow.30:37ECM explicit congestion notification to signal congestion to the end host before it becomes a critical issue. Double red, weighted random early detection, a classical QoS tool that intelligently drop packets to avoid congestion collapse.30:43Ensuring that critical traffic flows remain healthy. DCQC, this is our sophisticated congestion control algorithm, a combination of PFC and ECM that is specifically tuned for rocky version two environments. And finally ADF are.31:03Adapted flow dropping, this acts as our second layer of defense.31:23Dynamically managing large data sets to prevent in congestion and load balances, so we employed advanced load balancing techniques to ensure that traffic is distributed evenly across all available parts in the fabric. So as a conclusion, these aren't just.31:28Industry buzzwords, they are the essential building blocks of a modern AI fabric.31:48By combining these protocols, we transform a standard Ethernet network into a high performance lossless environment capable of supporting the most demanding AIML training worlds.31:54Oh, ok, that, that will be all for me. Good. Yeah, thank you Jorge. So with these upcoming few slides, we are going to recap our session on Cisco AI bots.32:11So, AI pods represent our integrated full stack approach to addressing all the complexities in your AI workloads. So our primary goal with this is to provide visibility, consistency, and control across your entire AI life cycle.32:31So these AI pods are designed to support all pillars of AI, including building your models through training, optimizing models through fine tuning, rag, deploying them through inferencing, so all, all the use cases, then what makes these pods so unique? So it is the integration.32:50With our entire portfolio. So we have our dense GPU UCS servers for compute. We have high performance networking with nexus, Cisco Nexus that Jorge talked about. Then we layer on top of our management tools like inter site and Nexus dashboard.33:10And then we have robust security features like Cisco Hypershield, AI defense, and then Splunk for observability. So basically making sure that you have secure, automated and in scalable environment.33:30Now since our focus is, for that session was on data center compute, so let's discuss the Cisco compute AI portfolio for servers. So Cisco offers two primary deployment to support your AI workloads and all, all these options, both these options are supported with AI.33:46Pods or you could use them without AI pods basically individually as as AI workloads. So the 1st one is UCS GPU server portfolio which includes C 880A885A and 845A.34:06These platform are purpose built for high performance AI and high performance computing workloads. They offer large scale GPU acceleration, high memory bandwidth, and high throughput networking as well. Now 2nd option for Cisco AI compute portfolio is our UCSX platform.34:22So these systems provide modular, scalable, and flexible compute that can support a broad spectrum of AI analytics and of course our traditional data center workloads as well.34:42Now coming to the overview of how you can deploy your AI pods. So deploying AI infrastructure is, is really complex, right? It involves different hardwares, networks, networking, software stacks, and that always don't communicate well with each other. So with AI pods deployed.34:55And with this entire deployment process, this is designed to solve this this challenge by providing a validated blueprint.35:15So we aren't just installing servers, we are building a AI ready factory that ensures your consistency from data center floor to your model inference layer. So we start this by leveraging Cisco inter site to create and deploy your domain profiles.35:24Server profile and chassis profile.35:44Then once the hardware is consistent, we layer on Red Hat openshift. So this provides the entire great Kubernetes foundation needed to manage your containerized AI workloads at scale. We don't just install it, so we can figure it specifically for AI by deploying.35:47And the node feature node feature discovery operator which allows the cluster to intelligently identify and utilize specified hardware like like a GPUs.36:06Then the 3rd one, which is our final step, we integrate the NVIDIA stack, specifically the GPU operator and the NIM operator. So the NIM operator is, is a critical final step, it allows us to perform an automated validation of the entire stack.36:18And by the time we reach this point, we have confirmed that our compute networking and software are perfectly aligned to support high performance model deployments. So by, by following this entire structured and validated approach, we enable our organizations to move quickly.36:35From raw infrastructure to a fully functional AI ready environment that is consistent, that is secure and and ready for production scale and frencing.36:55Okay, so now let's talk about the recap of from our 4th webinar, which was the real life use cases on hybrid cloud and on prem version. So our on prem version is going to be AI pods. So we looked, so we talked about the organizations that are more prepared.37:10But in terms of infrastructure, have cleared a use case defined, understand the workflow, workflow integration, they have better success of deploying AI.37:30Then some of our Cisco AI solutions, Cisco AI pods that I just talked about hyper fabric AI, which is cloud managed, then GPU enabled UCS servers for our core to edge. Now, AI isn't one.37:42The size fits all technology and we have seen a significant success by tailoring your infrastructure to specific vertical needs.38:01So in retails, we are moving beyond simple cameras to predictive drive through optimization and loss prevention in manufacturing we are using digital twins to predict maintenance need before a machine fails, which is kind of a game changer for their uptime. For financial and go.38:10Moment clients the focus is on security and resilience.38:30So we are using AI to detect fraud in real time and to manage complex infrastructure improving both citizen safety and operational efficiency. For health care, we are seeing AI assistant medical image imagining.38:35Helping a radiologist identify anomalies faster and more accurately. But you know how do we make these use cases actually run in real world like high high performance environment? So while looking ahead, the next phase of.38:52API is moving intelligence closer to where the data is generated.39:12So we are seeing a major shift towards Edge AI by running inferencing locally on Cisco AI pods. We kind of solve the critical issues of latency, data sovereignty, that cloud only models don't provide.39:16So and just to mention again that AI pods support various AI workloads including edge inferencing, your model optimization, rack and fine tuning and large scale inferencing as well. Making them kind of versatile for different stages of AI life cycle.39:32And then we are evolving from simple automation to agentic AI. So in this webinar, we looked at two demos. The 1st one was anticipating network failures due to physical interaction. And in this demo I showed you all how you can use vision language model to monitor data center physical.39:50Activity and actually correlate it with network logs in in real time using MCP servers for inter site.40:112nd demo that we looked at was collecting logs from network devices to Splunk and then to webhook application where the webhook application gives you an option to create a ServiceNow ticket with troubleshooting steps using LLM.40:18So, now the next steps, right? The path is very clear. Ensure your infrastructure is AI ready, integrate your workflows and prioritize the use cases that provides highest ROI to you and to your team.40:36And with that, I am gonna pass it to Brian to summarize his session on agentic AI. Thanks Priyanka. So to recap our previous session on agenc AI, we'll just briefly re anchor on why we're moving towards agenic AI. So we're currently faced with a reality where we.40:54Have dynamic environments that have a lot of different tools and our teams are spending a lot of time hunting for context rather than fixing issues. So helps change this by moving us from manual troubleshooting to more of natural language intent and significantly faster mean times to resolution.41:13So we need to distinguish between where we've been and where we're going. So in the past we've relied on automation or scripts for speed. We've had workflows and orchestrations for standard run books, but both of those have limits. Scripts for instance, will break with unknown inputs and then workflows still require constant human intervention.41:36Agents are the evolution, they understand intent, choose their own tools, adapt to the context of the problem, and then verify their own outcomes.41:56How do we actually implement this? So we look at the maturity ladder, we aren't jumping straight to full autonomy. So we start off at L one assisted moved to L two which is supervised execution with approval gates, then onto L three, which is conditional autonomy, and then finally L four which is a closed loop system. The.42:08Key takeaway is this that you want to start at L one or L two for critical high stakes changes and then reserve L three and L four for lower risk domains while you build trust in the system.42:28And then finally, the barrier to entry is lower than you think by using low code frameworks like Cisco workflows or any that we discussed in the previous sessions. We're shifting the focus from complex infrastructure code to pure business logic, so this allows for rapid prototyping, testing workflows.42:44In hours instead of weeks and having our agents up and running in NO time.43:04And with that, let me pass that back to Joe for questions and answers. Excellent. Great. Thank you for all our presenters there providing information to our audience. I realized one thing that I'm gonna send you that that may help. That's a link to.43:11The sessions that we are reviewing today, if you didn't have that already, we've got at least one of the recordings up so far. The others will be posted here once they are ready. Also two for our attendees that are able to use Slido.43:33We do have a Slido poll going. It's open and we like to know where you're joining us from today. So we'll give our presenters a quick break in their voice from talking so long right now. If you can join slido.com and use that join code April eight QA.43:52And the password is the same. If you can just let us know where you're joining from today, that would help our presenters understand where folks are coming from and where we're getting the best outreach from. I see things starting to populate here.44:14Folks are jumping in with some answers there, see Nashville, and I see Raleigh North Carolina, got some folks from California.44:36Let me share these out so we can all look at that together. We got RTP area, you source triangle Park here in Raleigh north Carolina, oaster Mesa.44:51So yeah, you guys feel free to keep putting in where you're joining us from. That only helps us better understand our audience. Of course, we want to make sure that that we do. And so as the slide says, we've got a question and answer session. I know.45:09There's been a lot of activity in chat.45:28If you've got a question that hasn't been answered there or still on the tip of your tongue, feel free to go to slido.com and log in and post that particular question so that one of our panelists can respond back to that. Alternatively, the.45:32Webex that you're using now has a Q and A functionality. You can access by the button in the bottom right, you got the three dots, the ellipse there to go to Q and A and is the last resort if you want to just put it in chat.45:49Maybe even reposted if you still need an answer for that. So moving along, we'll go to some of our Q and A questions that we have sourced here. 1st one looks like it maybe for our team.46:05Covering the AI portfolio overview and customer maturity model, like someone is curious or an answer from that team. We've got a question of what resources.46:21Are available to help me learn more about AI infrastructure. I'll take that. So as we share the the D cloud demo portal is a really good resource to go and.46:36Learn about AI is some of the hardware that's out there. I think we had a question in the chat as well about do we have hardware with GPUs, we certainly do. We'll post that link here back in the chat to the D cloud portal. So some some really good resources there that you can use to do some exploring on AI, the hardware.46:54The software, the observability, it's got a little bit of everything in there, so we'll I'll go ahead and.47:14Post that link back in the chat so it's at the top again. Excellent. Great. Thanks for your answer Mark. We'll make sure that link gets put in the chat for everyone so they can get that additional information there.47:19And let's see, we'll, well mark's posting that link hopefully you can access that. Let us know if you can't. We've got our next questions designed for that same team for the AI portfolio overview.47:37And customer maturity model session, the one that we kicked off with, we've got a question of how are other customers finding success with AI? So if someone from that team can maybe share.47:52Some of the relevant information there and any links that you have, I'm sure our audience attendees would be grateful for them. Hey Joe, so I'll post another link in chat that has a site that has some Cisco customer success stories around AI, so there are a lot of customers from different industry.48:11That are sharing their journeys on AI and how they were able to implement it and some of their success stories. So we'll post that link in chat in there so you can go and and view how how other customers have been using AI and had successful outcomes.48:30Awesome. Thanks again Mark. I'm sure our audience is grateful for that information. We'll keep going on our list here. So we've also got a questions regard to networking for.48:50The networking session that we had earlier, the deep dive. So we're gonna have a question of what's the main difference between distributed training and production inference in terms of network requirements?49:05Again, maybe we got somebody from our deep dive data center networking team that can speak to that. They're gonna take that one Dave. So distribute distribute training requires as I was talking just a moment ago, high note to know bandwidth and.49:22A large network with many GPU CPU hosts.49:42Since the key metric is how fast a model can be trained, in the other way, production inference requires a lower bandwidth but prioritize low latency and high availability. So it's going to need a smaller network to respond to.49:47Real time user requests, so that is the difference between those two.50:07Great. Thank you very much, Jorge. And I'll remind our audience the sessions that we covered today got a recap of and the ones that were in a live Q and A about now. I put a link in chat earlier for the.50:13Page for that if you want to check that out for recordings not there yet, it is gonna be on the way and I'll continue to look at our incoming questions here.50:30So again for our deep dive data center networking team, we've got a question with audience attendee, someone is curious and they are asking what is the back end GPU network need to be lostless?50:48And what two mechanisms work together to achieve this? Very good question. Ok, the the GPU to GPO communication is I think is the most demanding traffic.51:05So we are going to use rocky version two which runs over UDP and has NO bill in retransmissions. So any dropped packet is tolls the entire entire training job. That's, that is the reason because we that the traffic needs to be lostless. So the two mechanisms.51:21To ensure these lossless behavior are ECM that I explained explicit congestion notification, which is signals congestion early so sender is low down and PFC parity flow control which sends.51:41Both frames to stop traffic on a specific priority where buffers are about to overflow. Great answer. Thank you very much Jorge.51:59Now let me switch over and just do another peruse of our lab web live Webex chat at a recent question here. Do customers collapse training networks and inference networks?52:17If you guys covering the network team, if you can take a look at that question and provide an answer either verbally here, let me know or, yeah, you can reply back in chat, but just something so that everyone can, can be exposed to that answer for.52:33So jumping from that, that question in live Webex chat, we'll go back to, question we have on deck now. This is gonna be for our compute team.52:52Specifically the the ones that did a deep dive on data center compute for AI workloads it looks like. And we've got a question here. What is a Cisco AI pod and what are a couple of benefits there? Maybe.53:09Two or three benefits that you can provide for that. I can take that too. So Cisco AI part is a modular reference design that combines high performance AI infrastructure with UCS and Nexus devices, with, with full stack security and observability.53:23And some of the couple of benefits here. So it provides accelerated deployment in days rather than months. This is achieved through pre integrated and full stack solutions and unified management via Cisco Intersight and Nexus dashboard.53:42The next one could be high performance and scalability, so it delivers high AI workload performance using GPUs from NVIDIA combined with high bandwidth and lossless networking. And then of course the enterprise grade security and observability by integrating with.53:59Cisco AI defense, Cisco Hypershield and Splunk.54:19Back to you Joe. Yeah excellent. Thank you very much, Priyanka. I appreciate that answer. Now if you guys have any, you know, follow up questions or additional questions.54:24To what you're hearing us talk through now, feel free to throw that in Slido as well or or Webex chat, whichever you have at the easiest access to, we do want to make sure we cover those questions, even the follow up questions.54:39Now sticking on the data center compute train of thought here. We've got another one for our team from deep dive data center compute. We've got a question regarding the C eight 80 and C 08:40 fives.54:58And I'll read this out, what is the difference between the C 880AMA and the C 845AM8 both M eights in terms of their design and their intended use cases?55:14It's a very thorough question there for our deep dive data center compute team. Joe I can take that as well. So C 880A is built on Nvidia HDX platform with a fixed B 300 GPUs. So it's it is optimized. A use case will be.55:32More dense GPUs so LLM inferencing and an advanced training. And C 405A uses modular approach within video MGX design and it supports two to eight GPUs so making it more flexible for diverse workload and custom configurations.55:53Back to you Joe. Great. Thanks again for that answer, Pianka. Scanning the Webex chat here, looking again at our list of questions that are on deck.56:12We're gonna move on to one about hybrid cloud here. And the question is, why is a hybrid cloud model recommended for AI workloads that involve both training and inferencing? I'm assuming this, this question's intended for our real life use cases.56:29On Cisco infrastructure team. Let me just follow up, repeat that question. Why is a hybrid cloud model recommended for AI workloads that involve both training and inferencing?56:48Sorry Joe, could you please repeat that question? Yeah, let me let me rephrase and repeat that question there. It's regarding hybrid cloud models and it says, why is a hybrid cloud model recommended?57:18For your AI workloads that involve both training and inversing. I'm assuming this is, this is gonna clarify it for somebody there. Yep, thank you Joe. So a hybrid approach basically balances the strength of cloud and on prem environments so large scale training and orchestration are often handled in the.57:32Cloud, so due to their massive compute resources requirement and on prem offers ultra low latency, data sovereignty and in bandwidth optimization by by keeping your sensitive data and real time decision making close to your data.57:52So with this hybrid basically will get you best of both the worlds. Back to you Joe. Ah, it makes sense. Good answer, thanks. Unlike that localized inferencing.58:09The model there. That's gonna be big for some folks, some of our attendees, and now we'll take another question here. It's gonna be around the UCS portfolio line.58:24And could be again probably for our use case team for the real life AI use cases that we covered. I think this was one of the core concepts there, but we do have a question that addresses that. Someone is asking how does.58:41Is the Cisco UCS profile or port portfolio excuse me, how does the UCS.59:01Portfolio differentiate between any inter level AI and any high performance training or high performance compute workloads HPC workloads. Okay, thank you Joe. So I can take that as well. So Cisco kind of categorizes it.59:08UCS GPU options based on specific AI stages. So for entry level we offer workloads like UCS 02:40 UCSX series with L 40 GPUs.59:28And for demanding AI training, high performance computing and large scale data analytics, the portfolio scales to C 885AC845A, which supports high density GPU configurations like Nvidia H100, Nvidia H 200 pro.59:41Providing the necessary memory bandwidth and GPU density.01:00:01Back to you Joe. Great, thanks for those highly specific details, Priyanka. I know that filled a need that that one of more of our attendees had today regarding that Cisco UCS compute.01:00:06For that. Look back at our live Q and A and Webex chat sessions there, looks like we just had a question answered by Brian. I appreciate that Brian.01:00:24No, we'll go back to our stated questions here that are on deck. We'll take one here about the differences between automation and agents, and there's like somebody's got a question about.01:00:42Let me just read the question here. It's for our agentic AI team and their question is, what is the fundamental difference between traditional automation and an AI agent? I can grab that Joe. So traditional automation is deterministic. It's gonna go ahead and.01:00:59Follow a fixed script and it'll break if anything within the environment deviates from the expected inputs, whereas an AI agent is more adaptive, it understands the intent, it can choose what tools it wants to use based on the context, and then it also incorporates a reasoning loop to help verify the outcomes.01:01:18Ah, I got to deterministic versus adaptive. Great. You guys will have the recording of all of these Q and A's, I'm sure it's important information for some of you folks are gonna wanna reference it later, but we do plan to have the recording of the session posted later. You'll be able to.01:01:39Hear Brian's answers among the others.01:01:59So for a follow up question or maybe not, maybe just a standalone question. We're gonna have a question here that's gonna be around some risks there. One of our attendees wants to know what's the biggest risk.01:02:09When moving from L one or layer one, that's assisted over to L four, which would be a closed looped autonomy or autonomous environment.01:02:26I can grab that one Joe. So the primary risk is just loss of visibility and control. So as you move up the maturity ladder, the system's gonna go ahead and make more decisions independently. So this is why we recommend starting at like layer one or layer two or L one or L two for critical changes.01:02:41That require approval gates and then reserving L four for lower risk domains where the cycle has already been well tested.01:03:01Excellent. Thanks Brian. So we'll look back. Folks are still able to vote where you're joining us from. In a few minutes I'm gonna turn on a participant survey. You'll see that you have access to.01:03:11Now these votes won't be shared, but they'll provide us with.01:03:31Some more specific feedback that we'll use to make our future sessions even better. Now with that being said, take a look at our live Webex chat Q and A one answered recently. There's still a question.01:03:35Out there about.01:03:55Whether our customers collapse training networks and inference networks. Now we may not be able to provide specific, you know, information around our customers, obviously, and specific environments, but there maybe some reference documentation that that we can provide around that scenario to help folks make a informed decision there.01:03:57With that we'll go back to fielding more questions. We've got a question. Our next question, where can I find resources for designating or designing, sorry, an AI infrastructure? Now it looks like.01:04:20Designing the AI infrastructure that might be handled by our our 1st presentation Digging back to the start of this session, our AI portfolio team might have some input and let me restate the question. Where can I find resources for designing.01:04:36An AI infrastructure. Joe, I shared a link in the chat on our our Cisco validated designs page, so there are a number of design documents that reference building AI either for for medical or maybe certain use cases for like a rag pipe.01:04:54Line or building, lossless networks. So another really good resource to go out and, and explore what are the different ways that you can design your AI networks. And these design guides have the hardware, the configurations, the examples, and of course all of the different.01:05:14Configurations you would need to build your AI network, so that's a really good bookmark to go out and for reference. Thanks Mark, and looking at that now.01:05:34I encourage all our intendees to find that link that was just put in our chat. Click on that and read through it. Design guides for AI ready infrastructure that that nails the answer to the question that was posed there and it's gonna turn on a lot of lights for some of you folks.01:05:50You'll be able to explore that website, get some answers, maybe drill down, provide the scenario that you're particularly facing and get some information there. Thank you Mark.01:06:08So we'll go back to some on deck questions here, like we might have one for our networking folks or team that handled our deep dive on networking and we're gonna.01:06:26Field a question here is gonna say what's the rail optimized design rail optimized design and why that's preferred for your GPU clusters. So are you gonna take that one Joe?01:06:42So we reviewed that in during our presentation, so in rail optimized designs GPUs with the same IT across different notes are connected to the same leaf switch that is called rail. So this minimized latency for.01:06:58The same GPU communication, which is the most common traffic pattern during the typical operation that is called all reduce and all to all.01:07:18So these rail optimized designs improves overall training performance compared to a classical design. Remember that I showed a classical design in the slide in a classical design, the warehouse are simply mapped for rack, so that is that is what is rail optimized design.01:07:29Awesome. Thank you very much for that precise answer. I was reading through that document that Mark posted earlier, that is very thorough. There's a lot of links to a lot of different areas.01:07:52Around the AI infrastructure design that you'll be able to take advantage of. Thank you Jorge, a question about the rail optimized design that's gonna help a lot of folks there that had that same particular question. We've got a few minutes left folks, if you still have, have questions.01:08:08That you've been hesitant to answer or on the tip of your tongue, you just hadn't typed them in yet. Feel free to put those in our slido.com Q and A section or our Cisco Webex Q and A section in the bottom right of your screen.01:08:28Or even the chat if you need to. You'll find a lot of links in that chat as well. So move back to our list of questions and looking, looking.01:08:44Let's see, we might have another one regarding agentic AI I know there's a a hot topic right now that's in a lot of conversations. Agent to AI. We covered that in the session that that walks you through moving from.01:09:05Automation, a lot of folks maybe in that step and moving to autonomous networks there we covered that in an earlier session. And we've got someone with a question around that. Looks like they've got a question.01:09:20Around the term identify and looks like they're asking the question how should we prioritize which network task to identify 1st? Grab that one. So.01:09:37You can use like a simple matrix. You can map your use cases by business impact versus technical complexity, and you'll start with like the high impact low compell low complexity tasks that'll prove value, and then you can start working on the.01:09:56Lower impact tasks until you've completed all of those within your matrix. Thank you Brian. Again, if that.01:10:13Information Brian provided right there sparks another question or you have a follow up question, please engage us with that. We do have a few more minutes left in our session today. You don't see anything popping up.01:10:28Regarding that, so we'll try to keep the ball rolling here and move on to another question. It's gonna be our about our AI maturity model. Someone asked a question around the AI maturity model. And again, we covered that.01:10:46In a session provided by our AI maturity team. We covered the AI portfolio and overview earlier. We covered that earlier in this session. That was the 1st one. So the question at hand is.01:11:04What are the the stages of the AI maturity model and what characterizes the infrastructure at each stage? Oh, they've got, they've got five listed here. So what are the five maturity stages?01:11:20Of the AI maturity model and what characterizes the infrastructure at each stage that might be a rather involving answer. But I know we've got time for it. Joe I can quickly go over.01:11:37The different stages and as far as of the hardware being used. So this stage one is what we call it awareness, so this was just your learning about AI.01:11:56The infrastructure you currently have runs as it is. So you're just thinking about how can you use AI, you know, within your business, maybe there's certain apps you want to include some type of AI. Stage two is where you start to take those ideas and then.01:12:07You make use of maybe existing hardware you had, you add a couple of GPUs, you add additional storage faster that you can use to take the ideas you had, and you start staging them into like a development or a test lab, and you start.01:12:24To take that idea and then think of ways how can we take this into production? So once you've learned what's needed from stage two, when you go into stage three, this is what we call the the operational. So now you know what type of hardware you need. So you make additional purchases for.01:12:41For your GPUs or maybe some additional open shift licenses, all the software that you're also going to need.01:13:01To make that AI application into a production application. And then in stage four, we call that systemics, so now you know how AI works from a hardware perspective, you understand the software that's needed for it, you unders.01:13:08Understand how to wrap security around it. And then at this point, this takes you into the the stage five, which we called transformational. So at this point, new apps that come out, AI is already built into it.01:13:28It's already developed, you know how to to scale the infrastructure. So at stage five, you've seen how to deploy AI. Now you understand how to scale it by adding additional infrastructure, more building blocks for GPUs, storage compute. So you have a really solid understanding and then you're.01:13:43Now at the phase where using AI is just automatically a 1st priority in, you know, developing your apps and you know how to scale it from there and secure it.01:14:03Great. Thanks for that. Thanks for that answer mark. That was a lot of information again this will be recorded for attendees. You'll be able to refer back to this later we will take another.01:14:18Quick look here at our audience Q and A in Slido, Nothing outstanding. There's, there's been a couple that have been answered there, take a look at our Webex chat and Q and A.01:14:34I don't have anything pending there another audience questioned that we can touch on is gonna be around MCP.01:14:51Again, that's one of those hot topic phrases that's in the news, that's in probably most of the articles that you're reading nowadays MCP, and we've got a question probably for for our agent to AI team to be able to to answer that. I know that was covered in the.01:15:07F during this session and touched on a little bit today, but we've got a question about MCP and the question is gonna be, what role does the model context protocol for MCP? What role does that MCP play in a.01:15:28A Gentic architecture. I can grab that one Joe. So MCA MCP acts as a universal gateway between a large language model and tools. So instead of needing to build custom integrations for every combination of LM and tool, MCP creates.01:15:46It's that standard interface allowing agents to connect to any tool provider that supports the protocol.01:16:06Awesome. Thanks Mark. And, and if you can stay there, we've got another related question there around agents there and someone is asking how does the agent handle any unknown scenarios that you know might typically you see it break your script.01:16:14Somebody's asking about that. Yeah, so unlike a script an agent's gonna use a reasoning loop to analyze the observation of the output, so like if it encounters an unknown input, it can reflect on the results, it can adjust its plan and even choose like a different tool to gather the information that it needs before.01:16:34For attempting the action.01:16:54All right, I hope that that satisfied our person or persons with the, with the question around that do another quick check where everyone can ask a question at. Do you think pending outside of our, our list here?01:17:01Let's see, if you got, if you got time Brian, someone for, for your team, there's another question around AI and it's probably a common question, probably a lot of folks have this question. I know I've looked into it before and and figured out what I can, so something around AI or Gentic AI.01:17:23That we've got, we are being faced with the question about llmiss around that and they are asking if it's necessary to use the most powerful LLM for each and every agentic task.01:17:42Yeah, for the agentic tasks so you can match a model to the job so you consider the task type, any kind of requirements on like latency context size and cost. And in some cases or most cases the similar fine tuned model may work better and maybe cheaper for that particular agent than a massive.01:18:01Purpose model. Great. Good answering. I know I certainly had that question at some point. Like I said I researched it myself. And we've got another few minutes, few.01:18:21Moments folks, if you've got questions from in Slido or our Webex there, let me start our attendee poll. If you can take a look at that poll that's gonna be available to you, it'll give you the opportunity to provide.01:18:40Get some feedback and you'll be able to put in any specifics or any general questions that you might have. That'll be the place to do it. You should see that available to you now. I've got some additional information here. We don't seem to have any more pressing questions at the moment.01:19:00But we've got a few minutes left. Some additional information for our attendees. Here's a list of some upcoming expert insight series questions or not questions but sessions that we're gonna have. We've got dates nailed down that we, now we don't have these.01:19:19Advertise on a link that we can send you right now, feel free to take a screenshot of this or otherwise jot this down, copy this. It'll be, it'll be posted in the recording.01:19:38Once this session's finalized. There's a link that's on screen at the bottom there for our video portal. If you haven't, haven't looked at that yet, I encourage you to, I was in there earlier today looking around.01:19:59Hopefully there's a topic there that's of interest to you. I know there's there's hundreds of videos. There's a search box in the top left you want to make sure you pay attention to when you go on that video portal and you can type in a topic. I know I think I did inter site earlier just to see curious how many would come back and it was over.01:20:13For a hundred, hundred and 20.01:20:33Results returned just for the the topic of intersight. There's a lot of information available to you there at Will for you to be able to tap into. Now if you're through getting a screenshot here.01:20:36You can plan for those dates coming up. For the networking folks, we didn't forget about you. We do have yet another series of dates coming up and let me get this shared out to you.01:20:54I'm gonna share my screen and same thing let you take a screenshot of that here and this is for our networking folks.01:21:13Hopefully you can see my screen there. We've got for 2026 this year. We've got our upcoming expert insights series and this series is gonna be on the topic of software strategy and mastery, ACI and Nexus dashboard. Now we've got six.01:21:33Dates specified in the series so far, covering different topics. Hopefully some of those pertinent to you or important to you. I'll take a second and let you get a screenshot.01:21:53And again, we've got the same link, at the bottom of the screen as we had on the other to the Cisco video portal, Encourage you to make sure you get that in your screenshot. I put a link to it in our Webex chat, make sure that you click on it there and go to it. Use that.01:22:08The search bar to search for different topics, networking has something that's generic as that.01:22:28Or something that's more relevant to you, but that should be available to you stop sharing that. That will be covered in the recording of the session later on.01:22:34With that, I don't have any more pressing questions. If anybody on our team wants to make any closure remarks, feel free to to jump in. But otherwise here with a few minutes left, we're out of questions. Hopefully we answered yours and gave you an opportunity to present those. So again, we thank you.01:22:54Thank you for joining us today.01:23:14This was the recap and live Q and A for our series earlier this year about AI ready data center infrastructure and the use cases for that. Thank you very much.01:23:18