FlashStack with Red Hat OpenShift Containerization and Virtualization

 

Published: October 2025

In partnership with:

About the Cisco Validated Design Program

The Cisco Validated Design (CVD) program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information, go to: http://www.cisco.com/go/designzone.

Executive Summary

FlashStack is a validated, converged infrastructure solution developed jointly by Cisco and Pure Storage. The solution offers a predesigned data center architecture that incorporates compute, storage, and network to reduce IT risk by validating the architecture and helping ensure compatibility among the components. The FlashStack solution is successful because of its ability to evolve and incorporate both technology and product innovations in the areas of management, compute, storage, and networking. This document covers the design and deployment details of Red Hat OpenShift Container Platform (OCP) as well as Red Hat OpenShift Virtualization on FlashStack Bare Metal infrastructure. This solution allows customers to run and manage virtual machine workloads alongside with containerized workloads.

Some of the key advantages of FlashStack solution with Red Hat OpenShift Container Platform and Red Hat OpenShift Virtualization are:

●     Simplify IT operations with a unified Red Hat OpenShift platform: Red Hat OpenShift, a leading enterprise Kubernetes platform, offers a robust solution for managing both containers and virtual machines (VMs) through its integrated feature, Red Hat OpenShift Virtualization. Customers can run and manage both containers and virtual machines side-by-side within a single Red Hat OpenShift cluster avoiding operational complexity and challenges of maintaining separate platforms for running these workloads.

●     Cisco UCS AMD M8 Series servers: These servers, powered by AMD EPYC processors designed to deliver high core density, large memory capacity, and energy efficient for hosting modern enterprise workloads. Cisco UCS integrates compute, networking, and storage into a unified architecture with Cisco Intersight for cloud-based management, making them ideal for virtualization, AI/ML, big data, and cloud-native applications.

●     Consistent infrastructure configuration: Cisco Intersight and UCS help bring up the entire server farm with standardized methods and consistent configuration tools that helps to improve the compute availability, avoid human configuration errors and achieve higher Return on Investments (ROI). The Intersight Integration with OpenShift Assisted Installation method enhances deployment experience without hopping on to multiple management points.

●     Isovalent Networking for Kubernetes: Cisco’s cloud-native networking and security solution for Kubernetes built on Cilium and eBPF, designed to provide advanced observability, zero-trust security, and high-performance networking for Kubernetes and multi-cloud environments. It extends beyond traditional CNI (Container Network Interface) by offering deep visibility into workloads, identity-aware networking, and policy enforcement at the kernel level without relying on sidecars or iptables. Through this, Cisco positions Isovalent as a foundation for modern application connectivity and security, helping enterprises securely scale containerized and microservices-based architectures.

●     Portworx Enterprise integration with Pure Storage FlashArray: This integration offers a unique platform that combines Portworx’s Kubernetes-native storage and data management with the enterprise-grade performance and reliability of FlashArray.  With this integration, Portworx abstracts FlashArray’s block storage into a container-ready, software-defined layer, enabling dynamic provisioning, snapshots, backup, and disaster recovery for stateful Kubernetes workloads. FlashArray provides consistent low-latency storage, while Portworx adds capabilities like multi-cloud portability, encryption, and application-level policies—together ensuring scalable, highly available, and production-ready data services for cloud-native applications.

●     Splunk Observability Cloud is a SaaS-based, full-stack observability platform that unifies metrics, logs, traces, real user monitoring, and synthetic testing into a single solution for modern, distributed applications. Built on OpenTelemetry and designed for real-time, full-fidelity data ingestion, it helps DevOps, SRE, and IT teams quickly detect, investigate, and resolve issues across infrastructure, applications, and end-user experiences. Its key benefits include faster root cause analysis with AI-driven insights, reduced downtime through proactive alerting, seamless integration with Splunk logs, and improved visibility across hybrid and multi-cloud environments—ultimately driving better performance, reliability, and customer experience.

In addition to the compute-specific hardware and software innovations, integration of the Cisco Intersight cloud platform with Pure Storage FlashArray and Cisco Nexus delivers monitoring, orchestration, and workload optimization capabilities for different layers of the FlashStack solution.

If you are interested in understanding the FlashStack design and deployment details, including configuration of various elements of design and associated best practices, refer to the Cisco Validated Designs for FlashStack here: https://www.cisco.com/c/en/us/solutions/design-zone/data-center-design-guides/data-center-design-guides-all.html#FlashStack

Solution Overview

This chapter contains the following:

●     Introduction

●     Audience

●     Purpose of this document

●     Highlights of this Solution

Introduction

The FlashStack solution with Red Hat OpenShift on a Cisco UCS Bare Metal configuration represents a cohesive and flexible infrastructure solution that combines computing hardware, networking, and storage resources into a single, integrated architecture. Designed as a collaborative effort between Cisco and Pure Storage, this converged infrastructure platform is engineered to deliver high levels of efficiency, scalability, and performance, suitable for a multitude of data center workloads. By standardizing on a validated design, organizations can accelerate deployment, reduce operational complexities, and confidently scale their IT operations to meet evolving business demands. The FlashStack architecture leverages Cisco's Unified Computing System (Cisco UCS) servers, Cisco Nexus networking, Pure’s innovative storage systems, and Isovalent Enterprise Platform, providing a robust foundation for containerized, virtualized, and non-virtualized environments.

Audience

The intended audience for this document includes, but is not limited to IT architects, sales engineers, field consultants, professional services, NetOps teams, K8s Platform teams, Cloud Native teams, IT managers, IT engineers, partners, and customers who are interested to take the advantage of an infrastructure built to deliver IT efficiency and enable IT innovation.

Purpose of this document

This document provides deployment guidance for bringing up the FlashStack solution with Red Hat OpenShift container and virtualization platforms on Bare Metal Infrastructure. This document introduces various design elements and explains various considerations and best practices for a successful Red Hat OpenShift deployment.

Highlights of this Solution

The highlights of this solution are:

●     Red Hat OpenShift Bare Metal deployment on FlashStack solution enabling customers to run both containerized and virtualized workloads running alongside each other within a cluster.

●     Cisco UCS AMD M8 series servers, powered by the latest AMD EPYC processors, offers higher compute, memory densities for hosting modern enterprise workloads and AI-Ready design to support required GPUs for running AI/ML based workloads.

●     This OpenShift solution utilizes Isovalent Enterprise as Container Networking Interface (CNI) framework offering high performing networking, security, policy enforcement, and observability by leveraging Extended Berkeley Packet Filter (eBPF) technology.

●     With the latest Portworx Enterprise brings in support for ReadWriteMany (RWX) raw block volumes for KubeVirt virtual machines (VMs), enabling high-performance, shared storage configurations that support live migration of VMs in OpenShift environments. PX-Backup offers a data protection solution that provides application-consistent backup, restore, and disaster recovery for containerized and virtualized workloads across on-premises and cloud environments. The solution uses Pure Storage FlashArray as backend storage for Portworx which is a truly unified block, file, and object storage that continues to get faster, more reliable, more secure, smarter, and easier to manage over time. Upgrades are always with data-in-place and completely non-disruptively—without caveats or compromise.

●     Splunk Observability Cloud is a SaaS-based, full-stack observability platform designed for modern, distributed environments. It brings together the three pillars of observability—metrics, logs, and traces—into a unified interface, enabling teams to quickly detect, troubleshoot, and resolve issues across applications, infrastructure, and digital experiences.

Technology Overview

This chapter contains the following:

●     FlashStack Components

●     Red Hat OpenShift

●     Cisco UCS AMD M8 Series Servers and Cisco Intersight

●     Isovalent Networking for Kubernetes

●     Portworx Enterprise with Red Hat OpenShift Virtualization

●     Portworx Enterprise with Pure Storage FlashArray

●     Data Protection with PX-Backup and Pure Storage FlashBlade

●     Infrastructure and Application Monitoring with Splunk Observability

FlashStack Components

The FlashStack architecture was jointly developed by Cisco and Pure Storage. All FlashStack components are integrated, allowing you to deploy the solution quickly and economically while eliminating many of the risks associated with researching, designing, building, and deploying similar solutions from the foundation. One of the main benefits of FlashStack is its ability to maintain consistency at scale. Figure 1 illustrates the series of hardware components used for building the FlashStack architectures. Each of the component families (Cisco UCS, Cisco Nexus, Cisco MDS, Portworx by Pure Storage and Pure Storage FlashArray systems) offers platform and resource options to scale-up or scale-out the infrastructure while supporting the same features and functions.

Go to the Appendix for more information about the components used in this solution.

Red Hat OpenShift

Red Hat OpenShift is a family of containerization software products developed by Red Hat. Red Hat OpenShift Container Platform (OCP) is its flagship product that is Kubernetes-based enterprise container platform helps organizations build, deploy, and manage applications consistently across hybrid and multi-cloud environments. It is built on open-source Kubernetes but adds enterprise features like enhanced security, developer productivity tools, built-in CI/CD pipelines, and centralized management.

Red Hat OpenShift Virtualization is an add-on feature of OpenShift that lets us run and manage virtual machines (VMs) side by side with containers on the same OpenShift cluster. It uses the open-source KubeVirt project to extend Kubernetes APIs for VM lifecycle management so that you can create, start, stop, migrate, scale VMs through the OpenShift console or oc CLI. It leverages KVM, QEMU and few other libraries like libvirt, virt_launcher, Multus CNI, OVN-K etc. to provide a reliable type-1 hypervisor for virtualization tasks. It leverages CNI for VM networking and CSI for the persistent storage. It uses OpenShift security features such as SCCs, RBAC, and SELinux for VM isolation. When Red Hat OpenShift Virtualization is added to a Red Hat OpenShift environment, both containers and VMs can now be run side-by-side on the same infrastructure as shown in Figure 2.

Cisco UCS AMD M8 Series Servers and Cisco Intersight

Cisco UCS AMD M8 Series Servers use 4th-and 5th-gen AMD EPYC CPUs which offer large numbers of cores per socket. This gives high aggregate compute power and larger memory capacity offering high performance for multi-threaded workloads, virtualization, big data, analytics. Cisco UCS X215c M8 and C245 M8 offer dual socket configurations with DDR5 memory DIMMS (operating at high speeds 4800 MTs to 6000 MTs) for higher compute, better memory capacity, and bandwidth. Cisco UCS C225 M8 offers single socket configuration simplifying the configurations (no NUMA complications), reduce the costs but achieving high core counts and I/O. These servers also support multiple GPU options offering dense GPU-compute per rack useful for scale out workloads like analytics and AI/ML workloads. With Cisco UCS 5th generation VIC cards, these servers offer unified fabric for consolidating various traffics, high speed networking, agile infrastructure with virtualization feature, and simplified Management.

Cisco Intersight

Cisco Intersight is a lifecycle management platform for your infrastructure, regardless of where it resides. In your enterprise data center, at the edge, in remote and branch offices, at retail and industrial sites—all these locations present unique management challenges and have typically required separate tools. Cisco Intersight Software as a Service (SaaS) unifies and simplifies your experience of the Cisco Unified Computing System (Cisco UCS). In addition to the Cisco UCS management, Cisco switches, and Pure Storage FlashArray are also integrated into the Cisco Intersight. With this integration, FlashArray storage controllers can be managed and perform all day one management tasks, and these tasks can be incorporated in to the Intersight workflows.

Isovalent Networking for Kubernetes

Isovalent Networking for Kubernetes is available and supported across several Kubernetes platforms and offerings. One of the most popular is that of Red Hat OpenShift. Red Hat OpenShift provides a default networking model leveraging Open Virtual Network (OVN) which internally uses Open Virtual Switch (OVS). When business-critical applications are migrated to OpenShift, there is an increased need for a cloud native networking approach. Identity- and application-aware policy enforcement become standard requirements. Isovalent Networking for Kubernetes leverages eBPF (extended Berkeley Packet Filter) to address these needs, providing high-performance networking, robust security, and deep observability directly within the Kubernetes environment.

Isovalent's platform, built on the open-source Cilium project, utilizes eBPF technology to embed networking and security logic directly into the Linux kernel, offering an innovative approach to managing containerized environments. This enables capabilities such as L3/L4 and L7-aware network policy enforcement, identity-based security, zero-trust networking, and transparent encryption. Furthermore, it provides extensive observability through tools like Hubble, offering cluster-wide flow visibility and identity-aware network metrics. Designed for Kubernetes environments, including being certified for Red Hat OpenShift, Isovalent Enterprise for Cilium ensures optimal scale, performance, and compliance across cloud and on-prem infrastructure.

By using an eBPF based implementation, the default, iptables-based OpenShift provided CNI can be replaced with a dataplane that executes directly within the Linux kernel. The above enhanced capabilities are provided as part of the wider cloud native platform offering, in this case, Red Hat OpenShift, whilst avoiding the resource overheads associated with other typical cloud native networking implementation models. As a certified Operator, the solution integrates with OpenShift's Cluster Network Operator (CNO) for deployment and lifecycle management from the OpenShift console.

The eBPF data plane’s kernel-level operation provides performance advantages over iptables. By using eBPF maps for lookups instead of traversing linear iptables rule chains, it processes network traffic with lower CPU overhead and reduced latency. eBPF programs handle packet forwarding, policy enforcement, and telemetry collection within the kernel, which eliminates context switches between user and kernel modes.

For FlashStack deployments with Red Hat OpenShift, Isovalent Networking for Kubernetes provides a highly flexible overlay networking model using VXLAN encapsulation. This mode, which is the default, simplifies deployment by creating a virtual network that spans all cluster nodes, requiring minimal configuration on the underlying physical network fabric. The container network is decoupled from the physical infrastructure, allowing OpenShift nodes to span multiple L2 or L3 domains without complex fabric configurations. While the VXLAN overlay offers maximum flexibility, a native routing underlay model using Border Gateway Protocol (BGP) is also supported for use cases requiring direct fabric integration and the elimination of encapsulation overhead. 

General Architectural Benefits of Isovalent Networking for Kubernetes

●     EBPF-based Data Plane Performance: The eBPF data plane provides performance improvements over iptables-based CNIs by processing packets in the kernel. This reduces CPU overhead and network latency by using efficient map-based lookups instead of linear rule chain processing and by eliminating kernel-to-userspace context switches for packet forwarding and policy enforcement.

●     Identity-Based Security Model: The security model is based on workload identity derived from Kubernetes labels, rather than ephemeral IP addresses. Policies are enforced by eBPF in the kernel for L3/L4 traffic.

●     Integrated Network Observability: The architecture includes Hubble or Hubble Timescape (for future releases of Isovalent), an observability platform that uses the same eBPF data source to provide visibility into network flows, service dependencies, and policy enforcement without requiring separate agents.

●     Flexible Networking Models: The CNI supports multiple networking modes, including an overlay model using VXLAN for deployment flexibility and an underlay model using BGP for native routing performance.

 

Benefits of FlashStack with OpenShift Environments

●     Optimized Resource Utilization on UCS Compute: The CPU cycle reduction from the eBPF data plane frees compute resources on Cisco UCS X-Series nodes, allowing more capacity to be allocated to application workloads. This efficiency applies to complex server configurations, including those with multiple vNICs, without the need to manage iptables rules for each interface.

●     Flexible Overlay Networking with VXLAN: Cilium’s default VXLAN overlay mode provides significant operational simplicity for FlashStack deployments. It establishes a virtual network fabric on top of the existing Cisco Nexus infrastructure, requiring only standard IP connectivity between nodes. This decouples the lifecycle and management of the container network from the physical network, allowing for greater agility. Pod-to-pod traffic is encapsulated, meaning the physical network does not need to be aware of pod IP addresses. The VXLAN header can also carry metadata, which Cilium uses to transmit security identity information, optimizing policy enforcement between nodes.

●     XDP Acceleration and Hardware Offload Readiness: The solution supports eXpress Data Path (XDP) to accelerate services like NodePort and LoadBalancer by processing packets at the network driver layer. The eBPF architecture is also suited for future offloading to Data Processing Units (DPUs) and SmartNICs, a capability that aligns with the evolution of Cisco's hardware portfolio (upcoming capabilities).

●     Resilient Fabric Integration: The BGP implementation supports the high-availability design of the FlashStack fabric, which includes redundant Cisco UCS Fabric Interconnects and Nexus switches. It enables routing topologies that can reconverge in the event of a link or switch failure, maintaining network connectivity for applications by rerouting traffic through available paths.

Isovalent Networking for Kubernetes Deployment in OpenShift

Isovalent Networking for OpenShift can be deployed in the following methods:

●     Base Installation: In this method, the Isovalent is deployed when OpenShift cluster is being provisioned (commonly referred to as “bootstrap”). The installation of Isovalent Networking for Kubernetes varies depending on how the OpenShift cluster is being provisioned. Isovalent can be installed with OpenShift Installer binary (Agent-based Installer, Installer-Provisioned Infrastructure, User-Provisioned Infrastructure) and with  Red Hat Advanced Cluster Management (RHACM) and with Red Hat OpenShift Container Platform with hosted control planes (HCP).

●     Migrating to Isovalent Networking for Kubernetes: Isovalent can also be installed after the OpenShift cluster is being provisioned by migrating the default OVN-K CNI plugin to Isovalent.

In this FlashStack validation, Isovalent is used as CNI plugin for providing OpenShift networking. This is achieved by migrating the default OpenShift OVN CNI plugin to Isovalent Networking for Kubernetes.

The following sections provide the steps for migrating to Isovalent Networking for Kubernetes in an existing OpenShift cluster.

Portworx Enterprise with Red Hat OpenShift Virtualization

Portworx delivers integrated, enterprise-grade storage for VMs and containers, simplifying operations through Kubernetes while providing high availability, cross-site resiliency, advanced (sync/async) disaster recovery, automated scaling, and strong data security. With STORK, you gain VM migration between clusters plus policy-driven backup and restore to meet SLA commitments. This integration supports a unified infrastructure where traditional and cloud-native workloads coexist, offering flexible deployment across diverse environments. For more information, go to the Appendix.

Portworx by Pure Storage unlocks the value of Kubernetes data at enterprise scale. It’s a fully integrated container data platform that Automates, Protects, and Unifies modern applications across hybrid and multi-cloud, works with any underlying storage (on-prem or cloud) and any Kubernetes distribution, and simplifies developer actions and platform data management.

●     Automate: Portworx automates Kubernetes data management end-to-end, boosting efficiency and time-to-market across DevOps/MLOps. It abstracts heterogeneous on-prem/cloud storage and delivers a cloud operating model with self-service storage/database services.

●     Protect: Architect app-aware resiliency from Day 1 with synchronous DR (zero-data-loss targets) and automated Day-2 operations. Encrypt at cluster or storage-class scope, enforce RBAC, and use policy-driven backups with immutability/portability to counter ransomware.

●     Unify: Unify Kubernetes storage by removing per-array CSI dependencies so platforms stay fully declarative across hybrid/multi-cloud. Manage data for containers and VMs in one solution—reducing VM licensing overhead and preserving future flexibility.

Portworx with Red Hat OpenShift Virtualization and KubeVirt enhances data management for virtual machines and containers by offering integrated, enterprise-grade storage. Includes simplified storage operations through Kubernetes, high availability and resiliency across environments, advanced disaster recovery options, and automated scaling capabilities. This integration supports a unified infrastructure where traditional and modern workloads coexist, providing flexibility in deployment across diverse infrastructures and ensuring robust data security. Portworx and Stork offers capabilities like VM migration between clusters, Synchronous Disaster Recovery, Ability to backup and restore VMs running on Red Hat OpenShift to comply with the service level agreements.

Figure 6 shows the suite of products that Portworx offers as of today. For more information on these products, go to the Appendix.

This solution is validated with Portworx version 3.4, which offers following additional benefits alongside its established enterprise-grade storage features, including replication, snapshots, clones, compression, and encryption.

●     Shared ReadWriteMany (RWX) Block volumes for KubeVirt VMs: Portworx supports RWX raw block volumes for KubeVirt virtual machines (VMs) enabling high-performance, shared storage configurations that is required to support live migration of VMs in the OpenShift environments.

●     DR support for KubeVirt VMs with Portworx RWX Block Volumes: Portworx supports synchronous and asynchronous disaster recovery on KubeVirt virtual machines (VMs) with ReadWriteMany (RWX) raw block volumes running on OpenShift Virtualization version 4.18.5 or later and OpenShift Container Platform version 4.18.x. For more information  Synchronous Disaster Recovery or Asynchronous Disaster Recovery.

●     Application I/O Control: Portworx supports Application I/O Control on hosts that use cgroup v2, in addition to cgroup v1. Portworx automatically detects the available cgroup version and applies I/O throttling accordingly to ensure seamless operation across supported Linux distributions. This feature becomes handy to prevent a "noisy neighbor" problem by setting IOPS (input/output operations per second) and bandwidth limits for individual persistent volumes in a shared storage pool.

●     FlashArray Direct Access (FADA) Shared Block Devices: Portworx enables the seamless integration of KubeVirt virtual machines (VMs) within Kubernetes clusters, leveraging the high performance of ReadWriteMany (RWX) volumes backed by FlashArray Direct Access (FADA) shared raw block RWX volumes. This approach supports raw block devices, which provide direct block storage access instead of a mounted filesystem. This is particularly beneficial for applications that demand low-latency and high-performance storage.

Portworx Enterprise with Pure Storage FlashArray

Portworx on FlashArray offers flexible storage deployment options for Kubernetes. Using FlashArray as cloud drives enables Autopilot-driven automatic volume provisioning, and seamless cluster expansion. Direct Access volumes allow for efficient on-premises storage management, offering file system operations, IOPS, and snapshot capabilities. Multi-tenancy features isolate storage access per user, enhancing security in shared environments.

Portworx on FlashArray enhances Kubernetes environments with robust data reduction, resiliency, simplicity, and support. It lowers storage costs through FlashArray’s deduplication, compression, and thin provisioning, providing 2-10x data reduction. FlashArray’s reliable infrastructure ensures high availability, reducing server-side rebuilds. Portworx simplifies Kubernetes deployment with minimal configuration and end-to-end visibility via Pure1. Additionally, unified support, powered by Pure1 telemetry, offers centralized, proactive assistance for both storage hardware and Kubernetes services, creating an efficient and scalable solution for enterprise needs.

Figure 7 shows the high-level logical storage architecture of Portworx Enterprise deployment with Pure Storage FlashArray as backend storage for the OpenShift cluster.

This is the high-level summary of the Portworx Enterprise implementation of distributed storage on a typical Kubernetes based Cluster:

●     Portworx Enterprise runs on each worker node as Daemonset pod and based on the configuration information provided in the StorageCluster spec, Portworx Enterprise provisions one or more volumes on Pure Storage FlashArray for each worker node.

●     All these Pure Storage FlashArray volumes are pooled together to form one or more Distributed Storage Pools.

●     When a PVC requested by the application, Portworx Enterprise provisions the volume from the storage pool.

●     The PVCs consume space on the storage pool, and if space begins to run low, Portworx Enterprise can add or expand drive space from Pure Storage FlashArray.

●     Portworx is designed to minimize downtime and data unavailability during worker node failures by leveraging storage-level replication and intelligent pod scheduling. Portworx volumes can be configured with multiple replicas, each stored on different worker nodes. This means that if a node fails, other nodes still have up-to-date copies of the data. Also with the help of Stork, Portworx’s scheduler extender can reschedule pods to other nodes that already have a replica of the required volume.

Data Protection with PX-Backup and Pure Storage FlashBlade

PX-Backup is a leading data protection built for any Kubernetes, securing persistent data irrespective of their deployments (on-prem or cloud). It provides enterprise-grade data protection for containerized workloads, and virtual machines hosted on variants of Kubernetes clusters like OpenShift. In this solution, PX-Backup, backed by Pure Storage FlashBlade//S200, is used to back up the various container-based applications as well as virtual machines running on OpenShift cluster. The backups can be easily schedulable based on the RTO/RPO objectives and can be restored seamlessly on to the same or different OpenShift clusters.

Pure Storage FlashBlade//S200 is an all-flash, scale-out unified fast file and object storage solution that is an ideal high-performance and scalable backup target for PX-Backup. Using Pure Storage FlashBlade//S200 with PX-Backup gives Kubernetes environments a blazing-fast, S3-compliant, on-premises backup target that ensures rapid application recovery, high scalability, and enterprise-grade for mission-critical workloads.

Infrastructure and Application Monitoring with Splunk Observability

Splunk Observability is Splunk’s cloud-based platform for full-stack observability, helping organizations monitor, troubleshoot, and optimize the performance of applications, infrastructure, and user experiences in real time. It combines metrics, traces, logs, and events into a single platform so teams can quickly detect issues and improve reliability.

Explore the benefits of Splunk Infrastructure Monitoring, Splunk APM, Splunk RUM, Splunk Synthetic Monitoring and Splunk Log Observer Connect, all in one interface (one user seat):

●     Easily detect potential issues and identify root causes using high-resolution, streaming analytics.

●     Instantly see how every user experiences your app and how each code change impacts site performance.

●     Keep total control over your data with no vendor lock-in and OpenTelemetry — instrument once, observe everywhere.

Solution Design

This chapter contains the following:

●     Design Considerations

●     FlashStack Physical Topology

●     FlashStack Cabling

●     Red Hat OpenShift Cluster on Bare Metal Server Configuration

●     OpenShift Networking and VM Networking

Design Considerations

FlashStack with Cisco UCS and Cisco Intersight meets the following design requirements:

●     Resilient design across all the layers of infrastructure with no single point of failure

●     Scalable design with the flexibility to add compute capacity, storage, or network bandwidth as needed

●     Modular design that can be replicated to expand and grow as the needs of the business grow

●     Flexible design that can support different models of various components with ease

●     Simplified design with the ability to integrate and automate with external automation tools

●     AI-Ready design to support required NVIDIA GPUs for running AI/ML based workloads

●     Cloud-enabled design which can be configured, managed, and orchestrated from the cloud using GUI or APIs

●     Unified full-stack visibility for real-time monitoring, faster troubleshooting, and improved digital resilience by correlating metrics, logs, and traces across infrastructure and applications

To deliver a solution meets all these design requirements, various solution components are connected and configured as explained in later sections.

FlashStack Physical Topology

This Red Hat OpenShift Bare Metal deployment is built over Ethernet-based design using Cisco UCS AMD M8 series servers. For this solution, Cisco Nexus 93699CD-GX switches are used to provide the connectivity between the servers and storage. ISCSI configuration on the  Cisco UCS and Pure Storage FlashArray/FlashBlade is utilized to set up storage access. The physical components and connectivity details for Ethernet -based design are covered below

Figure 8 shows the physical topology and network connections used for this Ethernet-based FlashStack design.

The reference hardware configuration includes:

●     Cisco UCS X9508 chassis, equipped with a pair of Cisco UCS X9108 100G IFMs, contains six Cisco UCS X215c M8 compute nodes and two Cisco UCS X440p PCIe nodes each with couple of NVDIA L40S and H100 NVL GPUs. Other configurations of servers with and without GPUs are also supported. Each compute node is equipped with fifth-generation Cisco UCS VIC card 15231 providing 100-G ethernet connectivity on each side of the fabric. A pair of Fabric Modules installed at the rear side of the chassis enables connectivity between the X440p PCIe nodes and X215c M8 nodes.

●     Cisco UCS Cisco UCS C245 and C225 M8 C-Series servers also validated for this solution. UCS C245 M8 is a dual socket server that  support up to 24 memory DIMMs. These servers are ideal for cpu-intensive and memory-intensive workloads that benefit from dual-CPU configurations. On the other hand, UCS C225 M8 is a single socket server supporting highest core AMD CPUs (up to 160 Cores) with plenty of PCIe slots for additional I/O expansion. Both C245 and C225 servers are equipped with UCS 5th Gen VIC 15237 dual port 40/100Gbps mLOM network card. The servers are connected to Fabric Interconnects through the VIC card.

●     Cisco fifth-generation 6536 Fabric Interconnects (FIs) are used to provide connectivity to the compute nodes installed in the chassis. These FIs are configured in End-Host mode acting like a host (not a traditional switch) to the upstream network, optimizing traffic flow and simplifying network management.

●     A pair of Nexus C93600CD-GX switches are used in Virtual Port Channel (vPC) mode. This high-speed Cisco NXOS-based Nexus C93600CD-GX switching design supports up to 100 and 400-GE connectivity.

●     The Pure Storage FlashArray //XL170 is a high-performance, all-flash storage array designed for mission-critical enterprise workloads. It is part of Pure Storage’s FlashArray //XL family, aimed at delivering extreme performance, high availability, and scalability for demanding applications such as databases, virtualization, analytics, and large-scale consolidations. In this solution, FlashArray//XL170 is used as backend storage for Portworx that provides persistent storage for both containers and virtual machines hosted on the OpenShift cluster. The storage array controllers are connected Nexus switches using 100Gbps network cards.

●     The FlashBlade S200 is a next-generation unified fast file and object storage platform from Pure Storage, designed for high-performance workloads such as modern analytics, AI/ML, rapid backup and recovery, and large-scale unstructured data management. This unified storage supports both file (NFS, SMB) and  object (S3 compliant) protocols within a single platform. In this solution, FlashBlade S200 is used as S3 compliant back end storage for PX backup which provides consistent data protection for the pods and virtual machines hosted on the OpenShift cluster.

The software components consist of:

●     Cisco Intersight platform to deploy, maintain, and support the FlashStack components.

●     Cisco Intersight Assist virtual appliance to help connect the Pure Storage FlashArray and Cisco Nexus Switches with  the Cisco Intersight platform to enable visibility into these platforms from Intersight.

●     Red Hat OpenShift Container Platform for providing a consistent hybrid cloud foundation for building and scaling containerized and virtualized applications.

●     Isovalent Networking for Kubernetes replaces the default OVN. The seamless integration of Isovalent with OpenShift enables Isovalent to provide advanced eBPF-powered networking, security, and observability for containerized applications, offering high performance, zero-trust network policies, and a unified CNI and service mesh.

●     Portworx by Pure Storage (Portworx Enterprise) data platform for providing enterprise grade storage for  containerized and virtualized workloads hosted on OpenShift platform.

●     PX-Backup for data protection of container based applications and virtual machines hosed on the Red Hat OpenShift cluster.

●     Pure Storage Pure1 is a cloud-based, AI-driven SaaS platform that simplifies and optimizes data storage management for Pure Storage arrays, offering features like proactive monitoring, predictive analytics, and automated tasks.

FlashStack Cabling

The information in this section is provided as a reference for cabling the physical equipment in a FlashStack environment.

 

Compute Infrastructure Design

The compute infrastructure in FlashStack solution consists of the following:

●     Cisco UCS X215c M8 Compute Nodes

●     Cisco UCSC-C245-M8SX and UCSC-C225-M8N

●     Cisco UCS X-Series Chassis (Cisco UCSX-9508) with Intelligent Fabric Modules (Cisco UCSX-I-9108-100G)

●     Cisco UCS Fabric Interconnects (Cisco UCS-FI-6536)

Compute System Connectivity

The Cisco UCS X9508 Chassis is equipped with the Cisco UCS X9108-100G intelligent fabric modules (IFMs). The Cisco UCS X9508 Chassis connects to each Cisco UCS 6536 FI using four 100GE ports, as shown in Figure 9. If you require more bandwidth, all eight ports on the IFMs can be connected to each FI.

Cisco UCS C245 and C225 M8 C-Series servers are equipped with Cisco UCS 5th Gen VIC 15237 dual port 40/100Gbps mLOM network card. Each C-Series server is connected to Cisco UCS 6536 FIs using two 100GE ports as shown in Figure 10.

Compute UCS Fabric Interconnect 6536 Connectivity

Cisco UCS 6536 FIs are connected to Cisco Nexus 93600CD-GX switches using 100GE connections configured as virtual port channels. Each FI is connected to both Cisco Nexus switches using a 100G connections; additional links can easily be added to the port channel to increase the bandwidth as needed. Figure 11 illustrates the physical connectivity details.

Pure Storage FlashArray//XL170 Ethernet Connectivity

Pure Storage FlashArray controllers are connected to Cisco Nexus 93600CD-GX switches using redundant 100-GE. Figure 12 illustrates the physical connectivity details.

Pure Storage FlashBlade//S200 Ethernet Connectivity

Pure Storage FlashBlade uplink ports (2x 100GbE from each FIOM) are connected to Cisco Nexus 93600CD-GX switches as shown in the below figure. Additional links can easily be added (up to 8x 100GbE on each FIOM) to the port channel to increase the bandwidth as needed. Figure 13 illustrates the physical connectivity details.

Figure 14 details the cable connections used in the validation lab for the FlashStack topology based on the 5th Generation Cisco UCS 6536 Fabric Interconnects.

On each side of the Fabric Interconnect, two 100G ports on each UCS 9108 100G IFMs are used to connect the Cisco UCS X9508 chassis to the Fabric Interconnects. Up to five 100G ports ( from 7 to 11) are used for connecting UCS C-Series servers. Two 100G port on each FI are connected to the pair of Cisco Nexus 93600CD-GX switches that are configured with a vpc domain. Each controller of Pure Storage FlashArray//Xl170  and FlashBlade//S200 arrays is connected to the pair of Nexus 93600CD-GX switches over 100G ports. As mentioned earlier, Additional 1Gbps network ( not shown in the figure) is required for Out of Band management connectivity.

Red Hat OpenShift Cluster on Bare Metal Server Configuration

A simple Red Hat OpenShift cluster consists of at least five servers – Control Plane Nodes and two or more Worker compute nodes where applications and VMs are run. The control plane nodes require less cpu and memory resources compared to worker nodes, and the resource requirements depend on the number of worker nodes in the OpenShift cluster. Hence, for dedicated control-plane deployments, low end servers (configured with low end cpu and memory configs) can be used.

Each Node is booted from RAID1 disk created using two M.2 SSD drives. In the absence of M.2 cards, the front-loaded disks, in RAID 1 configuration, can also be used for OS installation. UCS X440p PCIe Nodes provides PCIe expansion for UCS X-Series compute nodes. NVIDIA L40S and H100 NVL GPUs are installed X440p nodes and exposed to the X215c M8 nodes. For the C-Series servers, GPUs can be installed directly in the PCIe slots.

From a networking perspective, in case of dedicated control-plane nodes deployment, only single vNIC with UCS Fabric Failover option enabled is required for the Management traffic. No other vNICs are required for control plane nodes as they do not run workloads pods/VMs.  In the case of combined or mixed deployments (control plane + worker role), all the nodes are configured with several vNICs to support different network traffics as detailed in the following sections.

Red Hat OpenShift Virtualization

Red Hat OpenShift Virtualization is a feature within Red Hat OpenShift that enables you to run and manage virtual machines (VMs) alongside your container workloads in a unified Kubernetes platform. It can be enabled by installing and configuring the OpenShift Virtualization Operator and a HyperConverged Deployment as shown below. By default, the OpenShift Virtualization Operator is deployed in the openshift-cnv namespace and initial VMs can be configured there, but before VMs can be configured, VM networking and a place to store VMs will need to be set up.

OpenShift Networking and VM Networking

By default, Red Hat OpenShift is installed with networking model leveraging OVN-K CNI plugin. However, in this solution, Isovalent Enterprise is used over OVN-K to provide OpenShift networking. It integrates seamlessly with OpenShift to offer enhanced network policy enforcement, load balancing, service mesh capabilities, security, and visibility. When used with OpenShift Virtualization, Isovalent Cilium can extend its capabilities to manage and secure both containerized workloads and virtual machines running as part of the same Kubernetes/OpenShift cluster. This unified networking and security layer simplifies operations by applying consistent policies and monitoring across containers and VMs. Steps for migrating from OVN-K to Isovalent Enterprise are detailed in the following sections.

In OpenShift, the NMState Operator automates the deployment and management of NMState, allowing administrators to declaratively configure host-level networking through Kubernetes APIs without having to manually log into each node.

In this solution, NMState Operator CRD NodeNetworkConfigurationPolicy (NNCP) is used to configure the desired network settings like interfaces, bonds, bridges, IP assignments, routes and so on, using declarative yaml files. Any primary CNI plugin, such as Isovalent Cilium or OVN-Kubernetes, is responsible for providing and managing the pod’s primary (default) network interface (eth0), which is handled entirely by that plugin itself. In OpenShift, Multus is a CNI plugin that allows us to configure PODs or VMs with the multiple network interfaces. This feature is especially important in OpenShift virtualization as it is very common that a VM attached one or more networks.

In this solution, NMState Operator CRD NodeNetworkConfigurationPolicy (NNCP) is used to configure the desired network settings like interfaces, bonds, bridges, IP assignments, routes and so on, using declarative yaml files. Then, using Multus’ NetworkAttachmentDefinition CRD, additional secondary networks have been defined by specifying things like VLAN tag, bridge devices, MTU and so on. Once the NADs for secondary networks are in place, PODs or VMs can be attached to NADs to have secondary network interfaces.

Figure 15 illustrates networking is configured for OpenShift.

Figure 15 illustrates how networking is configured in OpenShift. Eno5 vNIC is used for host management as well as internal pod network. During the OpenShift installation, a default bridge (br-ex) is created, and it serves as default bridge for pod networking. Eno8 is used for VM front end connectivity for VMs hence a bridge (br-vm-mgmt) is created using NNCP. Multiple networks (for instance VM_NW_VLAN1062, VM_NW_VLAN1063 and so on) each with different vlans can be created using NADS. Eno11 is used for host object storage network (connected to FlashBlade) for PX-backup traffic. The NNCP sets the IP addresses and MTU to 9000 on this vNIC. Eno12 is used for data replication traffic by Portworx nodes running on the OpenShift cluster and configured with mtu 9000. These vNICs (eno6, eno8, eno11, and eno12) are configured with Cisco UCS Fabric Failover which will fail the vNIC over to the other FI in case of an FI failure or reboot.

Eno6 and eno7 interfaces are used for host storage access over iSCSI or NVMe-TCP protocols each vNIC pinning to the Fabric A and B respectively. NNCP sets the IP address by using DHCP, MTU to 9000 and creates additional interfaces of type VLAN for storage traffic using NVMe-TCP protocol.

Optionally, eno9 and eno10 interfaces are used for In-Guest direct access to the storage for the virtual machines using iSCSI protocol each vNIC pinning to Fabric-A and Fabric-B respectively. NNCP creates a linux bridges on each interface (iscsi-vm-a, iscsi-vm-b) and NADs will define two networks (VM_iSCSI_VLAN3010, VM_iSCSI_VLAN3020) for VMs storage access.

VLAN Configuration

Table 1 lists the VLANs configured for setting up the FlashStack environment along with their usage

Table 1.        VLANs used in this solution

Table 2 lists the infrastructure services running on either virtual machines or bar mental servers required for deployment as outlined in the document. All these services are hosted on pre-existing infrastructure with in the FlashStack.

Table 2.        Infrastructure services

Software Revisions

The FlashStack Solution with Red Hat OpenShift on Bare Metal infrastructure configuration is built using the following components.

Table 3 lists the required software revisions for various components of the solution.

Table 3.        Software Revisions

Deployment Hardware and Software

This chapter contains the following:

●     Physical Connectivity

●     Cisco Nexus Switch Manual Configuration

●     Claim Cisco Nexus Switches into Cisco Intersight

Physical Connectivity

Physical cabling should be completed by following the diagram and table references in section FlashStack Cabling.

The following procedures describe how to configure the Cisco Nexus 93600CD-GX switches for use in a FlashStack environment. This procedure assumes the use of Cisco Nexus 9000 10.1(2), the Cisco suggested Nexus switch released at the time of this validation.

The procedure includes the setup of NTP distribution on both the mgmt0 port and the in-band management VLAN. The interface-vlan feature and ntp commands are used to set this up. This procedure also assumes that the default VRF is used to route the in-band management VLAN.

This document assumes that initial day-0 switch configuration is already done using switch console ports and ready to use the switches using their management IPs.

Cisco Nexus Switch Manual Configuration

Procedure 1.    Enable features on Cisco Nexus A and Cisco Nexus B

Step 1.          Log into both Nexus switches as admin using ssh.

Step 2.          Enable the switch features as described below:

Procedure 2.    Set Global Configurations on Cisco Nexus A and Cisco Nexus B

Step 1.          Log into both Nexus switches as admin using ssh.

Step 2.          Run the following commands to set the global configurations:

Sample clock commands for the United States Eastern timezone are:

●     clock timezone EST -5 0

●     clock summer-time EDT 2 Sunday March 02:00 1 Sunday November 02:00 60

Procedure 3.    Create VLANs on Cisco Nexus A and Cisco Nexus B

Step 1.          From the global configuration mode, run the following:

Procedure 4.    Add NTP Distribution Interface

Cisco Nexus - A

Step 1.          From the global configuration mode, run the following commands:

Cisco Nexus - B

Step 1.          From the global configuration mode, run the following commands:

Procedure 5.    Define Port Channels on Cisco Nexus A and Cisco Nexus B

Cisco Nexus – A and B

Step 1.          From the global configuration mode, run the following commands:

Procedure 6.    Configure Virtual Port Channel Domain on Nexus A and Cisco Nexus B

Cisco Nexus - A

Step 1.          From the global configuration mode, run the following commands:

Cisco Nexus - B

Step 1.          From the global configuration mode, run the following commands:

Procedure 7.    Configure individual Interfaces

Cisco Nexus-A

Step 1.          From the global configuration mode, run the following commands:

Cisco Nexus-B

Step 1.          From the global configuration mode, run the following commands:

Procedure 8.    Update the port channels

Cisco Nexus-A and B

Step 1.          From the global configuration mode, run the following commands:

Step 2.          To check for correct switch configuration, run the following commands:

Cisco Nexus Configuration for Storage Traffic

Procedure 1.    Configure Interfaces for Pure Storage on Cisco Nexus and Cisco Nexus B

Cisco Nexus - A

Step 1.          From the global configuration mode, run the following commands:

Cisco Nexus - B

Step 1.          From the global configuration mode, run the following commands:

Claim Cisco Nexus Switches into Cisco Intersight

Cisco Nexus switches can be claimed into the Cisco Intersight either using Cisco Intersight Assist or Direct claim using Device ID and Claim Codes.

This section provides the steps to claim the Cisco Nexus switches using Cisco Intersight Assist.

Procedure 1.    Claim Cisco Nexus Switches into Cisco Intersight using Cisco Intersight Assist

Cisco Nexus - A

Step 1.          Log into Nexus Switches and confirm the nxapi feature is enabled:

Step 2.          Log into Cisco Intersight with your login credentials. From the drop-down list select System.

Step 3.          Under Admin, click Target then click Claim a New Target. Under Categories, select Network, click Cisco Nexus Switch and then click Start.

Step 4.          Select the Cisco Assist name which is already deployed and configured. Provide the Cisco Nexus Switch management IP address, username and password details and click Claim.

Step 5.          Repeat steps 1 through 4 to claim the remaining Switch B.

Step 6.          When the switches are successfully claimed, from the drop-down list, select Infrastructure Services. Under Operate, click the Networking tab. On the right you will find the newly claimed Cisco Nexus switch details and browse through the Switches for viewing the inventory details.

The L2 neighbors of the Cisco Nexus Switch-A is shown below:

Cisco Intersight Managed Mode Configuration for Cisco UCS

The chapter contains the following:

●     Fabric Interconnect Domain Profile and Policies

●     Server Profile Templates and Policies

●     Create Pools

●     vNIC Templates and vNICs

●     Ethernet Adapter Policy for Storage Traffic

●     Storage Policy

●     Compute Configuration Policies

●     Management Configuration Policies

The procedures in this chapter describe how to configure a Cisco UCS domain for use in a base FlashStack environment. A Cisco UCS domain is defined as a pair for Cisco UCS FIs and all the X-Series and C-Series servers connected to it. These can be managed using two methods: UCSM and IMM. The procedures detailed below are for Cisco UCS Fabric Interconnects running in Intersight managed mode (IMM).

The Cisco Intersight platform is a management solution delivered as a service with embedded analytics for Cisco and third-party IT infrastructures. The Cisco Intersight Managed Mode (also referred to as Cisco IMM or Intersight Managed Mode) is an architecture that manages Cisco Unified Computing System (Cisco UCS) fabric interconnect–attached systems through a Redfish-based standard model. Cisco Intersight managed mode standardizes both policy and operation management for Cisco UCS C-Series M8 and Cisco UCS X-Series M8 compute nodes used in this deployment guide.

Fabric Interconnect Domain Profile and Policies

This section contains the procedures to create fabric interconnect domain profiles and policies.

Procedure 1.    Create Fabric Interconnect Domain Profile and Policies

Step 1.          Log into the Intersight portal and select Infrastructure Service. On the left select Profiles then select UCS Domain Profiles.

Step 2.          Click Create UCS Domain Profile to create a new domain profile for Fabric Interconnects. Under the General tab, select the Default Organization, enter a name and descriptions of the profile.

Step 3.          Click Next to go to UCS Domain Assignment. Click Assign Later.

Step 4.          Click Next to go to VLAN & VSAN Configuration.

Step 5.          Under VLAN & VSAN Configuration > VLAN Configuration, click Select Policy then click Create New.

Step 6.          On the Create VLAN page, go to the General tab, enter a name (AA06-FI-VLANs) and click Next to go to Policy Details.

Step 7.          To add a VLAN, click Add VLANs.

Step 8.          For the Prefix, enter the VLAN name as OOB-Mgmt-VLAN. For the VLAN ID, enter the VLAN ID 1061. Leave Auto Allow on Uplinks enabled and Enable VLAN Sharing disabled.

Step 9.          Under Multicast Policy, click Select Policy and select Create New to create a Multicast policy.

Step 10.       On the Create Multicast Policy page, enter the name (AA06-FI-MultiCast) of the policy and click Next to go to Policy Details. Leave the Snooping State and Source IP Proxy state checked/enabled and click Create. Select the newly created Multicast policy.

Step 11.       Repeat steps 1 through 10 to add all the required VLANs to the VLAN policy.

Step 12.       After adding all the VLANs, click Set Native VLAN ID and enter the native VLANs (for example 2) and click Create. The VLANs used for this solution are shown below:

Step 13.       Select the newly created VLAN policy for both Fabric Interconnects A and B. Click Next to go to Ports Configuration. Create new Ports Configuration Policy.

Step 14.       Enter the name of the policy (AA03-FI-PortConfig) and skip unified ports and Break out Options by clicking Next then click Next again to go to Port Roles Page.

Step 15.       For defining server ports, click Port Roles, select ports from 3 to 11 and click Configure. For Role, select Server for and click Save.

Step 16.       Go to Port Channels > Create Port Channel . Set role to Ethernet Uplink Port Channel. Enter 201 for the Port Channel ID. Set Admin speed to 100Gbps and FEC to Cl91.

Step 17.       Under Link Control, create a new link control policy with the following options. Once created, select the policy.

Table 4.        UDLD Policy

Step 18.       For the Uplink Port Channel select Ports 1 and 2 and click Create to complete the Port Roles policy.

Step 19.       Click Next to go to UCS Domain Configuration page.

The following tables lists the Management and Network related policies that are created and used.

Table 5.        NTP policy

Table 6.        Network Connectivity Policy

Table 7.        SNMP Policy

Table 8.        QoS Policy

Step 20.       When the UCS Domain profile is created with the policies, edit the policy and assign it to the Fabric Interconnects.

Intersight will go through the discovery process and discover all the Cisco UCS C and X-Series compute nodes attached to the Fabric Interconnects.

Server Profile Templates and Policies

In the Cisco Intersight platform, a server profile enables resource management by simplifying policy alignment and server configuration. The server profiles are derived from a server profile template. A Server profile template and its associated policies can be created using the server profile template wizard. After creating the server profile template, you can derive multiple consistent server profiles from the template.

The server profile templates captured in this deployment guide supports Cisco UCS X210c M7 compute nodes with 5th Generation VICs and can be modified to support other Cisco UCS blades and rack mount servers.

Create Pools

The following pools need to be created before proceeding with server profile template creation.

MAC Pools

Table 9 lists the two MAC pools for the vNICs that will be configured in the templates.

Table 9.        MAC Pool Names and Address Ranges

UUID Pool

Table 10 lists the settings for the UUID pools.

Table 10.     UUID Pool Names and Settings

Procedure 1.    Create Out-Of-Band (OOB) Management IP Pool

Step 1.          Create the OOB management IP pool (AA06-OCP-OOB-MGMT-IPPool) with the following settings:

vNIC Templates and vNICs

For this mixed cluster validation, where control plane nodes are configured to run the workloads, control plane nodes also need to be configured with storage network interfaces. Hence single server profile template is created for both worker and control plane nodes.

The following vNIC templates are used for deriving the vNICs for OpenShift nodes for host management, VM management and storage traffics.

Table 11.     vNIC Templates for Ethernet Traffic

Ethernet Adapter Policy for Storage Traffic

The ethernet adapter policy is used to set the interrupts, send and receive queues, and queue ring size. The values are set according to the best-practices guidance for the operating system in use. Cisco Intersight provides a default Linux Ethernet Adapter policy for typical Linux deployments.

Optionally, you can configure a tweaked ethernet adapter policy for additional hardware receive queues handled by multiple CPUs in scenarios where there is a lot of traffic and multiple flows. In this deployment, a modified ethernet adapter policy, AA06-EthAdapter-16RXQs-5G, is created and attached to storage vNICs eno6, eno7, eno9, eno10, eno11 and eno12. Other vNICs (eno5 and eno8) will use the default Linux-v2 Ethernet Adapter policy. Table 12 lists the settings that are changed from defaults in the Adapter policy used for the iSCSI traffic. The remaining settings are left at defaults

Table 12.     Ethernet Adapter Policy used for Storage traffic

Using the templates listed in Table 11, single LAN connectivity policy is created for control plane and worker nodes.

Storage Policy

For this solution, Cisco UCS AMD M8 (X and C-Series) nodes are configured to boot from local M.2 SSD disks. Two M.2 disks are used and configured with RAID-1 configuration. Boot from SAN option will be supported in upcoming releases. The following screenshot shows the storage policy, and the settings used for configuring the M.2 disks in RAID-1 configuration.

Compute Configuration Policies

Boot Order Policy for M2

To facilitate the automatic boot from the Red Hat CoreOS Discovery ISO image, CIMC Mapped DVD boot option is used. The following boot policy is used for both controller and workers nodes.

The Local Disk boot option being at the top ensures that the nodes always boot from the M.2 disks once after CoreOS installed. The CIMC Mapped DVC option at the second is used to install the CoreOS using Discovery ISO which is mapped using a Virtual Media policy (CIMCMap-ISO). KVM Mapped DVD will be used if you want to manually mount any ISO to the KVM session of the server and install the OS. This option will be used when installing CoreOS during the OpenShift cluster expansion by adding additional worker node.

Virtual Media (vMedia) Policy

Virtual Media policy is used to mount the Red Hat CoreOS Discovery ISO to the server using CIMC Mapped DVD policy as previously explained.

BIOS Policy

Create a BIOS policy by selecting pre-defined policy “Virtualization-M8-AMD” as shown below:

For more information about tunings for the AMD CPU BIOS, refer to : https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-c-series-rack-servers/ucs-c245-m8-rack-ser-4th-gen-amd-epyc-pro-wp.html

Firmware Policy

The following screenshot shows the firmware versions used for each type of servers in this validation:

Power Policy

Create a Power Policy using the default values as shown below. Ensure to select UCS Server (FI-Attached) option.

Management Configuration Policies

The following policies will be added to the management configuration:

●     IMC Access to define the pool of IP addresses for compute node KVM access

●     IPMI Over LAN to allow the servers to be managed by IPMI or redfish through the BMC or CIMC

●     Local User to provide local administrator to access KVM

●     Virtual KVM to allow the Tunneled KVM

Cisco IMC Access Policy

Create a CIMC Access Policy with settings as shown in the following screenshot.

Since certain features are not yet enabled for Out-of-Band Configuration (accessed using the Fabric Interconnect mgmt0 ports), you need to access the OOB-MGMT VLAN (1060) through the Fabric Interconnect Uplinks and mapping it as the In-Band Configuration VLAN.

IPMI over LAN and Local User Policies

The IPMI Over LAN Policy can be used to allow both IPMI and Redfish connectivity to Cisco UCS Servers. Red Hat OpenShift platform uses these two policies to power manage (power off, restart, and so on) the OpenShift bare metal servers.

Create IPMI over LAN policy and Local User policies as shown below:

Virtual KVM Policy

The following screenshot shows the virtual KVM policy used in the solution:

Create Server Profile Templates

When you have the required pools, polices, vNIC templates created, Server profile templates can be created. Single Server Profile Template is used for control plane and worker nodes.

Table 13 lists the polices and pools used to create the Server Profile template (AA06-OCPM2Boot-Worker-M8AMD).

Table 13.     Policies and Pools for Control Nodes

The following screenshot shows the server profile templates created for the control and worker nodes:

Create Server Profiles

Once Server Profile Templates are created, the server profiles can be derived from the template. The following screenshot shows a total of eight server profiles are derived (three for control nodes and five for worker nodes).

When the Server profiles are created, associate these server profiles to the control and workers nodes as shown below:

Now the Cisco UCS AMD M8 servers are ready for OpenShift installation.

Pure Storage FlashArray Configuration

This chapter contains the following:

●     iSCSI Interfaces

In this solution, Pure Storage FlashArray//XL170 is used as the storage provider for all the application pods and virtual machines provisioned on the OpenShift cluster using Portworx Enterprise. The Pure Storage FlashArray//XL170 array will be used as Cloud Storage Provider for Portworx which allows us to store data on-premises with FlashArray while benefiting from Portworx Enterprise cloud drive features.

This chapter describes the high-level steps to configure Pure Storage FlashArray//X170 network interfaces required for storage connectivity over iSCSI.

The compute nodes are redundantly connected to the storage controllers through 4 x 100Gb connections (2 x 100Gb per storage controller module) from the redundant Cisco Nexus switches.

The Pure Storage FlashArray network settings were configured with three subnets across three VLANs. Storage Interfaces CT0.Eth0 and CT1.Eth0 were configured to access management for the storage on VLAN 1030. Storage Interfaces (CT0.Eth18, CT0.Eth19, CT1.Eth18, and CT1.Eth19) were configured to run iSCSI/NVMe-TCP Storage network traffic on the VLAN 3010 and VLAN 3020.

The following tables list the IP addressing configured on the interfaces used for storage access.

Table 14.     iSCSI A Pure Storage FlashArray//XL170 Interface Configuration Settings

Table 15.     iSCSI B Pure Storage FlashArray//XL170 Interface Configuration Settings

iSCSI Interfaces

This section contains the procedures to configure the iSCSI interfaces.

Procedure 1.    Configure iSCSI Interfaces

Step 1.          Log into Pure FlashArray//XL170 using its management IP addresses.

Step 2.          Click Settings > Network > Connectors > Ethernet.

Step 3.          Click Edit for Interface CT0.eth18.

Step 4.          Click Enable and add the IP information from Table 14 and Table 15 and set the MTU to 9000.

Step 5.          Click Save.

Step 6.          Repeat steps 1 through 5 to configure the remaining interfaces CT0.eth19, CT1.eth18 and CT1.eth19.

Procedure 2.    Configure Storage Interfaces for NVMe-TCP

The following steps are required only when you want to use nvme-tcp (instead of iscsi) protocol for accessing storage target

Step 1.          ssh to the Pure FlashArray//Xl170 using its management ip and pureuser credentials.

Step 2.          Enable nvme-tcp service on all the four ethernet interfaces as shown below:

Procedure 3.    Claim Pure Storage FlashArray//XL170 into Intersight

Step 1.          Log into Cisco Intersight using your login credentials. From the drop-down menu select System.

Step 2.          Under Admin, select Target and click Claim a New Target. Under Categories, select Storage, click Pure Storage FlashArray and then click Start.

Step 3.          Select the Cisco Assist name which is already deployed and configured. Provide the Pure Storage FlashArray management IP address, username, and password details and click Claim.

Step 4.          When the storage is successfully claimed, from the drop-down list, select Infrastructure Services. Under Operate, click Storage. You will see the newly claimed Pure Storage FlashArray; browse through it to view the inventory details.

Pure Storage FlashBlade Configuration

This chapter contains the following:

●     Object Store Configuration

In this solution, Pure Storage FlashBlade//S200 is used as the S3 compliant on-prem object storage provider for the PX-Backup as explained in the previous sections. The FlashBlade is directly accessed by the PX-Backup pods by using Workers node’s interface that carries object storage traffic (eno11). This section describes high-level steps to configure Pure Storage FlashBlade//S200 network interfaces required for storage connectivity over ethernet.

The FlashBlade//S200 provides up to 8x 100GbE interfaces for data traffic. In this solution, 2x 100GbE from each FIOM (with aggregated network bandwidth of 400 GbE) are connected to a pair of Nexus switches.

Table 16 lists the IP addressing configured on the interfaces used for objects storage access.

Table 16.     Link Aggregation Groups (LAG)

A Link Aggregation Group is created using CH1.FM1.ETH3, CH1.FM1.ETH4, CH1.FM2.ETH3, and CH1.FM2.ETH4 interfaces as shown below. Notice that the aggregated bandwidth of the LAG is 400GbE as it is created with 4x 100GbE interfaces as shown below:

Once the LAG is created, a Subnet (AA06-Object) and an interface (AA06-Obj-Interface) are created as shown below. For the subnet, ensure to set MTU as 9000, VLAN as 3040 and select aa03 for the LAG. Ensure to set Services as Data for the interface.

Object Store Configuration

FlashBlade Object Store configuration involves creating an Object Store Account, User, Access Keys and a Bucket.

Procedure 1.    Create Object Store Account, User, Access Keys and Bucket

Step 1.          Log into Pure FlashBlade//S200 using its management IP addresses.

Step 2.          Click Storage > Object Store > Accounts > Click + to create an account.

Step 3.          Provide a name for Account, Quota Limit and Bucket Default Quota Limit as per your requirements. Click Create.

Step 4.          Click the newly created Account name and click + to create a new user for the account.

Step 5.          Provide a username and click Create.

Step 6.          In the Add Access Policies window, select the pre-defined access policies as shown below. Or create your own access policy with a set of rules and finally select it.

Step 7.          Click Add when Access Policy is selected.

Step 8.          Click Create a new key and click Create to create a pair of access and secret keys. Preserve the Access and Secret Keys for later use. Click Close.

Step 9.          Ensure to note and backup the Access Key ID and Secret Access key as you need these details to configure the PX-Backup.

Step 10.       Click Account name and go to the Buckets sections. Click + to create a bucket and add it to the account.

Step 11.       Select the account created in the previous step and input values for Bucket Name and Quota Limit. Click Create.

OpenShift Installation and Configuration

This chapter contains the following:

●     OpenShift Container Platform – Installation Requirements

●     Prerequisites

●     Network Requirements

OpenShift v4.18 is deployed on Cisco UCS infrastructure as M.2 booted bare metal servers. The Cisco UCS AMD M8 servers need to be equipped with an M.2 controller (SATA or NVMe) card, and two identical M.2 drives. Three control plane nodes and four worker nodes are deployed in the validation environment and additional worker nodes can easily be added to increase the scalability of the solution. This document will guide you through the process of using the Assisted Installer to deploy OpenShift 4.18.

OpenShift Container Platform – Installation Requirements

The Red Hat OpenShift Assisted Installer provides support for installing OpenShift Container Platform on bare metal nodes. This guide provides a methodology to achieving a successful installation using the Assisted Installer.

Prerequisites

The FlashStack for OpenShift utilizes the Assisted Installer for OpenShift installation. Therefore, when provisioning and managing the FlashStack infrastructure, you must provide all the supporting cluster infrastructure and resources, including an installer VM or host, networking, storage, and individual cluster machines.

The following supporting cluster resources are required for the Assisted Installer installation:

●     The control plane and compute machines that make up the cluster

●     Cluster networking

●     Storage for the cluster infrastructure and applications

●     The Installer VM or Host

Network Requirements

The following infrastructure services need to be deployed to support the OpenShift cluster, during the validation of this solution we have provided VMs on your hypervisor of choice to run the required services. You can use the existing DNS and DHCP services available in the data center.

There are various infrastructure services prerequisites for deploying OpenShift 4.18. These prerequisites are as follows:

●     DNS and DHCP services – these services were configured on Microsoft Windows Server VMs in this validation

●     NTP Distribution was done with Nexus switches

●     Specific DNS entries for deploying OpenShift – added to the DNS server

●     A Linux VM for initial automated installation and cluster management – a Rocky Linux / RHEL VM with appropriate packages

NTP

Each OpenShift Container Platform node in the cluster must have access to at least two NTP servers.

NICs

NICs configured on the Cisco UCS servers based on the design previously discussed.

DNS

Clients access the OpenShift Container Platform cluster nodes over the bare metal network. Configure a subdomain or subzone where the canonical name extension is the cluster name.

The following domain and OpenShift cluster names are used in this deployment guide:

●     Base Domain: flashstack.local

●     OpenShift Cluster Name: fs-ocp

The DNS domain name for the OpenShift cluster should be the cluster name followed by the base domain, for example fs-ocp.flashstack.local.

Table 17 lists the information for fully qualified domain names used during validation. The API and Nameserver addresses begin with canonical name extensions. The hostnames of the control plane and worker nodes are exemplary, so you can use any host naming convention you prefer.

Table 17.     DNS FQDN Names Used

For initial deployment, only first seven nodes ( three control plane and four worker nodes) are used for OpenShift deployment. The last node amdn8.fs-ocp.flashstack.local will be added later as part of the cluster expansion by adding a new server node to the already existing cluster.

DHCP

For the bare metal network, a network administrator must reserve several IP addresses, including:

●     One IP address for the API endpoint

●     One IP address for the wildcard Ingress endpoint

●     One IP address for each control node (DHCP server assigns to the node)

●     One IP address for each worker node (DHCP server assigns to the node)

Procedure 1.    Gather MAC Addresses of Node Bare Metal Interfaces

Step 1.          Log into Cisco Intersight.

Step 2.          Go to Infrastructure Service > Profiles > click any UCS Server Profile.

Step 3.          In the center pane, go to Inventory > Network Adapters > Network Adapter.

Step 4.          In the center pane, click Interfaces.

Step 5.          Record the MAC address for NIC Interface eno5.

Step 6.          Select the General tab and click Identifiers.

Step 7.          Record the Management IP assigned from the AA06-OCP-OOB-MGMT-IP Pool.

Table 18 lists the IP addresses used for the OpenShift cluster including bare metal network IPs and UCS KVM Management IPs for IMPI or Redfish access.

Table 18.     Host BMC Information

Step 8.          From Table 18, enter the hostnames, IP addresses, and MAC addresses as reservations in your DHCP and DNS server(s) or configure the DHCP server to dynamically update DNS.

Step 9.          You need to pipe VLAN interfaces for all three storage VLANs (3010, 3020 and 3040) and three management VLANs (1061,1062,1063) into your DHCP server(s) and assign IPs in the storage networks on those interfaces.

Step 10.       Create a DHCP scope for each management and storage VLANs with the appropriate subnets.

Step 11.       Ensure that the IPs assigned by the scope do not overlap with the already consumed IPs (like FlashArray//XL170 storage iSCSI interface IPs, FlashBlade//S200 interfaces and OpenShift reserved IPs).

Step 12.       Either enter the nodes in the DNS server or configure the DHCP server to forward entries to the DNS server. For the cluster nodes, create reservations to map the hostnames to the desired IP addresses as shown in below:

Step 13.       Setup either a VM (installer/bastion node) or spare server with the network interface connected to the Host management VLAN (1061).

Step 14.       Install either Red Hat Enterprise Linux (RHEL) 9.4 or Rocky Linux 9.4 Server with GUI and create an administrator user. Once the VM or host is up and running, update it and install and configure XRDP. Connect to this host with a Windows Remote Desktop client as the admin user.

Step 15.       ssh into the installer node VM, open a terminal session and create an SSH key pair to use to communicate with the OpenShift hosts:

Step 16.       Copy the public SSH key to the user directory:

Step 17.       Add the private key to the ssh-agent:

Procedure 2.    Install Red Hat OpenShift Container Platform using the Assisted Installer

Step 1.          Launch Firefox and connect to https://console.redhat.com/openshift/cluster-list. Log into your Red Hat account.

Step 2.          Click Create cluster to create an OpenShift cluster.

Step 3.          Select Datacenter and then select Bare Metal (x86_64).

Step 4.          Select Interactive to launch the Assisted Installer.

Step 5.          Provide the cluster name and base domain.

Step 6.          Select the latest OpenShift version (4.18.20 or above), scroll down and click Next.

Step 7.          Do not select the operator and click Next.

Step 8.          Click Add hosts.

Step 9.          Under Provisioning type, from the drop-down list select the Full Image file. Under SSH public key, click Browse and browse to, select, and open the id_ed25519.pub file. The contents of the public key should now appear in the box. Click Generate Discovery ISO. Now the Discovery ISO is ready for download. Click the Add Hosts from Cisco Intersight link to select the server list to boot them using Discovery ISO.

Step 10.       A new Intersight workflow Boot server from ISO URL window appears. Click the pencil symbol and select the required servers and click Close.

The Intersight workflow will be created to download the Discovery ISO, mount the ISO to the selected hosts, and boot them using the ISO. The workflow task details is shown below:

Step 11.       Wait for all servers to boot and appear in the Red Hat Assisted installer console. The server will get names and IPs assigned from the DHCP reservations configured in the previous steps.

Step 12.       Enable the Run workloads on the control plane nodes radio button to schedule workloads on the control plane nodes.

Step 13.       From the drop-down list under Role assign the appropriate server roles. Scroll down and click Next. Configure the first nodes as control plane nodes and the rest as workers.

Step 14.       Expand each node and confirm the role of the M.2 disk is set to Installation disk. Click Next.

Step 15.       Under Network Management, make sure Cluster-Managed Networking is selected. Under Machine network, from the drop-down list, select the subnet for the host management network. Enter the IPs for API IP (api.fs-ocp.flashstack.local) and Ingress IP (*.apps.fs-ocp.flashstack.local).

Step 16.       When all the nodes are in ready status, click Next. Review all the settings and click Install Cluster. The OpenShift Cluster will be installed as shown below:

Step 17.       Click Web Console URL to log into the newly created OpenShift cluster with kubeadmin user and verify the cluster settings.

Step 18.       Once the cluster is installed, ensure to download the kubeconfig file. Upload the file Installer VM and from a terminal window, setup a cluster directory and save the kubeconfig credentials:

Step 19.       In the Assisted Installer, click the icon to copy the kubeadmin password:

Step 20.       Click Open console to launch the OpenShift Console. Log in using the kubeadmin and the kubeadmin password.

Step 21.       Click the ? mask. Links for various tools are provided in that page. Download oc for Linux for x86_64 and virtctl for Linux for x86_64 Common Line Tools.

Step 22.       To enable oc tab completion for bash, run the following:

You should now be able to use oc to fetch the OpenShift node details:

Procedure 3.    Update Bare metal Hosts with IPMI user details

This procedure provides the steps to edit the bare metal host configuration with IPMI user details to perform some of the basic maintenance tasks, like restart and poweroff operations.

Step 1.          In the Red Hat OpenShift Console, go to Compute -> Bare Metal Hosts. For each Bare Metal Host, click the ellipses to the right of the host and select Edit Bare Metal Host. Select Enable power management.

Step 2.          From Table 18, fill in the BMC Address and make sure the Boot MAC Address matches the MAC address. For the BMC Username and BMC Password, use what was entered into the Cisco Intersight IPMI over LAN policy. Click Save to save the changes. Repeat this step for all Bare Metal Hosts.

Step 3.          Go to Compute > Bare Metal Hosts. When all hosts have been configured, the Status displays “Externally provisioned,” and the Management Address are populated. You can now manage power on the OpenShift hosts from the OpenShift console.

Step 4.          To automatically determine and allocate the system-reserved resources on nodes, create a KubeletConfig CUSTOM RESOURCE (CR) to set the autoSizingReserved: true parameter as shown below and apply the machine configuration files:

Procedure 4.    Install NMState Operator and configure Host Network Interfaces

As previously explained, NMState operator is used to configure the storage networking interfaces and other Linux bridges on the interfaces. Use the following steps to configure storage network interfaces for iSCSI and NVMe-TCP protocols.

Step 1.          In the Red Hat OpenShift Console, go to Operators > OperatorHub. Search for NMState and install Kubernetes NMState Operator.

Step 2.          Click Install. Leave all the defaults in place and click Install again. The operator will take a few minutes to install. Once the operator is installed, click View Operator.

Step 3.          Select the NMState tab. On the right, click Create NMState. Leave all defaults in place and click Create. The nmstate will be created. You will also need to refresh the console because additional items will be added under Networking.

Step 4.          Create the following yaml scripts in a folder:

Step 5.          Apply the files and verify that all interfaces are configured accordingly:

In the following sections, the required NADs will be created for VMs/PODs to get access to these bridges.

Procedure 5.    Add an Additional Administrator User

It is recommended to install a permanent administrative user to an OpenShift cluster to provide an alternative to logging in with the “temporary” kubeadmin user.

For this validation, an HTPasswd user is created with admin rights on the OpenShift cluster by following the instructions detailed here: https://access.redhat.com/solutions/4039941.

Step 1.          After creating an admin user using the HTPasswd method, logout from the current session and login again by selecting the htpasswd option as shown below:

Install NVIDIA GPU Operator and Drivers

This chapter contains the following:

●     NVIDIA GPU Operator

This chapter provides the procedures to install the required NVIDIA GPU operators, CUDA drivers and so on, to work with NVIDIA GPUs.

NVIDIA GPU Operator

This section provides the procedures to install the NVIDIA GPU operators.

Procedure 1.    Install NVIDIA GPU Operators

If you have GPUs installed in your Cisco UCS servers, you need to install the Node Feature Discovery (NFD) Operator to detect NVIDIA GPUs and the NVIDIA GPU Operator to make these GPUs available to containers and virtual machines.

Step 1.          In the OpenShift Container Platform web console, click Operators > OperatorHub.

Step 2.          Type Node Feature in the filter box and then click the Node Feature Discovery Operator with Red Hat in the upper right corner. Click Install.

Step 3.          Do not change any settings and click Install.

Step 4.          When the Install operator is ready for use, click View Operator.

Step 5.          In the bar to the right of Details, click NodeFeatureDiscovery.

Step 6.          Click Create NodeFeatureDiscovery.

Step 7.          Click Create.

Step 8.          When the nfd-instance has a status of Available, Upgradeable, click Compute > Nodes.

Step 9.          Select a node that has one or more GPUs and then click Details.

The label feature.node.kubernetes.io/pci-10de.present=true should be present on the host.

This label appears on all nodes with GPUs:

Step 10.       Go to Operators > OperatorHub.

Step 11.       Type NVIDIA in the filter box and then click the NVIDIA GPU Operator. Click Install.

Step 12.       Do not change any settings and click Install.

Step 13.       When the Install operator is ready for use, click View Operator.

Step 14.       In the bar to the right of Details, click ClusterPolicy.

Step 15.       Click Create ClusterPolicy.

Step 16.       Do not change any settings and scroll down and click Create. This will install the latest GPU driver.

Step 17.       Wait for the gpu-cluster-policy Status to become Ready.

Step 18.       Connect to a terminal window on the OpenShift Installer machine. Type the following commands. The output shown is for two servers that are equipped with GPUs:

Step 19.       Connect to one of the nvidia-driver-daemonset containers and view the GPU status:

Procedure 2.    Enable the GPU Monitoring Dashboard

Step 1.          Go to https://docs.nvidia.com/datacenter/cloud-native/openshift/latest/enable-gpu-monitoring-dashboard.html, enable the GPU Monitoring Dashboard to monitor GPUs in the OpenShift Web-Console.

Migrate OpenShift OVN to Isovalent Networking for Kubernetes

This chapter contains the following:

●     Migrate OpenShift Cluster from OpenShift OVN to Isovalent Networking for Kubernetes

●     Deploy Hubble UI and CLI

Migrate OpenShift Cluster from OpenShift OVN to Isovalent Networking for Kubernetes

The OVN-Kubernetes network plugin is the default network provider for OpenShift. As previously stated,  this FlashStack deployment uses Isovalent Enterprise as CNI plugin for providing OpenShift networking.

This section provides the steps to migrate to Isovalent Networking for Kubernetes in an existing OpenShift cluster.

Procedure 1.    Disable the Cluster Network Operator

The Cluster Network Operator (CNO) deploys and manages cluster network components in OpenShift, including OVN-Kubernetes. The first step in a migration is to temporarily disable this operator to prevent overwriting changes you make to the cluster network configuration. The following screenshot shows the components of OpenShift-network-operator namespace:

Step 1.          Log into Installer VM, create a folder and then create .yaml file as shown below. Patch the cluster with this file.

Step 2.          Scale down the network operator from 1 to 0:

Step 3.          Delete the applied-cluster configmap in the openshift-network-operator namespace. This removes the state file created when the cluster was initially deployed.

Procedure 2.    Change Network plugin to Cilium

When changes are made to network objects, the Machine Config Operator will automatically reboot nodes to make them compliant with the new configuration. To prevent this until the nodes are rebooted later in this process, you must put a "pause" on management.

Step 1.          Apply the pause to the automatic node reboots:

Step 2.          Configure the Cluster Network Operator to use Isovalent Enterprise. Review the network.config object specifications before making changes.

Step 3.          Patch the network.config object with network CIDR you want to use for Cilium. This example uses 10.253.0.0/16 as CIDR for Cilium pod network.

Step 4.          Patch the network.cluster object to set the default network for the cluster to Cilium, keep kube-proxy from being deployed, and use the CIDR you've chosen. Isovalent Networking for Kubernetes will replace kube-proxy functionality:

Procedure 3.    Configure Isovalent Networking Kubernetes Manifests

Step 1.          Download the Isovalent Networking for Kubernetes manifests from the Install Networking for Kubernetes on Red Hat OpenShift guide and extract them locally.

The following screenshot shows the files that make up the entire Cilium CNI and are important to configure for any customization:

Step 2.          Customize the Isovalent deployment according to our environment and requirements by updating ciliumconfig.yaml file.

The following screenshot highlights the changes in bold made to the ciliumconfig.yaml:

Some layered products like OpenShift Service Mesh, OpenShift Virtualization and OpenShift sandboxed containers may not work well with socket level load balancing. Cilium supports bypassing the socket level load balancers by setting spec.socketLB.hostNamespaceOnly: true.

We need to explicitly mention the openvswitch devices under spec.devices: “br-ex,eno5”. Here, br-ex is the openvswitch used by OVN-K and eno5 is the default network device used by Cilium. Replace k8sServiceHost and k8sServicePort with the internal API server hostname and API port for your environment.

Hubble is part of Cilium platform, and it is a fully distributed networking and security observability platform for cloud native workloads. It is built on top of Cilium and eBPF to enable deep visibility into the communication and behavior of services as well as the networking infrastructure in a completely transparent manner. Relay is an extra component that aggregates observability data across multiple Hubble server instances across all nodes (or even clusters) in your setup. Enable both Hubble and Relay by editing the spec.hubble section as previously stated.

Step 3.          Add your OpenShift server details to the apps_v1_deployment_clife-controller-manager.yaml file under the env section as shown below:

Step 4.          Apply all files under clife folder:

Step 5.          Wait until all the corresponding pods comes up neatly in Cilium namespace.

Step 6.          The OpenShift Multus configuration will need to be changed to point to Cilium instead of OVN-Kubernetes. The following screenshot shows Multus pointing to OVN configuration file:

Step 7.          Run the following command to replace the string /host/run/Multus/cni/net.d/10-ovn-kubernetes.conf with host/run/multus/cni/net.d/05-cilium.conflist. When the string is replaced, restart multus operator:

The following screenshot shows the multus configuration after replacing the OVN config file with the Cilium config file:

Step 8.          Wait for Isovalent Networking for Kubernetes to fully deploy in the cilium namespace before proceeding. Run the following command and ensure all the Cilium pods in cilium namespace are in running state:

Procedure 4.    Enable OpenShift Operator Management

Step 1.          Restart the OpenShift API server and Machine Config Operator pods. Wait for few minutes and confirm that all API server pods are up and running:

Step 2.          Scale the Cluster Network Operator to start managing the network with Cilium as the default network plugin:

Step 3.          Configure the Cluster Version Operator to manage the Network Operator once again:

Step 4.          Reboot the nodes by removing the pauses on the Machine Config Operator:

This step will immediately begin rebooting nodes in a safe fashion, by cordoning and draining workloads before a reboot. The cluster will be in a degraded state until all nodes are back online. The impact can be reduced by using multiple machine pools for workloads and removing the pause in sequence.

Step 5.          During node reboot, it is possible that some pods cannot be evicted. Run the following command to check the following logs for further information:

Procedure 5.    Verify the Migration and Test Network connectivity with Cilium

Step 1.          Once all nodes have been rebooted, check that all nodes are reporting as "Ready" and that all pods are running as expected:

Step 2.          Make sure all Cluster Operators are working as expected:

Step 3.          If you set devices in ciliumconfig.yaml file, you can now delete them. Cilium should be able to auto-detect non-openvswitch network devices:

Step 4.          Verify that Cilium restarted and is healthy:

Step 5.          When everything is healthy, remove the openshift-ovn-kubernetes namespace:

Procedure 6.    Test network connectivity with Cilium

An OpenShift Security Constraint (SCC) will be created to allow the required connectivity tests in Cilium-test namespace.

Step 1.          Run the following scripts to create OpenShift SSC and to deploy the Cilium network connectivity testing pods:

This will configure a series of deployments that will use various connectivity paths to connect to each other. Connectivity paths include with and without service load-balancing and various network policy combinations. The pod name indicates the connectivity variant, and the readiness and liveness gate indicates success or failure of the test:

 

Deploy Hubble UI and CLI

To access the Hubble-Relay services, Hubble client tools like Hubble CLI and UI needs to be installed. This section covers the deploying Hubble UI and CLI on top of Isovalent in OpenShift Cluster.

Procedure 1.    Install Hubble UI

Step 1.          Verify Isovalent helm repo is added. If not, add it by running the following commands. If the repo is already added, upgrade it:

Step 2.          Search for the latest version of hubble UI image available. For relay-address, set service name of hubble-relay as shown below. Wait for few minutes and check the hubble UI pod.

Step 3.          Set the hubble-ui service type by setting .spec.type from ClusterIP to NodePort to as shown below. Note down the nodeport number and access the hubble-ui service by using this URL https://nodeip:nodeport.

Procedure 2.    Install Hubble CLI

The hubble-cli can be leveraged to observe network flows from Cilium agents. Network flows are digested in userspace from the eBPF maps within each cilium agents' hubble server component. You can observe the flows from your local machine workstation for troubleshooting or monitoring.

Step 1.          Download the hubble cli here: https://github.com/isovalent/hubble-releases/releases/latest/download/hubble-linux-amd64.tar.gz

Step 2.          Create a directory hubblecli and move it to the binary file. Extract the binary from the archive using tar.

Step 3.          Move the hubble cli binary (hubble) to a directory listed in your $PATH environment variable.

Step 4.          For the hubble cli to access the hubble-relay service, use the oc port-forward command to forward the hubble-relay service to a local port. Use the hubble status command to view the overall health of hubble service:

Procedure 3.    Observe and troubleshoot with Hubble CLI and UI

Some useful commands to observe the network traffic with in the OpenShift cluster are provided below:

To showcase observability capabilities, a demo application Cilium Stat Wars is installed using this GitHub link: https://github.com/cilium/star-wars-demo:

Step 1.          Login to Hubble UI, click Policies and select namespace where the star-wars application is installed. Generate one http request from any xwing to deathstar service and observe the level of detail that Hubble captures for just one simple request:

The usual flow of tasks creating a policy from a service map include:

●     Observe Traffic: An administrator navigates to the Hubble Service Map in the Isovalent Enterprise UI. This view provides a real-time visualization of services and the traffic flowing between them.

●     Select a Flow: The user identifies and clicks on a specific traffic flow (the line connecting two services) that they wish to authorize with a network policy.

●     Generate the Policy: Upon selecting a flow, a context-aware side panel appears with details about the traffic. The user clicks the "Create Network Policy" button within this panel.

●     Review and Apply: Hubble automatically generates the CiliumNetworkPolicy YAML required to allow that specific connection. The administrator can then review this auto-generated policy, make any necessary adjustments, and apply it to the cluster directly from the UI.

Install and Configure Portworx Enterprise on OpenShift with Pure Storage FlashArray

This chapter contains the following:

●     Prerequisites

●     Configure Physical Environment

●     Deploy Portworx and Configure Storage Classes

●     Volume Snapshots and Clones

●     Portworx Enterprise Console Plugin for OpenShift

Portworx by Pure Storage is fully integrated with Red Hat OpenShift. Hence you can install and manage Portworx Enterprise from OpenShift web console itself. Portworx Enterprise can be installed with Pure Storage FlashArray as a cloud storage provider. This allows you to store your data on-premises with Pure Storage FlashArray while benefiting from Portworx Enterprise cloud drive features, such as:

●     Automatically provisioning block volumes

●     Expanding a cluster by adding new drives or expanding existing ones

●     Support for PX-Backup and Autopilot

Portworx Enterprise creates and manages the underlying storage pool volumes on the registered arrays.

Prerequisites

These prerequisites must be met before installing the Portworx Enterprise on OpenShift with Pure Storage FlashArray:

●     SecureBoot mode option must be disabled.

●     The Pure Storage FlashArray should be time-synced with the same time service as the Kubernetes cluster.

●     Pure Storage FlashArray must be running a minimum Purity//FA version of at least 4.8. Refer to the Supported models and versions topic for more information.

●     Both multipath and iSCSI, if being used, should have their services enabled in systemd so that they start after reboots. These services are already enabled in systemd within the Red Hat CoreOS Linux.

Configure Physical Environment

Before you install Portworx Enterprise, ensure that your physical network is configured appropriately and that you meet the prerequisites. You must provide Portworx Enterprise with your Pure Storage FlashArray configuration details during installation:

●     Each Pure Storage FlashArray management IP address can be accessed by each node.

●     Your cluster contains an up-and-running Pure Storage FlashArray with an existing data plane connectivity layout (iSCSI, NVMe-TCP).

●     If you're using iSCSI or NVMe-TCP, ensure the storage node initiators are on the same VLAN as the Pure Storage FlashArray iSCSI or NVMe-TCP target ports.

●     You have an API token for a user on your Pure Storage FlashArray with at least storage_admin permissions.

●     Ensure IP addresses have been assigned to the storage interfaces on each node and configured with MTU 9000. Verify all the workers reach target with large packet sizes without fragmenting the packets.

Procedure 1.    Prepare for the Portworx Enterprise Deployment

Step 1.          Secure the boot option already disabled at Intersight Boot Policy level. To reconfirm at the OS level, SSH into any of the worker node from the installer VM and ensure that SecureBoot mode is disabled at the OS level.

Step 2.          Apply the following MachineConfig to the cluster configures each worker node with the following:

●     Enable and start the multipathd.service with the specified multipath.conf configuration file.

●     Enable and start the iscsid.service service.

●     Applies the Queue Settings with Udev rules.

●     Copy the /etc/nvme/discovery.conf file

The settings of multipath and Udev rules are defined as shown below:

The following is the MachineConfig file that takes the base64 encode results of the previous two files and copies them to the corresponding directory on each worker node. It also enables and starts iscsid and multipathd services:

For NVMe-TCP protocol implementation, create another MachineConfig file to ensure device mapper multipath is used instead of native nvme multipathing for managing nvme volumes over nvme-tcp. Output of command "cat /sys/module/nvme_core/parameters/multipath” should return ‘N’. This indicates native nvme multipath is supported but disabled:

Step 3.          Apply the multipath machine configuration file:

Step 4.          This machine config is applied to each worker node one by one. To view the status of this process, from the cluster console go to Administration > Cluster Settings.

Step 5.          After the MachineConfig is applied on all the worker nodes, ssh into one of the worker nodes and verify if the two files multipath.conf and 99-pure-stroage.rules are created and iscsid and multipathd services are running.

Step 6.          The hostnqn (NVMe Host NQN) is expected to be a unique identifier, often generated using the UUID of the host system. However, in some OpenShift deployments, particularly during the installer boot process, duplicate NQNs are observed across different nodes, meaning multiple nodes end up with the same hostnqn. For this validation, we used the “nvme gen-hostnqn" command that generates a unique hostnqn id using host’s UUID. Host UUID is assigned from the Intersight UUID pool. For this, you need to connect to each worker node using the “oc debug node/<nodename> command and generate the hostnqn id as shown below:

Step 7.          Create Kubernetes secret object constituting the Pure Storage FlashArray API endpoints and API Tokens that Portworx Enterprise needs to communicate with and manage the Pure Storage FlashArray storage device.

Step 8.          Log into Pure Storage FlashArray and go to Settings > Users and Polices. Create a dedicated user (for instance, ocp-user) with storage Admin role for Portworx Enterprise authentication.

Step 9.          Click the ellipses of the previously created user and select Create API Token. In the Create API Token wizard, set the number of weeks (for instance 24) for the API key to expire and click Create. A new API Token for the ocp-user is created and displays. Copy the API key and preserve it since it will be used later to create Kubernetes Secret.

Step 10.       Create Kubernetes secret with the API key (located above) using the following manifest:

Deploy Portworx and Configure Storage Classes

This section covers the steps for deploying Portworx Storage Cluster and creating Storage Classes for dynamic storage provisioning of ReadWriteMany (RWX) volumes.

Procedure 1.    Deploy Portworx Enterprise

Step 1.          Log into the OpenShift cluster console using the kubeadmin account and go to Operators > OperatorHub.

Step 2.          In the right pane, enter Portworx Enterprise to filter the available operators in the Operator Hub. Select Portworx Enterprise and click Install.

Step 3.          In the Operator Installation window, from the Installed Namespace drop-down list, select Create Project and create a new project (for instance, px) and select the newly created project to install the Portworx Operator.

Step 4.          Install the Portworx plugin for OpenShift by clicking Enable under the Console Plugin. Click Install.

Step 5.          When the Portworx operator is successfully installed, the StorageCluster needs to be created. To create the StorageCluster Specifications (manifest file) log into https://central.portworx.com/ and use your credentials and click Get Started.

Step 6.          Select Portworx Enterprise and click Continue.

Step 7.          From the Generate Spec page, select the latest Portworx version (version 3.4 was the latest when this solution was validated). Select Pure Storage FlashArray as the platform. Select OpenShift 4+ from the Distribution drop-down list, provide px for the Namespace field. Click Customize.

Step 8.          Get the Kubernetes version by running kubectl version | grep -i 'Server Version'. Click Next.

Step 9.          From the Storage tab, select iSCSI for Storage Area Network. Provide the size of the Cloud drive (1500GB) and click plus (+) to add additional disks. Click Next.

Step 10.       From the Network tab, set eno12 for both Data and Management Network Interfaces. Click Next.

Step 11.       From the Customize tab, click Auto for Data Network Interface and Auto for Management Network Interfaces. Click Next.

Step 12.       Click Next. Under Environment Variables, add the variable and value as shown below:

Step 13.       Click Advanced Settings, enter the name of the portworx cluster (for instance, ocp-pxclus). Click Finish. Click Download.yaml to download the StorageCluster specification file.

Step 14.       From the OpenShift console, go to Operators > Installed Operators > Portworx Enterprise. Click the StorageCluster tab and click Create Storage Cluster to create StorageCluster. This opens the YAML view of Storage Cluster.

Step 15.       Copy the contents of the spec file previously downloaded and paste it in the yaml body. Verify that both the iSCSI/NVMe-TCP subnet networks are listed under env: as shown below. Click Create to create the StorageCluster. The following screenshot shows environmental variables used for Portworx deployment with nvme-tcp. For iSCSI deployment, set PURE_FLASH_SAN_TYPE will be set to “ISCSI.”

Step 16.       Wait until all the Portworx related pods come online.

Step 17.       Verify the cluster status by running the command on any worker node: sudo /opt/pwx/bin/pxctl status.

Step 18.       Run the sudo multipath -ll command on one of the worker nodes to verify all four paths from the worker node to storage target are being used. There are four active running paths for each volume:

Step 19.       Verify how FlashArray hosts are created and configured for each of the portworx storage node in the FlashArray. The following screenshot shows hosts created for nvme-tcp deployment:

Procedure 2.    Dynamic Volume Provisioning and Data Protection

Portworx by Pure Storage offers a set of pre-configured storage classes out of the box for various use cases. These storage classes support CSI features and dynamic provisioning. You are advised to choose from the available out-of-the-box storage classes for traditional use cases.

The list of pre-configured storage using the pxd.portworx.com provisioner implementing kubernetes CSI specifications for Portworx is shown below:

Step 1.          Execute oc describe sc <sc-name> commands to view various parameters used by each Storage Class. For more details on the storage class parameters go to: https://docs.portworx.com/portworx-enterprise/platform/provision-storage/create-pvcs/dynamic-provisioning.

Step 2.          Run the following command to set the Storage Class as the default Storage Class:

To enable special features such as KubeVirt VM live migration and use cases that require shared filesystem volumes across multiple pods, Portworx supports the ReadWriteMany(RWX) Block and File System volumes. The following are some of the available options:

●     Shared File System Volumes (sharedV4 volumes): These volumes allow multiple pods ( across the nodes) to read and write the same volume simultaneously. The file system level abstraction, access, and permission will be managed care by Portworx’s Sharedv4. These sharedv4 volumes are best suited for most RWX workloads in kubernetes. Some of the typical use cases that use sharedV4 volumes are web servers accessing shared file system volumes, Logs aggregation and so on. These shared file system volumes can be used for provisioning OS and Data disks for the OpenShift virtual machines. These shared disks support RWX option for enabling VMs live migration within the OpenShift cluster. The Storage Class provided below provides shared file system volumes for KubeVirt VMs.

●     Shared Block Devices (RWX Block): The shared block devices provide shared access (RWX) at the block level hence recommended to use them in specific scenarios like KubeVirt VM live migration. As the volumes are accessed at the block level, these shared block volumes provide better performance compared to shared file system volumes. Ensure to use these shared block volumes only for the cluster-aware applications like clustered file systems or databases. The Storage Class provided below provides shared block volumes for KubeVirt VMs.

●     FlashArray Direct Access Shared Block Device: Portworx enables the seamless integration of KubeVirt virtual machines (VMs) within Kubernetes clusters, leveraging the high performance of ReadWriteMany (RWX) volumes backed by FlashArray Direct Access (FADA) shared raw block RWX volumes. This approach supports raw block devices, which provide direct block storage access instead of a mounted filesystem. This is particularly beneficial for applications that demand low-latency and high-performance storage. These volumes eliminate filesystem overhead, provide direct access to the underlying storage for improved performance and enable efficient live migration of VMs in OpenShift clusters.

Volume Snapshots and Clones

This section provides the procedures to create SnapshotClass for enabling snapshots of PVCs and also explains clones.

Snapshots and Clones: Portworx Enterprise offers data protection of volumes using volume snapshots and restore them for point in time recovery of the data. Any Storage Class that implements portworx csi driver pxd.portworx.com supports volume Snapshots.

Procedure 1.    Create volume snapshots and clones

Step 1.          Run the following scripts to create the VolumeSnapshotClass and a PVC. Then create Snapshot of the PVC and restore the snapshot as a new PVC.

Step 2.          Use the following sample manifest to create a sample PVC:

The following screenshot shows px-snaptest-pvc-snap1 is the snapshot of the PVC px-snaptest-pvc:

Step 3.          You can now restore this snapshot as a new PVC, and it can be mounted to other pods.

Step 4.          Click the ellipses of the snapshot and select Restore as new PVC. In the Restore as new PVC window, click Restore.

Step 5.          You can view the original and restored PVC under PVC list.

Step 6.          To clone PVC (px-snaptest-pvc), click three of the PVC and select Clone PVC. Click Clone.

Portworx Enterprise Console Plugin for OpenShift

Portworx by Pure Storage has built an OpenShift Dynamic console plugin that enables single-pane-of-glass management of storage resources running on Red Hat OpenShift clusters. This allows platform administrators to use the OpenShift web console to manage not just their applications and their OpenShift cluster, but also their Portworx Enterprise installation and their stateful applications running on OpenShift.

This plugin can be installed with a single click during the Installation of Portworx Operator. Once the plugin is enabled, the Portworx Operator automatically installs the plugin pods in the same OpenShift project as the Portworx storage cluster. When the pods are up and running, administrators will see a message in the OpenShift web console to refresh their browser window for the Portworx tabs to show up in the UI.

With this plugin, Portworx has built three different UI pages, including a Portworx Cluster Dashboard that displays in the left navigation menu, a Portworx tab under Storage > Storage Class section, and another Portworx tab under Storage > Persistent Volume Claims.

Portworx Cluster Dashboard

Platform administrators can use the Portworx Cluster Dashboard to monitor the status of their Portworx Storage Cluster and their persistent volumes and storage nodes. Here are a few operations that are now streamlined by the OpenShift Dynamic plugin from Portworx.

To view detailed inventory about the Portworx Cluster, click the Drives and Pools tabs.

Portworx PVC Dashboard

This dashboard contains some of the important attributes of the PVC Replication Factor, node details of replicas, attached node, and so on. Previously, you would have had to use multiple pxctl inspect volume CLI commands to obtain these details. Now all this information can be found in Console Plugin.

Portworx StorageClass Dashboard

From the Portworx storage cluster tab, administrators can get details about the custom parameters set for each storage class, the number of persistent volumes dynamically provisioned using the storage class, and a table that lists all the persistent volumes deployed using that storage class. The OpenShift dynamic plugin eliminates the need for administrators to use multiple “kubectl get” and “kubectl describe” commands to find all these details—instead, they can just use a simple UI to monitor their storage classes.

Add a Worker Node to an OpenShift Cluster

This chapter contains the following:

●     OpenShift Cluster Expansion

This chapter provides the procedures to scale-up the worker nodes of OpenShift cluster by adding a new worker node to the existing cluster. For this exercise, a UCS X215 M8 blade will be added as a new worker node as amdn8.fs-ocp.flashstack.local to the already existing OpenShift cluster.

OpenShift Cluster Expansion

This section details how to create and configure a OpenShift Cluster expansion.

Procedure 1.    Create and configure the OpenShift Cluster

Step 1.          Launch a web browser and go to https://console.redhat.com/openshift/cluster-list. Log into your Red Hat account.

Step 2.          Click your cluster name and go to Add Hosts.

Step 3.          Under Host Discovery, click Add hosts.

Step 4.          In the Add hosts wizard, for the CPU architecture select x86_64 and for the Host’s network configuration select DHCP Only. Click Next.

Step 5.          For the Provision type select Full image file from the drop-down list, for SSH public key browse or copy/paste the contents of id-ed25519.pub file. Click Generate Discovery ISO and when the file is generated and click Download Discovery ISO file.

Step 6.          Click Add hosts from Cisco Intersight and select the node from the intersight. Click Save. Click Execute. This will mount the Discovery ISO to the host and boot the server into it.

When the server has booted RHEL CoreOS (live) from the newly generated Discovery ISO, it will appear in the assisted installer under Add hosts:

Step 7.          When the node status shows Ready, click Install ready hosts. After few minutes, the required components will be installed on the node and displays the status as Installed.

Step 8.          Log into the cluster with kubeadmin user and go to Compute > Nodes >  and select the newly added worker node and approve the Cluster join request of the worker node and request for server certificate signing.

Step 9.          Wait for a few seconds and the node will be ready, and the pods are scheduled on the newly added worker node.

Step 10.       Create secret and BareMetalHost objects in openshift-machine-api namespace by executing the following manifest (add-host-bm-x215-3-amdn8.yaml). See Table18 for MAC addresses and OOB IP of the newly added node.

Step 11.       Apply the bare metal configure for the newly added node.

A new entry will be created for the newly added worker node under Compute > Bare Metal Hosts.

Step 12.       Increase the worker machineset count by one since you are adding one machine to the existing cluster. Go to Compute > MachineSets. Click the ellipses of worker-0 machineset and select Edit Machine Count and increase the count by 1. Click Save.

A new worker node will be provisioned to match to the worker new machine count. It is under the provisioning state until the node is logically mapped to the Bare Metal Host.

Procedure 2.    Link the Machine and Bare Metal Host, Node and Bare Metal Host

Step 1.          To logically link Bare Metal Host to Machine, obtain the name of the newly created machine from its manifest file or by running oc get machine -n openshift-machine-api:

Step 2.          Update the machine name in the Bare Metal Host’s manifest file under spec. consumerRef as shown below. Save the Yaml and reload:

Step 3.          The bare metal host providerID needs to be generated and updated in the newly added worker (amdn8.fs-ocp.flashstack.local). The providerID is a combination of the name and UUID of the Bare Metal Host and is shown below. These details can be gathered by running providerID: baremetalhost:///openshift-machine-api/<Bare Metal Host Name>/<Bare Metal Host UID>.

By using the provided information, the providerID for of the newly added Bare Metal Host is built as providerID: ‘baremetalhost:///openshift-machine-api/amdn8.fs-ocp.flashstack.local/1091e690-3735-49e7-bdce-a2a29f39559d’.

Step 4.          Copy the providerID of the Bare Metal Host into the third node yaml manifest file under spec. as shown below:

When the providerID of amdn8 is updated, the node details are automatically populated for the newly added Bare Metal Host as shown below:

Step 5.          After the node is added to the cluster, verify if daemonset pod of deployments Isovalent, Portworx and so on, are created automatically on the amdn8.fs-ocp.flashstack.local node as shown below:

OpenShift Virtualization

This chapter contains the following:

●     OpenShift Virtualization Operator

●     Post Installation Configuration

●     Create OpenShift Virtual Machines

●     Migrate Virtual Machines from VMware vSphere Cluster to OpenShift Virtualization

Red Hat OpenShift Virtualization, an included feature of Red Hat OpenShift, provides a modern platform for organizations to run and deploy their new and existing virtual machine (VM) workloads. The solution allows for easy migration and management of traditional virtual machines onto a trusted, consistent, and comprehensive hybrid cloud application platform.

OpenShift Virtualization is an operator included with any OpenShift subscription. It enables infrastructure architects to create and add virtualized applications to their projects from OperatorHub in the same way they would for a containerized application.

Existing virtual machines can be migrated from other platforms onto the OpenShift application platform through the use of free, intuitive migration tools. The resulting VMs will run alongside containers on the same Red Hat OpenShift nodes.

The following sections and procedures provide detailed steps to create custom network policies for creating management and iSCSI networks for virtual machines to use, steps to deploy  Red Hat virtual machines using pre-defined templates, steps to create custom Windows Server template and create windows virtual machine from this custom template.

OpenShift Virtualization Operator

This section provides the procedures necessary for the OpenShift Virtualization Operator.

Procedure 1.    Deploy OpenShift Virtualization Operator

Step 1.          If the Red Hat OpenShift Virtualization is not deployed, go to Operators > OperatorHub. Type virtualization in the All Items checkbox. From the available list, select the OpenShift Virtualization tile with Red Hat source label. Click Install to install OpenShift Virtualization with default settings.

Step 2.          When the operator is installed successfully, go to Operators > Installed Operators and type virtualization under the Name checkbox and verify that operator is installed successfully.

Post Installation Configuration

The procedures in this section are typically performed after OpenShift Virtualization is installed. You can configure the components that are relevant for your environment:

●     Node placement rules for OpenShift Virtualization Operators, workloads, and controllers: The default scheduling for virtual machines (VMs) on bare metal nodes is appropriate. Optionally, you can specify the nodes where you want to deploy OpenShift Virtualization Operators, workloads, and controllers by configuring node placement rules. For detailed options on VM scheduling and placement options, see: https://docs.redhat.com/en/documentation/openshift_container_platform/4.18/html/virtualization/postinstallation-configuration#virt-node-placement-virt-components.

●     Storage Configuration: Storage profile must be configured for OpenShift virtualization. A storage profile provides recommended settings based on the associated storage class. When the Portworx Enterprise is deployed on the OpenShift Cluster, it deploys several storage classes with different settings for different use-cases. OpenShift Virtualization automatically creates a storage profile with the recommended storage settings based on the associated storage class. There is no need for additional configuration.

●     A Default Storage Class must be configured for the OpenShift Cluster. Otherwise, the cluster cannot receive automated boot source updates. To configure an existing storage class (created by Portworx)  as Default Storage Class run the following command:

kubectl patch storageclass <StorageClassName> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

●     Network configuration: By default, OpenShift Virtualization is installed with a single, internal pod network. After you install OpenShift Virtualization, you can install networking Operators and configure additional networks. The following sections provides more details on network configurations validated in this solution.

In this solution, the NMState operator and Multus CRDs are used for connecting the virtual machines to the secondary networks in the Red Hat OpenShift cluster.

NMState NodeNetworkConfigurationPolicies are already configured in Procedure 4. Install NMState Operator. The following NodeAttachmentDefinitions (NAD) are created for attaching the VMs to secondary networks.

Table 19.     Node Attachment Definitions

The following yaml manifests are used for creating NAD definitions:

The following scripts shows the NADs created for secondary VM networks:

Create OpenShift Virtual Machines

This section explains the creation of linux and windows virtual machines. Before proceeding with virtual machines, ensure the following prerequisites are completed.

Procedure 1.    Create Linux Virtual Machine

Step 1.          To create a VM, go to Virtualization > Virtual Machines > Create > From template. Select one of the templates which has the Source available label on it.

Step 2.          Provide a name to the VM and change the Disk size to the required size. Click Customize Virtual Machine.

Step 3.          From the Network Interface tab, remove the preconfigured Pod network interface and add new interface on vmnw-vlan1062 network as shown below:

Step 4.          From the Scripts tab, click the pen icon near Public SSH. Click Add New and upload the public key of your installer VM. Provide a name for the public key and check the box to use this key for all the VMs you create in future. Make a note of the default user name (cloud-user) and the default password. You can click the pen icon and change default user and password.

Step 5.          Optionally, from the Overview tab, change the CPUs and memory resources to be allocated to the VM. Click Create Virtual Machine.

In a few seconds a new virtual machine will be provisioned. The interface of the VM will get a  DHCP IP from vmnw-vlan1062 network and the VM can be accessed directly from rhel-installer VM (aa06-rhel9) using its public ssh key without using password as shown below:

Procedure 2.    Create Windows Template and provisioning Windows VMs from Template

Follow this procedure to create a temporary windows Server 2025 virtual machine using Windows Server ISO, then install and configure the VM with all the required software components. Use this VM to create sysprep image. Then use this sysprep image as gold image for all the future windows Server 2025 VMs. Before proceeding with VM creation, Download the Windows Server 2025 ISO from Microsoft website

Step 1.          To create a new VM, go to Virtualization > Virtual Machines > Create > Using Template. Select Windows server 2025 (windows2k25-server-medium) template.

Step 2.          From the Create VM window, check the box for Boot from CD and select Upload (upload a new file to PVC) option for CD source. Click Browse. The Upload data to PVC window opens. Provide a name to the PVC (win2k25-iso) and set the storage class to px-rwx-kubevirt and click Upload.

Step 3.          Once the PVC is created, from the Create VM window, provide a name to the windows VM.  (win2k25-tempvm) and to create a new VM, go to Virtualization > Virtual Machines > Create > Using Template. Select Windows server 2025 (windows2k25-server-medium) template.

Step 4.          From the Create VM window, check the box for Boot from CD and select Upload (upload a new file to PVC) option for CD source. Set Blank for Disk Source and set 80G for the Disk size. Select Mount Windows Drivers disk. Click Customize VirtualMachine. This starts the uploading of the ISO to the PVC.

Step 5.          Click the Disks tab. By editing the root disk, set the disk driver from SATA to Virtio and storage class to px-rwx-kubevirt. Click Create VM. When the VM is connected, connect to the VM console and press any key to start Windows Server 2025 installation. From the Windows Server setup window, select Install Windows Server and check the box and then click Next.

Step 6.          From the Select location to install Windows Server window, click Load Driver link and select the folder E:\virtio-win-XX\amd64\2k25\.

Step 7.          Select Red Hat VirtIO SCSI controller driver and click Install. Once driver is installed successfully, the 80G volumes display for the OS installation. Select the volume, click Next and then click Next again. Once OS installed, provide Administrator password and then login to the Windows server 2025.

Step 8.          When the Windows is installed successfully, go to E:\ disk and install the virtio driver by double-clicking the virtio-win-gt-x64.msi. Complete the drivers installation with default options.

Step 9.          When the drivers are installed, the network interfaces display and are assigned with a DHCP IP address. Turn off the firewalls temporarily to verify the VM can reach the outside services like AD/DNS. Enable Remote Desktop.

Step 10.       Convert this temporary VM into a sysprep image by executing the sysprep command as shown below. When sysprep is completed, the VM tries to restart. Before it restarts, stop the VM from OpenShift console.

Step 11.       Delete this temporary VM. While deleting the VM, ensure you DO NOT delete the root disk of this VM by unchecking the checkbox as shown below:

Step 12.       Create a new template by cloning the existing windows2k25-server-large template. Click the ellipses and select Clone. Provide a name to the new template (windows2k25-server-large-Cisco) and click Clone.

Step 13.       Click the newly created template and then go to the Disks tab. Click Add Disk > Clone Volume to add a new disk. Select the template. Edit this newly created windows2k25-template by clicking Edit. Select the PVC that has the Windows Server 2025 sysprep image previously created. Click Save.

Step 14.       Delete the existing root disk. The newly added disk automatically sets as new root disk.

Step 15.       Go to Network Interfaces, add two additional network interfaces from iscsi-vm-a-nad and iscsi-vm-b-nad.

Step 16.       Create a fresh Windows Server 2025 virtual machine using the template created in the previous steps. Go to Virtualization > VirtualMachines > Create > From template. Click User templates. Select the new windows2k25-server-large-Cisco.

Step 17.       Provide a name for the VM and click Quick Create VirtualMachine. Since the template is pre-configured with everything according to our requirements, nothing has to be changed to create the virtual machine. The VMs displays in seconds.

Step 18.       When the VM is fully provisioned, verify all three interfaces get corresponding DHCP IP addresses and are able to reach the FlashArray//XL170 target IP addresses with larger packet sizes as shown below:

Step 19.       Since the storage target IP addresses are reachable from the VM, the VM can be connected to the FlashArray volumes directly by configuring the in-guest iscsi initiator.

Step 20.       To migrate the VM from one to other, click ellipses and select Migrate. The VMs will be migrated to a different worker node.

Migrate Virtual Machines from VMware vSphere Cluster to OpenShift Virtualization

The Migration Toolkit for Virtualization (MTV) is an operator-based functionality that enables us to migrate virtual machines at scale to Red Hat OpenShift Virtualization. MTV supports migration of virtual machines from VMware vSphere, Red Hat Virtualization, OpenStack, OVA and OpenShift Virtualization source providers to OpenShift Virtualization. VMware vSphere VMs were migrated to OpenShift. RHEL and Windows VMs were successfully imported and all migrated VMs were brought into OpenShift with both virtio network interfaces and virtio disk controllers.

The following are some of the vSphere prerequisites to be met before planning for migration of virtual machines from vSphere environments:

●     VMware vSphere version must be compatible with OpenShift virtualization. At a minimum, vSphere 6.5 or later is compatible with OpenShift Virtualization 4.14 or later. For more details on compatibility, go to: https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.9/html-single/installing_and_using_the_migration_toolkit_for_virtualization/index#compatibility-guidelines_mtv

●     Not all guest operating systems migration is supported. Only the specific Guest Operating Systems can be migrated to OpenShift Virtualization. See the list of guest operating systems supported by OpenShift Virtualization here: https://access.redhat.com/articles/1351473?extIdCarryOver=true&sc_cid=RHCTG0250000454097

●     The guest OS must be supported by virt-v2v utility to convert them into OpenShift virtualization compatible images as listed here: https://access.redhat.com/articles/1351473

●     You must have a user with at least the minimal set of VMware privileges. Required privileges listed here: https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.9/html-single/installing_and_using_the_migration_toolkit_for_virtualization/index#vmware-prerequisites_mtv

●     The Secure boot option must be disabled on the VMs.

●     For a warm migration, changed block tracking (CBT) must be enabled on the VMs and on the VM disks. Here are the steps for enabling CBT on the VMs running on vSphere cluster: https://knowledge.broadcom.com/external/article/320557/changed-block-tracking-cbt-on-virtual-ma.html

●     The MTV can use the VMware Virtual Disk Development Kit (VDDK) SDK to accelerate transferring virtual disks from VMware vSphere. Optionally, VDDK can be used for the faster migration. For this validation, VDDK is installed as detailed here: https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.9/html-single/installing_and_using_the_migration_toolkit_for_virtualization/index#creating-vddk-image_mtv

The following screenshot shows the VDDK image version 8.0.0 is available in the OpenShift local image registry. The full local image path to pull the VDDK image is Image-registry.openshift-image-registry:5000/openshift/vddk:8.0.0.

For more information on the MTV prerequisites, see: https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.9/html-single/installing_and_using_the_migration_toolkit_for_virtualization/index#rhv-prerequisites_mtv

https://docs.redhat.com/en/documentation/migration_toolkit_for_virtualization/2.9/html-single/installing_and_using_the_migration_toolkit_for_virtualization/index#vmware-prerequisites_mtv

The virtual machines with guest-initiated storage connections, such as Internet Small Computer Systems Interface (iSCSI) connections or Network File System (NFS) mounts, are not handled by MTV and could require additional planning before or reconfiguration after the migration.

This section describes the migration of the following two VMs running on vSphere 8.0 cluster:

The following screenshot shows the vSphere cluster with above two VMs (rhe8-vm1 and win2k19-vm2) to be migrated to OpenShift cluster:

Procedure 3.    Install the Migration Toolkit for Virtualization

Step 1.          Install the MTV operator, go to Operators > OperatorHub, search for mtv, and select the Migration Toolkit for Virtualization operator and click Install. When the operator is installed successfully, click Create ForkliftController to install the ForkliftController. With the default option, complete the FortliftController installation. Refresh the browser to view the Migration tab on the console.

The actions to perform the migration of VMs from vSphere environment to OpenShift Virtualization are as follows:

●     Identify the VMs on the vSphere environment and ensure the VMs are ready for migration by verifying that all the prerequisites are met

●     Create Provider

●     Create Migration Plan

●     Execute the migration plan

Procedure 4.    Migrate VMs from vSphere to OpenShift Virtualization

Step 1.          To create vSphere Provider, go to Migration > Providers for Virtualization and click Create Provider. Select the required project name and select vm VMware. Provide a name to the provider, URL to the provider sdk in the format https://host-example.com/sdk, URL for VDDK, and credential details as shown below. Click Create Provider to create the vSphere cluster as provider.

The following vSphere provider is created:

Step 2.          Click the vSphere provider, then click ESXI Hosts, select the hosts that are currently owning the VMs and click Select Migration Network. Select the Management Network 10.106.1.X/24 and provide root credentials. Click Save.

Step 3.          Click the Virtual machines tab and review the list of Warning concerns (yellow colored) reported for each source VM. You need to address each and concern and clear them.

Procedure 5.    Create Migration Plan

Step 1.          Go to Migration > Plan for Virtualization and click Create Plan. Provide a name to the migration plan and select the project name. Select the vSphere provider previously created. Select the OpenShift cluster (host) as Target Provider and Target project as default. Click Next.

Step 2.          Select the VMs rhel9-vm1 and win2k222-vm1 that need to be migrated to OpenShift. Click Next.

Step 3.          From the Network Map tab, click Use New Network Map and select the appropriate source and target networks as shown below. Click Next.

Step 4.          From the Storage Map tab, click Use New Storage Map and select the appropriate source and target networks as shown below. Click Next. Ensure to the storage class that support RWX option. Click Next.

Step 5.          Select the migration type. For this validation, warm migration is selected. Click Next.

Step 6.          From the Additional Steps tab. Optionally, select Perverse Static IPs and Migrate shared disks options.

Step 7.          Review the migration plan and click Create Plan.

All the options in the migration plan will be validated and the plan status will be set to Start when all the options are valid.

Step 8.          Click Start to start the migration of VMs.

Step 9.          To schedule cutover time, click the migration plan > virtual machines. Click Schedule cutover located on the top right. Once you set the time for cutover, the migration plan will resume at the cutover you set.

When the VMs migrated successfully to the OpenShift cluster the Migration status is complete:

When the VMs are migrated to the OpenShift, ensure the agents are running inside the virtual machines, all the disks are mounted to the right folders, IP addresses and MAC addresses are retained.

Data Protection with Portworx Backup

This chapter contains the following:

●     Prerequisites

●     Backup and Restore Virtual Machines

Prerequisites

The following prerequisites need to be met before proceeding with this PX-backup validation.

●     PX-Backup can be installed on a dedicated Kubernetes cluster or on the application clusters where application workloads are running. For this validation, PX-Backup is installed on a dedicated OpenShift cluster (fs-ocp1.flashstack.local) installed on a different set of hardware. Later, this FlashStack AMD M8 based cluster (fs-ocp.flashstack.local) is registered as an application cluster for protecting the virtual machines and applications running on the cluster.

●     Pure Storage FlashBlade//S200 should be reachable from both clusters. For this connectivity, network interface eno11 is used for carrying the backup/restore traffic with VLAN 3040 from clusters to the FlashBlade. Refer vNIC Template and vNICs section for more details on eno11 interface.

●     Ensure User Accounts, users, Access policies, and buckets are already configured on FlashBlade. These details will be used while adding backup locations in PX-backup.

●     Ensure that stork scheduler is installed on all the application clusters. For this validation, the stork component is automatically installed when Portworx Enterprise is installed on the clusters. For more details on stock requirement, refer: https://docs.portworx.com/portworx-backup-on-prem/install/install/install-stork

Procedure 1.    Install and Configure PX-Backup

Step 1.          Connect to the fs-ocp1.flashstack.local cluster and create a namespace or project px-bkp.

Step 2.          Login to the PX-Central with your credentials. Check the box for the EULA located under Backup Services and click Start Free Trial. Select 2.9.1 as the backup version. Select On-Premises for Environment and provide a Storage Class name defined in the OpenShift cluster for creating PVCs required by PX-Backup deployment. Click Next.

Step 3.          The required installation scripts are generated as shown below. Run the generated scripts on the cluster (fs-ocp1.flashstack.local):

Step 4.          Wait until all the pods display in the px-bkp namespace. Get the node port of px-backup-ui or px-central-ui services and access the services using http://nodeip:nodeport>.

Step 5.          Login with the default userid and password and reset the admin password.

Step 6.          When the password is reset, click Add Cluster to add both application cluster (fs-ocp.flashstack.local) and backup cluster (fs-ocp1.flashstack.local) by using the kubeconfig file. Use the following command to get the kubeconfig files of each cluster. The following screenshot shows adding application cluster (fs-ocp.flashstack.local) with the name fs-ocp-amd.

Step 7.          Repeat step 6 to add backup cluster (fs-ocp1.flashstack.local).

Step 8.          Click the application fs-ocp-amd cluster and verify if all the container workloads and virtual machines are populated.

Step 9.          Add the backup location. Click the cloud symbol. Add a cloud account by using the Access token, Secrets created in the chapter Pure Storage FlashBlade Configuration.

Step 10.       Add backup location as shown below. Note that 192.168.40.40 is the IP addresses on FlashBlade array.

Backup and Restore Virtual Machines

This section provides the procedures to create a backup schedule (policy), backup and restore of the virtual machines to achieve a point in time recovery.

Procedure 1.    Backup and Restore of Virtual Machines

Step 1.          Click Policies. Create a policy that periodically backs up every 30 minutes.

Step 2.          Click + to create policy. Provide a Name for the policy and select Periodic and set 30min and leave the other settings with defaults as show below:

Step 3.          Create a backup of a VM, go to Clusters > click fs-ocp-amd cluster > applications > VM. Select the VM (win2k25-vm1) and click Backup. Provide the required details as show below. Select a backup location and volume snapshot class. Enable On a schedule backups and select schedule policy previously created. Click on Create to start the VM backup.

The full backup of the entire virtual machine will start and complete as shown below:

Step 4.          After the completion of the first scheduled backup, wait for 30 minutes to start the second scheduled backup or snapshot. Log into the VM, copy or create some files in the VM. When the second backup completes, log into the VM and make changes.

Step 5.          Recover the deleted files, by selecting the latest snapshot copy and restore it. Before restoring, pause the next scheduled backup (third backup).

Step 6.          Click the Backups tab to view the list backups available for your VMs. The following screenshot shows two backups created in the previous steps with 30 minutes apart:

Step 7.          Select the latest backup of the VM and click Restore as shown above. Provide a name to the Restore operation. Select the Destination cluster.

Step 8.          Click Customer Restore to customize the restore operation. Select the appropriate Storage Class that supports RWX volumes in the destination cluster. Provide a name space in which the VM needs to be restored. Click Restore to start the restore of the VM.

Step 9.          After the successful completion of restore, a new VM is created and running in the dev-sql namespace, in the same application cluster as shown below. Verify if the deleted files are available. This verifies that data is being recovered from accidental deletion.

When the backup or restore operations are in progress, you can login to the FlashBlade and check for data traffic. The following screenshot shows nearly 1GB write bandwidth during the backup operation.

Splunk Observability

This chapter contains the following:

●     Prerequisites

●     Install and Configure OTEL Collector Agents

This section provides the procedures to install and configure OTEL agents for various components of the solution to gain deep observability capabilities with Splunk Observability Cloud.

Prerequisites

The following prerequisites need to be met before installing and configuring Splunk OpenTelemetry Collector (OTEL):

●     To install Splunk OpenTelemetry Collector in a Kubernetes cluster environment, either of the destinations (splunkPlatform or splunkObservabilitycloud) to be configured. For this validation, Splunk Observability Cloud is pre-configured and required access to the observability cloud is provided.

●     Once access to observability cloud is enabled, generate Realm and Access token with which the OTEL collectors authenticates with Observability cloud and sends the metrics data. To create the Access token and realm, see: https://help.splunk.com/en/splunk-observability-cloud/administer/authentication-and-security/authentication-tokens/org-access-tokens#admin-org-tokens and https://dev.splunk.com/observability/docs/realms_in_endpoints/

●     Administrator Access to OpenShift cluster for deploying OTEL collector agents and sending the logs to the Observability cloud.

●     Ensure Helm version 3 is installed on the Linux client machine.

Install and Configure OTEL Collector Agents

Procedure 1.    Install OpenShift OTEL Collector Kubernetes

Step 1.          From the Linux VM (jump host), log into your OpenShift cluster with CLI using kubeconfig or admin user.

Step 2.          Create Install otel collector agents for OpenShift by running the following commands from the linux client machine

Step 3.          Add helm repository of Splunk OTEL collector charts.

Step 4.          Install the Splunk otel collector using values.yaml file. Update the exporters, receivers and processors sections with appropriate values for your environment and run the below command to install the otel collector for your environment. Sample values.yaml file is provided below:

Step 5.          After deploying the splunk otel collector, wait for pods (collector agents for each node) to come up as shown below:

Step 6.          Install the Intersight otel collector using the following values.yaml. Set intersight.organization to any arbitrary name:

Step 7.          After installing the otel agents for Intersight, wait for the pod to come up.

Step 8.          Portworx metrics can be pushed to the Splunk observability cloud by enabling the user defined project monitoring by running the following:

Step 9.          Log into the Splunk Observability site with your credentials and start creating your own dashboards and dashboard groups. You can also leverage built-in dashboards by importing them and customize them as per your requirements. The following dashboards have been built using the metrics pushed by Otel agents running on the OpenShift cluster. For more information about dashboards, go to: https://help.splunk.com/en/splunk-observability-cloud/create-dashboards-and-charts/create-dashboards

About the Authors

Gopu Narasimha Reddy, Technical Marketing Engineer, Cisco Systems, Inc.

Gopu Narasimha Reddy is a Technical Marketing engineer with the UCS Solutions team at Cisco. He is currently focused on validating and developing Cisco UCS infrastructure solutions for enterprise workloads with different operating environments including Windows, VMware, Linux, and OpenShift. Gopu is also involved in publishing database benchmarks on Cisco UCS servers. His areas of interest include building and validating reference architectures, and development of sizing tools in addition to assisting customers in database deployments.

Vijay Bhaskar Kulari, Solution Architect, Pure Storage, Inc.

Vijay Kulari works at Pure Storage, part of the Portworx Technical Marketing team. He specializes in designing, developing, and optimizing solutions across Storage, Converged Infrastructure, Cloud, and Container technologies. His role includes establishing best practices, streamlining automation, and creating technical content. As an experienced Solution Architect, Vijay has a strong background in VMware products, storage solutions, converged and hyper-converged infrastructure, and container platforms

Acknowledgements

For their support and contribution to the design, validation, and creation of this Cisco Validated Design, the authors would like to thank:

●     Chris O’Brien, Senior Director, Technical Marketing, Cisco Systems, Inc.

●     John George, Technical Marketing Engineer, Cisco Systems, Inc.

●     Paniraja Koppa, Technical Marketing Engineer, Cisco Systems, Inc.

●     Andrew Riconosciuto, Customer Success, Isovalent@Cisco

●     Piotr Jablonski, Customer Success, Isovalent@Cisco

●     Marcos Hernandez, Distinguished Engineer, Isovalent@Cisco

●     Craig Waters, Solutions Director, Pure Storage, Inc.

●     Eric Shanks, Principal Technical Marketing Engineer, Pure Storage, Inc.

Appendix

This appendix contains the following:

●     Appendix A – References used in this guide

Appendix A – References used in this guide

Compute

Cisco Intersight: https://www.intersight.com

Cisco Intersight Managed Mode: https://www.cisco.com/c/en/us/td/docs/unified_computing/Intersight/b_Intersight_Managed_Mode_Configuration_Guide.html

Cisco Unified Computing System: http://www.cisco.com/en/US/products/ps10265/index.html

Cisco UCS M8 AMD Servers: https://www.cisco.com/c/m/en_us/solutions/computing/ucs-amd.html#~models

Cisco UCS 6536 Fabric Interconnect Data Sheet: https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs6536-fabric-interconnect-ds.html

Network

Cisco Nexus 9300-GX Series Switches: https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/nexus-9300-gx-series-switches-ds.html

Pure Storage

FlashStack: https://flashstack.com

Pure Storage FlashArray//X: https://www.purestorage.com/products/unified-block-file-storage/flasharray-x.html

Pure Storage FlashArray//XL: https://www.purestorage.com/products/unified-block-file-storage/flasharray-xl.html

Pure Storage FlashBlade//S: https://www.purestorage.com/products/unstructured-data-storage/flashblade-s.html

Red Hat OpenShift

Documentation: https://docs.openshift.com/

Red Hat OpenShift Container Platform: https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform

Red Hat OpenShift Virtualization: https://docs.redhat.com/en/documentation/openshift_container_platform/4.16/html/virtualization/index

Red Hat Hybrid Cloud Console: https://cloud.redhat.com/

Isovalent Networking for Kubernetes

Isovalent Enterprise: https://docs.isovalent.com/documentation.html

Migration from OpenShift OVN: https://docs.isovalent.com/operations-guide/migrating/ovn-kubernetes.html

Portworx Enterprise

Portworx Enterprise documentation: https://docs.portworx.com/

Portworx Backup

Portworx Backup documentation: https://docs.portworx.com/portworx-backup-on-prem/

Cisco Splunk Observability Cloud

OTEL Collectors: https://help.splunk.com/en/splunk-observability-cloud/manage-data/splunk-distribution-of-the-opentelemetry-collector/get-started-with-the-splunk-distribution-of-the-opentelemetry-collector

Built-in Dashboards: https://help.splunk.com/en/splunk-observability-cloud/create-dashboards-and-charts/create-dashboards

Interoperability Matrix

Cisco UCS Hardware Compatibility Matrix: https://ucshcltool.cloudapps.cisco.com/public/ 

Pure Storage FlashStack Compatibility Matrix. This interoperability list will require a support login from Pure: https://support.purestorage.com/bundle/m_product_information/page/FlashStack/Product_Information/topics/reference/r_flashstack_compatibility_matrix.html

Feedback

For comments and suggestions about this guide and related guides, join the discussion on Cisco Community here: https://cs.co/en-cvds.

CVD Program

"DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.

CCDE, CCENT, Cisco Eos, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, the Cisco logo, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unified Computing System (Cisco UCS), Cisco UCS B-Series Blade Servers, Cisco UCS C-Series Rack Servers, Cisco UCS S-Series Storage Servers, Cisco UCS X-Series, Cisco UCS Manager, Cisco UCS Management Software, Cisco Unified Fabric, Cisco Application Centric Infrastructure, Cisco Nexus 9000 Series, Cisco Nexus 7000 Series. Cisco Prime Data Center Network Manager, Cisco NX-OS Software, Cisco MDS Series, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study,  LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trade-marks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. (LDW_P2)

All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0809R)