Test Automation in 5G Core

 

Introduction

The promise of 5G is not only a faster network service but a revolutionary system. A system capable of hosting multiple ‘slices’ (i.e. discrete virtual instances) possibly serving use cases with conflicting requirements. A network that is genuinely software-based that could be seasonally built and torn down (e.g. for the Olympics or World Cup) on shared infrastructure (e.g. Cloud). Imagine a network that provides a tailored quality of experience to its users through an unprecedented level of demographic insights. An elastic, dynamic, and on-demand network capable of self-remediation and automatic capacity management (geographically or periodically).

Like prior network generations, 5G also enables much increased data rates through a more efficient use of a wider high-frequency spectrum and ultra-low latency though a network architecture which disaggregates control and data elements. Thus, enabling new use cases such as autonomous vehicles, Internet of Things, augmented reality, and the Industrial Internet.

However, to achieve the promised revolution, the 5G network and its ownership must undergo a radical transformation in its system engineering, covering its development, testing, deployment, as well as its operation. Instead of allowing commercial strategy to drive this transformation, the network must be delegated to be a real agent of change and not just a traditional place of finding cost savings.

It is evident that the adoption of virtualization and cloud technologies will play a crucial role in this transformation. Figure 1 below shows the evolution of the mobile network core from an appliance-based platform, through virtualization and now cloudification, thereby leading to increased innovation, lowered operational costs, higher and cheaper connectivity.

 

Figure 1: Evolution of the Mobile Core Platform

5G Use Cases & Network Architecture

It is true to say that how 5G is deployed and operated will determine its profitability rather than just being another bandwidth upgrade for subscribers. 5G technology promises to deliver innovative services beyond the traditional mobile broadband offering that earlier generations have provided. Below are the defined 5G use cases:

·         Enhanced Mobile Broadband (eMBB) - eMBB is an evolution of today’s 4G use case for mobile users — streaming video, conferencing, and basic broadband connectivity—with increased uplink and downlink capacity.

·         Massive Machine Type Communications (mMTC): mMTC is an evolution of IoT but with orders of magnitude increased endpoint density with a particular focus on power efficiency.

·         Ultra-Reliable and Low Latency Communications (URLLC): This aims to deliver low latency, high reliability, and in many cases, high bandwidth capabilities, enables a new set of applications such as autonomous vehicles, augmented reality, remote real-time surgery and so on.

At the 5G network core, the 3rd Generation Partnership Project (3GPP) defines a native Application Programming Interface (API) for control plane elements’ communications to support the evolution towards virtualization and cloud technology suitability. Figure 2 below shows a 5G network architecture based on virtualization and cloud technologies (i.e. the Telco Cloud).

Figure 2: 5G Telco Cloud Network Architecture

The Telco Cloud is essentially an operator's private cloud to meet the mobile network requirements, thus allowing the deployment of telco workloads at large scale. These workloads are the various 5G architecture components (such as UPF, SMF, or PCF), with specific set of requirements that are very different from traditional IT workloads.

Key Technologies & Considerations for 5G Core

To satisfy the requirements of 5G core network (such as higher bandwidth, lower latency, rapid service provisioning, and dynamic application workloads) while also reducing the overall Total Cost of Ownership (TCO), there are crucial technologies that must be taken into consideration. These include:

1.    Network Functions Virtualization (NFV) & Software-Defined Networking (SDN): The requirements for the 5G network to be highly programmable means the adoption of technologies such as SDN and NFV becomes very fundamental. SDN and NFV are technologies that foster increased virtualization of network appliances, and programmability of network devices and protocols that present a consistent, dynamic, and programmable interfaces. Control and User Plane Separation (CUPS) is an example of the adoption of SDN in the 5G network. NFV enables automated, dynamic, and rapid deployment of network elements (such as UPF, firewalls, or load-balancers) to lower operational and management costs. SDN and NFV are complementary technologies yet increasingly co-dependently drive the full realization of network programmability. The European Telecommunications Standards Institute (ETSI) is leading efforts in the standardization of these technologies and has defined a reference architecture for NFV and its impact on 5G. The reference architecture specifies the components for management and orchestration (MANO) of Virtual Network Functions (VNFs) and defines three functional layers on the management and orchestration side:

·         NFV Orchestrator (NFVO): The orchestrator provides lifecycle management of the network services.

·         VNF Manager (VNFM): The VNFM handles the configuration, lifecycle management, and element management of VNFs.

·         Virtualized Infrastructure Manager (VIM): This layer handles the virtualization of physical hardware in by integrating with virtual-machine managers. Using a hypervisor, the virtual machine manager provides the ability to create multiple virtual compute, network, and storage elements.

2.    Network and Service Orchestration: Given the growing pressure to control the rising operating costs and squeeze in revenue, deploying new network services as demanded by the market has become a persistent challenge for operators. Moreover, the systems used today for the provisioning of these services are not only proprietary and expensive in terms of deployment, maintenance, and enhancement, but too slow to cope with the speed of these deployments. Consequently, the architecture introduces an orchestration layer which covers.

●     Service Orchestration – the instantiation and deployment of E2E multi-vendor network service and configuration across various part of the network (e.g. RAN, Core, Security, Testing) using multi-domain orchestrators.

●     Elements Instantiation – the deployment and placement of network elements in the distributed network (e.g. UPF) within minutes.

●     Auto-Healing & Elastic Scaling – the recovery of failed network elements, Scale-Out/In (dynamic instantiation of additional instances) or Scale-Up/Down (dynamic expansion of resources) of these elements based on network load/performance.

3.    Cloud-Native Software: The term cloud-native is fundamentally associated with containerized microservices application that runs on a cloud platform. The Cloud-Native Computing Foundation defines this as ‘Cloud native computing uses an open-source software stack to deploy applications as microservices, packaging each part into its own container, and dynamically orchestrating those containers to optimize resource utilization’. Additionally, cloud-native applications are characterized as being automatable, ubiquitous, and flexible, resilient, and scalable, dynamic, observable, and distributed. These applications are composed of small granular, highly cohesive, and loosely coupled services, where each microservice fulfills a specific functionality that is self-contained, with the interaction between microservices implemented through a standard light-weight interface (e.g. gRPC or REST). While considerable progress has been achieved through VNFs, several limitations of Physical Network Functions (PNFs) persist such as slow upgrades and restarts, traditional interfaces such as CLI, monolithic software build, complex hypervisor interactions, as well as elasticity and scaling challenges. Cloud-Native Network Functions (CNFs) are the next generation Network Functions (NFs) that are built using Cloud Native principles and technology. These CNFs, running inside telecommunications premises on a public or private cloud, impose a completely new set of networking requirements on the cloud infrastructure different from those of traditional VNFs.

4.    Multi-Access Edge Computing (MEC): MEC makes interesting possibilities become reality in 5G - the promise of a distributed network based on CUPS, where the dynamic placement of decoupled user plane element (i.e. UPF) at the most optimal location (i.e. the network edge), satisfying low latency communications and/or edge offload requirements. MEC is also central in the transformation of the RAN through initiatives such as Open vRAN and Cloud RAN, aimed at virtualizing and disaggregating the otherwise composite radio network components (i.e. the eNodeB or gNodeB) into software-based instances that are hosted on the edge compute nodes, resulting in a very lean with only Remote Radio Heads and antennas. Other applications of MEC in 5G include CDN (Content Distribution Network) for improved managed video delivery (Live & VOD), fog-computing for IOT data pre-processing, connected and autonomous vehicles, streaming games, and so on.

5.    DevOps & Automation Everywhere: DevOps methodology has its roots in enterprise environments based on processes and techniques that rely heavily on automation, enabling significant increases in the efficiency of engineering and delivery of software into operations. In the case of 5G, an entity may be a NF, developed orchestration component, or simply a configuration file. Validating the traversal of an artifact across the various deployment environments allows for rapid feedback for software quality improvement and process optimization. DevOps emphasizes communication, collaboration, integration, and automation - working together with various stakeholders (developers, testers, operations, and so on) every step of the way. The DevOps approach covers the following:

·         Continuous Integration (CI) – the automated process of a secure and frequent integration of source code from various sources (e.g. a team of developers creating code branches for feature development).

·         Continuous Delivery (CD) – the automated process of a frequent build of integrated software (this includes CI as well as the compilation of the source code into executable software).

·         Continuous Release (CR) – the automated process of frequent publishing of software packages to an external repository, ready for deployment (may include various software packages that make up a release).

·         Continuous Deployment (CD) – the automated process of frequent deployment of software packages to the production environment.

·         Continuous Testing (CT) – continuous execution of automated testing based on incremental changes such as configuration or code changes.

Figure 3: DevOps Principles for 5G Core

6.    DevOps & Automation Tools: To achieve the desired objective of high levels of automation and agility required for the deployment and management of the 5G architecture in a DevOps methodology, certain tools must be adopted. These tools have their roots in traditional software development and have since become necessary in the efficient deployment and operation of a 5G network. These tools are:

·         Source Artifact Management: Several components of this architecture require management in the form of a source code. For example, CNF Helm charts, device configuration files, container definition files, and so on. Git, GitHub, GitLab are examples of tools that can be used for this purpose.

·         Binary Artifact Management: Traditional methods of file transfer such as FTP have become obsolete and replaced with solutions (such as JFrog Artifactory, Sonatype Nexus, BinTray) that support native automation and management of binaries and file across the solution delivery process, thereby improving overall efficiency.

·         Test Automation: The automation of testing is an absolute requirement of the DevOps methodology that operators employ to integrate these architecture stacks vertically and horizontally. A well-defined test automation strategy across the various layers of the stack (i.e. network, infrastructure, orchestration, cloud, and so on) and across the various environments (i.e. dev, integration, and production) is central to the success of keeping their total cost of ownership low.

·         DevOps Orchestration: To orchestrate the various elements and tools in the overall DevOps process, a tool (such as Jenkins, CloudBees CI, CircleCI) is required to perform the continuous integration of source and binary artifacts, building software binaries, deployment of configuration and artifacts into various environments, the trigger of automation across various domains (such as deployment, testing, or reporting).

·         Requirements & Scope Management: The definition, management, and traceability of the deployment requirements throughout the customer solution and software life cycle across the environments, tools components, vendors, and stakeholders.

·         Release Management: Automated release and deployment remains the target of this CICD/DevOps model and achieving this in a highly diverse and automated environment will not only remain an incredible achievement but a necessary one too.

Figure 4: Test Automation Requirement

Testing Considerations for 5G Core

The technologies involved in 5G mandate an unprecedented level of automation in testing, deployment, and the operation of the network. It is therefore critical to consider multiple test environments with respect to the development and execution of test automation. Below is an overview of the various environments to be considered for a 5G network test strategy:

·         Pre-Integration Test Environment: This environment essentially covers system testing aimed at the vertical integration testing of the solution. Virtualization and simulation may be employed to cover the testing of functional and non-functional requirements. This environment is also ideal for the initial development of system test automation.

·         Integration Test Environment: This is a multi-vendor environment for both vertical and horizontal end-to-end integration testing where a customer validates N+1 software release of the production network. Thus, the use of test automation is critical in lowering costs, as well as maximizing the efficient use of the environment.

·         Pre-Production Test Environment: This environment represents a replica of the production environment and usually subjected to the same change control procedures. Owned and operated by the customer, this s used for issue replication or testing hotfixes before production deployment.

·         Production Environment: This is environment that provides end-user services.

With a varying degree of constraints in terms of change control and resource allocation, each of these environments indeed has benefits in the customer solution life cycle. Hence, there may be scope for collapsing some of the environments as permitted for the target solution or technology to achieve the desired deployment outcomes.

The deployment of 5G requires a modern and consistent infrastructure that can run both VNFs and CNFs, side-by-side throughout the network (from core to edge). Given the various constituent technologies and their integration (both vertically and horizontally), key areas of test requirements must be identified and defined as outlined below:

·         Infrastructure Stack Testing: The promise of a horizontally and vertically open infrastructure, towards faster innovation, increased agility, and performance, with a reduced TCO, comes with the responsibility of integration of the infrastructure stack. The following are the various areas of testing:

o    Network Infrastructure Testing: This covers the validation of the network infrastructure that provides interconnectivity for the compute, virtualization, and cloud infrastructure. This covers both the DC (i.e. Leaf and Spine architecture) and the transport (e.g. Segment Routing, SR) network infrastructure – covering functionality that is simple, consistent and provides high performance connectivity.

o    Compute & Storage Infrastructure Testing: This covers the validation of the physical server hardware that hosts the virtualization and OS software as well as the data storage capability.

o    Virtualization Platform (i.e. PaaS) Testing: This covers the validation of infrastructure that supports VM-based workloads (i.e. VNF’s). This includes features and functionality of the VIM required to support the placement and lifecycle management of workloads (i.e. VNF Onboarding).

o    Container Platform (i.e. CaaS) Testing: This covers the validation of the features and functionality of container engine and orchestration infrastructure that support CNFs, covering the onboarding and lifecycle management.

o    Network Services Orchestration Testing: This covers the validation of the deployment and arrangement of workloads, and their maintenance and lifecycle management in day-2 (i.e. post-deployment) including the un-deployment (and the consequent reconfiguration) of these workloads.

·         Workload Onboarding Testing: This covers the testing of the onboarding of VNFs and CNFs onto the telco cloud infrastructure, covering the provisioning of the infrastructure (i.e. network, compute, storage), workload packaging, resource assignment, configuration, instantiation and placement.

·         Mobility Application Testing: This covers the validation of the 5G mobile network application and scenarios. This may cover both the radio access and core network functionality such as mobile device access and registration, mobility functions, network slicing, and so on.

·         Assurance Testing: This covers the testing of the Operations Support System (OSS) and Business Support System (BSS) interaction with the 5G network across all layers of the network infrastructure and application. This may cover areas such as billing, workflows, support, analytics, and so on.

Figure 5: Test Automation Strategy for 5G Architecture

Figure 5 above outlines the overall test automation strategy for the 5G architecture. This strategy covers what requirements should be automated and executed against the various test environments.

Due to the possibility of a stable early design of the infrastructure stack, test automation should be developed between the pre-integration and integration environments for execution against the pre-production and production environments. Workload onboarding test automation should be developed in the integration environment to cover both vertical and horizontal integration. Vertical integration requirements cover the placement of the workload onto the infrastructure, its lifecycle management, and its interaction with northbound system component such as MANO or OSS/BSS, while horizontal integration requirements cover the system interaction between workloads to enable an end-to-end call-flow scenarios. Mobility application test automation covers the actual mobile device call-flow scenarios. This is feasible in the integration and pre-production environments, but perhaps also possible in the pre-integration environment using simulation. Similarly, OSS/BSS testing may only be executed in the integration and pre-production environments.

The industrialization of test automation development is critical to the success of DevOps methodology to enable automation re-usability across the various environments. This is the abstraction of test automation scripts from environment artifacts such as configuration files and data (such as environment IP addressing, device access credentials, and so on) and defining these in standard artifact formats (such as YAML or XML files) for compatibility with the employed toolchain.

Conclusion

The promise of 5G revolution requires network operators to undergo a radical transformation. This transformation is underpinned by the adoption of virtualization and cloud-native technologies to deliver faster innovation and reduced operational cost through a vertically and horizontally open network. These disruptive technologies require the use of DevOps principles and tools to achieve the desired levels of automation in testing, deployment, and operations of the network. We have highlighted the critical technologies and considerations that network operators must consider to successfully deploy and manage a 5G network.

Authors