Patent Application
20250126521
The present disclosure relates to wireless communication networks.
5G radio-access network (RAN) in wireless communications uses 5G radio frequencies to provide wireless connectivity to devices. 5G mobile communication revolutionized the way to interact with each other with a number of applications (e.g., telemedicine, robotic surgery, self-organizing autonomous cars, internet of things (IOT), and so on) that are riding on 5G. The 5G offers Gigabit speeds with ultra-low latencies. Further, the telecommunication industry is accelerating as transition to 5G business, container orchestration platform, and cloud-native network functions (CNFs) solutions are getting more attention and deployment.
A RAN is a key component of a mobile telecommunication system that connects devices (e.g., smartphones, personal computers, and the like) to a network via a radio link. This is achieved by converting voice and data into digital signals and transmitting them as radio waves to RAN transceivers, which then forward them onto the core network. From the core network, the data can be sent to the internet. The RAN is the radio element of a cellular network. A cellular network is made up of land areas called base stations or cell sites. In such an environment, stringent latency requirements and the ultra-high speeds imply that a significantly large number of cell sites may have to be deployed. For example, the number of base-stations that has to be deployed in a country such as USA may be in the range of 250,000-300,000.
The drawings described herein are for illustrative purposes and are not intended to limit the scope of the present subject matter in any way.
Examples described herein may provide an enhanced computer-based and/or network-based method, technique, and system to automatically provision cell sites in 5G radio-access networks (RAN). The paragraphs [0011] to [0019] present an overview of 5G RAN, existing methods to provision the cell sites in the 5G RAN, and drawbacks associated with the existing methods.
A RAN is a part of a telecommunications system that connects individual user devices (e.g., a smart phone, a personal computer, and the like) to other parts of a network through radio connections. The RAN may reside between the user devices and provides the connection with its core network. The RAN is a major component of wireless telecommunications and has evolved through the generations of mobile networking leading up to 5G. The RAN provides access to and coordinates the management of resources across cell sites.
The telecommunication industry is accelerating as transition to 5G business, container orchestration platform, and cloud-native network functions (CNFs) solutions are getting more attention and deployment. A container orchestration platform can have multiple pods, with each pod representing a group of one or more application containers, as well as some shared resources for those containers. A container orchestration platform can host different container-based platforms that support different functions. For example, a container-based platform can be added to a container orchestration platform to support telco CNFs. Further, a container-based platform can be added to a container orchestration platform to support Telco RAN CNFs and to make sure that the Infra as a Service (IaaS) and Containers as a service (CaaS) platforms meet network functions' (NFs') requirements and achieve good performance per 5G standards.
In some examples such as 4G network deployments, the RAN cell sites had vertically integrated solutions and the management interface was a closed source. Further, the hardware that was running the RAN software was also proprietary. There was no play for companies like VMware in the 4G network deployments. In 5G, the RAN cell site for the first time would be deployed on commercial-off-the-shelf (COTS) hardware. Further, the RAN software (e.g., containerized network function (CNF)) is deployed on Kubernetes platform, which is also open source. Thus, it is possible for third-party vendors (e.g., such as VMware) to deploy and manage the life cycle management (LCM) of the cell-sites.
Further, along with the progression of mobile technology, the complexity of its management has significantly increased many folds. For example, the manpower to deploy a cell site manually at every tower is cost-prohibitive due to the scale/complexity. The need of the hour is a remote automation solution that can provision the cell site. The physical presence of the human at the cell-site is only utilized for connecting the cables and setting up the networking to reach the automation server.
In some existing scenarios, 5G vendors (e.g., Ericsson, Nokia, and the like) have solutions to deploy their network functions, once the platform is ready. However, they lack the solution to provision the platform or the hardware remotely. In other examples, there are solutions from hardware management companies that can provision their hardware remotely (such as DELL). There are K8s management companies that can deploy Kubernetes platform and configure them remotely (such as rancher). However, with the existing solution an operator may have to:
In an example, Telco Cloud Automation (TCA) is a VMware solution that is responsible for all three functions (e.g., hardware provisioning, K8s provisioning, and CNF deployment). In TCA terminology, the cell-site needs to go through three stages for site provisioning such as host provisioning (e.g., zero-touch provisioning (ZTP)), K8s provisioning (e.g., containers as a service (CAAS)), and network function deployment (e.g., CNF). However, even though TCA has the functionality to deploy a cell site end-to-end (E2E), each of the provisioning steps is independent. Consequently, the results of one phase may not be automatically propagated to the subsequent stage. So, the entire process has to be manual, which reduces the overall deployment velocity. For example, the number of base-stations that need to be deployed may be in the range of 250,000-300,000 for a country such as USA. An example single tier-II operator would own and manage upwards of 35,000 sites. An example tier-I operator is looking at 70,000-100,000 sites. Hence, manually provisioning each of the cite sites may be significantly time consuming.
Further, each of the function has its own user interface (UI) and the UI may not be useful to manage a significantly large number of cell sites (e.g., tens of thousands of cell sites). The UI may have to be minimalistic and only report issues that require human attention. Furthermore, there is no central place to validate whether all the inputs are correct. This can cause misconfigurations relatively easily and lead to failed deployments. Misconfigurations of a specific stage are discovered as the task is executed. Thus, significant effort may be wasted in reverting the changes made due to misconfigurations.
In addition, when a deployment fails for any reason, the error has to be fixed first. Further, a specific stage may have to be identified and the tasks has to be started. There can be issues with this method as the user needs to go to the specific stage, identify the failed task, and follow the procedure listed in the documentation to retrigger. Thus, the user may have to learn each stage. Each stage may have its own payload. Thereby, the user may have to translate the input payload to the format that's accepted by that stage.
Further, the CNF imposes a number of requirements on K8s and hardware. Such requirements may have to be available as part of hardware provisioning or K8S provisioning. However, if any configuration is missing, then the only way to enable it is to write a script and manage it manually. The user may have to make sure that the script is called at the right time (e.g., when a Container Network Interface (CNI) needs to be enabled, the corresponding script may have to be called only after the K8S is deployed). However, a large number of out-of-band modifications are necessary to deploy the CNF. In an example use-case, no less than 15-20 such modifications were used in production deployments. Moreover, there is no easy way to track the progress of a single cell site with all the scripts. Managing such a solution at scale may be challenging and invariably leads to significantly more manpower being employed to troubleshoot the cell sites.
Examples described herein may provide a management node to automatically provision cell sites in 5G radio-access networks. The management node may automate deployment, management, and scaling of containerized network function (CNF) instances. During operation, the management node may receive a plurality of steps involved in provisioning a cell site for a 5G RAN. In an example, provisioning the cell site may include provisioning of a physical infrastructure layer, a container orchestration platform on the physical infrastructure layer, and a CNF instance associated with the 5G RAN in the container orchestration platform. Further, the management node may convert the plurality of steps into a dependency graph of tasks. The dependency graph may represent workflows and relationships between the tasks. Furthermore, based on feeding the dependency graph as an input, the management node may provision the cell site by executing the tasks in an order according to the dependency graph.
Thus, examples described herein may enable to deploy a generic infrastructure of the cell site and then customize the cell site on the fly. Also, examples described herein may make it possible to deploy the RAN application on top of the generic infrastructure.
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present techniques. However, the example apparatuses, devices, and systems, may be practiced without these specific details. Reference in the specification to “an example” or similar language means that a particular feature, structure, or characteristic described may be included in at least that one example but may not be in other examples.
Referring now to the figures,
Further, the RAN is a key component of a mobile telecommunication system that connects the user devices to a network via a radio link. This may be achieved by converting voice and data into digital signals and transmitting them as radio waves to RAN transceivers, which then forward them onto the core network. From the core network, the data can be sent to the internet. Furthermore, to achieve such connections between the user devices and the network, at least one cell site (e.g., 114A-114N) may have to be provisioned.
In an example, cell sites 114A-114N may be provisioned by a management node 102. Further, management node 102 may include a processor 104. Processor 104 may refer to, for example, a central processing unit (CPU), a semiconductor-based microprocessor, a digital signal processor (DSP) such as a digital image processing unit, or other hardware devices or processing elements suitable to retrieve and execute instructions stored in a storage medium, or suitable combinations thereof. Processor 104 may, for example, include single or multiple cores on a chip, multiple cores across multiple chips, multiple cores across multiple devices, or suitable combinations thereof. Processor 104 may be functional to fetch, decode, and execute instructions as described herein. Further, management node 102 includes memory 106 coupled to processor 104. In an example, memory 106 may include a container orchestrator 108, cell site provisioning unit 110, and a validation unit 112. Furthermore, processor 104 may execute container orchestrator 108, cell site provisioning unit 110, and a validation unit 112 stored in memory 106.
In an example, container orchestrator 108 may automate deployment, management, and scaling of containerized network function (CNF) instances. During operation, cell site provisioning unit 110 may receive a plurality of steps involved in provisioning a cell site (e.g., 114A-114N) for a 5G RAN. In an example, provisioning cell site 114A-114N may include provisioning of a physical infrastructure layer, a container orchestration platform on the physical infrastructure layer, and a CNF instance associated with the 5G RAN in the container orchestration platform. In an example, based on requirements of the CNF instance, container orchestrator 108 may provision the physical infrastructure layer by preparing a physical host computing system to configure hardware, software, and network resources.
Further, cell site provisioning unit 110 may convert the plurality of steps into a dependency graph of tasks. In an example, the dependency graph may represent workflows and relationships between the tasks. For example, the dependency graph may include a plurality of vertices and a plurality of edges, each vertex representing a task for execution and each edge representing a path that the container orchestrator needs to take upon completion of each task. An example task may include validation of input, validation of the host, install an operating system (OS) on the host, upgrade OS/Packages on the host, configure networking on the host, update networking on the host, copy VM images to the host, configure firewall rules on the host, deploy Kubernetes worker node on the host, update/upgrade Kubernetes (i.e., docker engine), deploy CNF, upgrade CNF, delete CNF, or any combination thereof.
Furthermore, based on feeding the dependency graph as an input to container orchestrator 108, cell site provisioning unit 110 may provision cell site 114A-114N by executing the tasks in an order according to the dependency graph.
In an example, prior to provisioning cell site 112A-112N, validation unit 112 may receive method of procedure (MOP) steps to validate hardware and/or software requirements of an OS of the physical infrastructure layer, the container orchestration platform, and the CNF instance from multiple vendors. Further, validation unit 112 may convert the MOP steps into a defined data structure. In an example, the defined data structure is generic and extensible. Furthermore, validation unit 112 may validate, based on feeding the defined data structure as an input to the container orchestrator, the hardware and/or software requirements of the operating system, the container orchestration platform, and the CNF.
In an example, for each hardware and/or software requirement, validation unit 112 may enable container orchestrator 108 to initiate a command on the physical infrastructure layer by making a call to an interface exposed by a respective hardware and/or software component in the physical infrastructure layer. Further, validation unit 112 may compare a result of executing the command with an expected output in the defined data structure. In one example, when the result of executing the command matches the expected output, validation unit 112 may determine that the validation is successful. In another example, when the result of executing the command does not match the expected output, validation unit 112 may determine that the validation is not successful.
In an example, container orchestrator 108 may determine whether the physical infrastructure layer and the container orchestration platform support requirements of the CNF instance. In one example, when the physical infrastructure layer and the container orchestration platform support requirements of the CNF instance, container orchestrator 108 may invoke, by the tasks, documented application programming interfaces (APIs) to achieve a required functionality. In another example, when the physical infrastructure layer and the container orchestration platform do not support requirements of the CNF instance, container orchestrator 108 may generate and add additional tasks to the orchestrator to support the requirements of the CNF instance.
Example system 200 may include a data center 202 including a management node to deploy and manage multiple cell sites 214A-214N. As shown in
Further, Telco cloud platform or Telco Cloud Automation (TCA) 206 is a VMware solution that is responsible for functions such as hardware provisioning (e.g., 208), K8s provisioning (210) and CNF deployment (e.g., 212). In TCA, cell sites 214A-214N may have to go through three stages for site provisioning such as host provisioning (e.g., Zero-touch provisioning (ZTP) 208), K8s Provisioning (e.g., containers-as-a-service CAAS 210), and network function deployment (e.g., CNF 212). An orchestrator would run on top of TCA to stitch together these functions and provide a single workflow for the end-user to execute.
During operation, container orchestrator 204 may convert steps (e.g., sequence of steps that need to be followed to provision a cite site (e.g., 214A)) involved in bringing up cell site 214A to orchestrator tasks (e.g., a graph of tasks (e.g., task 1-task N) that can be consumed by container orchestrator 204). For example, the vertices of the graph may represent the tasks that may have to be executed. The edges may represent the path that container orchestrator 204 may have to take upon completion of each task. Further, the graph is fed to container orchestrator 204 so that container orchestrator 204 knows the sequence and the order of tasks that need to be executed to provision cell site 214A. In the example shown in
In an example, the tasks (e.g., task 1-task N) in container orchestrator 204 may be developed in an idempotent fashion. For example, upon correcting the failed sites, a user may have to call only one application programing interface (API) irrespective of where the failure occurred. The idempotency ensures that the user may have to know the schema of the first task. In case of retriggering failed sites, idempotency ensures that user needs to use one payload and one API call irrespective of where the failure occurred. Further, container orchestrator 204 may manage all the translations between vendors as tasks.
In an example, container orchestrator 204 may monitor each of tasks 1-task N. Thus, container orchestrator 204 may track whether the orchestrator task has started execution and whether the execution is successful or not. Further, container orchestrator 204 may send information about the progress of the task to external stakeholders using technologies such as message bus, Representational State Transfer (REST) APIs, or so on. These updates can be processed and stored in a database (such as Prometheus, Victoria Metrics, Elastic Search, and the like). For example, for a given cell site, container orchestrator 204 may use a common ID (e.g., a site-id) across different tasks to enable creation of the site view. Further, every update to the external stakeholders may include the site-id so that the identity of the cell site associated with the update is established. Furthermore, queries on the database may be used to create a site level view (e.g., using the site-id as a key) from the task updates. The use of a common site-id ensures to create a site-view using task specific updates. Further, the site-view provides the user everything they need to know about the cell site progress in a single graphical user interface (GUI).
In one example, when underlying products (e.g., host provisioning and K8s provisioning) support the required CNF capability, the task would invoke the documented APIs of the product to achieve the required functionality. In another example, when the underlying products do not support required CNF capability, additional tasks are added in container orchestrator 204 itself to support the capability. In an example, container orchestrator 204 may control the order in which these steps need to be executed.
Thus, container orchestrator 204 can manage execution of thousands of tasks in parallel. This makes container orchestrator 204 approach ideal to reach significantly high cell site provisioning velocities. Thus, with the examples described herein, the E2E pipeline may significantly reduce human errors and in general the need for human interaction and the product UI is required only for identifying failures, which is significantly smaller. Also, examples described herein may allow operator to provision the entire cell site (e.g., from hardware provisioning to CNF deployment) in a single workflow. Further, the use of an orchestrator framework allows the user to add custom tasks in the future making the solution extensible to handle new CNFs, Kubernetes/host versions, and so on. Thus, examples described herein may allow the operator to deploy/manage a significantly large number (e.g., hundreds) of cell sites in parallel, and improving the speed of bringing up the cell sites.
In general, the vendors list these requirements as part of a method of Procedure (MOP) or the installation guide. Further, a customer may prepare the environment as per the MOP/Installation guides. Furthermore, the customer may validate the requirements before triggering the deployment of the specific software. In the examples described herein, the pre-requisites may be validated by converting the MOP steps into a task that gets invoked by a container orchestrator. As a first step, notice that every software component includes well defined interfaces to interact with it. Even if no operating system is deployed on the server, commercial servers may expose an intelligent platform management interface (IPMI) to interact with the hardware. Examples described herein may propose that the validation task in the container orchestrator may be a collection of plugins for interacting with specific interfaces (e.g., Secure Socket Shell (SSH), Representational State Transfer (REST), IPMI, Redfish, and so on).
In an example, the validations required for a software component may be converted to JSON (or YAML) data 300 as shown in
In an example, the plugins may go through the collection, execute the command using the appropriate protocol, and compare the output to the expected output. For example, validation may be successful when all the actual outputs match their expected outputs. In another example, the validation may fail when any of the outputs do not match the expected output.
Thus, in the examples described herein, the validation task in the container orchestrator may include a number of protocol plugins to interact with different infrastructure components. For example, the plugins execute the commands in the JSON input and compare the output with the expected output in the JSON file.
Examples described herein may provide an approach to provision cell sites in a 5G RAN that can be considered universal. For example, examples described herein may be applicable to all vendors, so long as their requirements can be validated via the supported interfaces. Further, the approach may be future proof. The component upgrades may only require changes to the container orchestrator input and no changes to the task may be necessary. For Example, consider Kubernetes Baremetal Deployment 1.22 requires BIOS version of 1.6.5 on a Dell server. Further, consider that the Kubernetes version is upgraded to 1.23 and this introduces two new requirements such as BIOS version 1.7.2 and a memory speed of 3200 MHz. This may introduce two changes to the input. In the BIOS section, the expected output is modified to 1.7.2. A new section is added that checks the memory speed and matches it to 3200 MHz or greater.
At 402, a plurality of steps involved in provisioning the cell site for the 5G RAN may be received. In an example, provisioning the cell site may include provisioning of a physical infrastructure layer, a container orchestration platform on the physical infrastructure layer, and a containerized network function (CNF) instance associated with the 5G RAN in the container orchestration platform. For example, provisioning the physical infrastructure layer may include, based on requirements of the CNF instance, preparing a physical host computing system to configure hardware, software, and network resources.
At 404, the plurality of steps may be converted into a dependency graph of tasks. In an example, the dependency graph may represent workflows and relationships between the tasks. For example, converting the plurality of steps into the dependency graph may include constructing the dependency graph including a plurality of vertices and a plurality of edges, each vertex representing a task for execution and each edge representing a path that the orchestrator needs to take upon completion of each task. In an example, each task in the orchestrator may be designed to be idempotent, where a user needs to call an application programming interface (API) irrespective of where the failure occurred in the provisioning of the cell site.
Based on feeding the dependency graph as an input to an orchestrator, at 406, the cell site may be provisioned by executing the tasks in an order according to the dependency graph. In an example, provisioning the cell site may include determining whether the physical infrastructure layer and the container orchestration platform support requirements of the CNF instance. In one example, when the physical infrastructure layer and the container orchestration platform support requirements of the CNF instance, the tasks may invoke documented application programming interfaces (APIs) of the physical infrastructure layer and the container orchestration platform to achieve a required functionality. In another example, when the physical infrastructure layer and the container orchestration platform do not support requirements of the CNF instance, additional tasks may be generated and added to the orchestrator to support the requirements of the CNF instance.
In an example, prior to provisioning the cell site, method 400 may include receiving method of procedure (MOP) steps to validate hardware and/or software requirements of an operating system of the physical infrastructure layer, the container orchestration platform, and the CNF instance from multiple vendors. Further, the MOP steps may be converted into a defined data structure. Furthermore, based on feeding the defined data structure as an input to the orchestrator, the hardware and/or software requirements of the operating system, the container orchestration platform, and the CNF may be validated.
In an example, for each hardware and/or software requirement, validating the hardware and/or software requirements of the operating system, the container orchestration platform, and the CNF may include initiating, by the orchestrator, a command on the physical infrastructure layer by making a call to an interface exposed by a respective hardware and/or software component in the physical infrastructure layer. Further, the result of executing the command may be compared with an expected output in the defined data structure by the orchestrator. In an example, when the result of executing the command matches the expected output, the orchestrator may determine that the validation is successful. In another example, when the result of executing the command does not match the expected output, the orchestrator may determine that the validation is not successful.
Further, method 400 may include monitoring the tasks to determine progress of execution of each task by the orchestrator. Further, the orchestrator may send the progress of each task to a monitoring platform using a common identifier associated with the cell site to establish an identity of the cell site.
In another example, the orchestrator may monitor the tasks to determine the progress of execution of each task. Further, the orchestrator may store the progress of each task in a database using a common identifier associated with the cell site. In response to receiving a request, the orchestrator may create a site level view of the cell site by querying the database using the common identifier.
Computer-readable storage medium 504 may store instructions 506, 508, and 510. Instructions 506 may be executed by processor 502 to receive a plurality of steps involved in provisioning the cell site for the 5G RAN. In an example, provisioning the cell site may include provisioning of a physical infrastructure layer, a container orchestration platform on the physical infrastructure layer, and a containerized network function (CNF) instance associated with the 5G RAN in the container orchestration platform.
Instructions 508 may be executed by processor 502 to convert the plurality of steps into a dependency graph of tasks. The dependency graph may represent workflows and relationships between the tasks. In an example, instructions 508 to convert the plurality of steps into the dependency graph may include instructions to construct the dependency graph comprising a plurality of vertices and a plurality of edges, each vertex representing a task for execution and each edge representing a path that the orchestrator needs to take upon completion of each task.
Instructions 510 to may include instructions to provision, based on feeding the dependency graph as an input to an orchestrator, the cell site by executing the tasks in an order according to the dependency graph. In an example, instructions 510 to provision the cell site may include instructions to determine whether the physical infrastructure layer and the container orchestration platform support requirements of the CNF instance. In an example, when the physical infrastructure layer and the container orchestration platform support requirements of the CNF instance, documented application programming interfaces (APIs) of the physical infrastructure layer and the container orchestration platform may be invoked by the tasks to achieve a required functionality. In another example, when the physical infrastructure layer and the container orchestration platform do not support requirements of the CNF instance, additional tasks may be generated and added to the orchestrator to support the requirements of the CNF instance.
In an example, prior to provisioning the cell site, computer-readable storage medium 504 may store instructions to receive method of procedure (MOP) steps to validate hardware and/or software requirements of an operating system of the physical infrastructure layer, the container orchestration platform, and the CNF instance from multiple vendors. Further, computer-readable storage medium 504 may store instructions to convert the MOP steps into a defined data structure and validate, based on feeding the defined data structure as an input to the orchestrator, the hardware and/or software requirements of the operating system, the container orchestration platform, and the CNF.
Further, computer-readable storage medium 504 may store instructions to monitor, by the orchestrator, the tasks to determine a progress of execution of each task, store, by the orchestrator, the progress of each task in a database using a common identifier associated with the cell site and in response to receiving a request, create, by the orchestrator, a site level view of the cell site by querying the database using the common identifier.
The above-described examples are for the purpose of illustration. Although the above examples have been described in conjunction with example implementations thereof, numerous modifications may be possible without materially departing from the teachings of the subject matter described herein. Other substitutions, modifications, and changes may be made without departing from the spirit of the subject matter. Also, the features disclosed in this specification (including any accompanying claims, abstract, and drawings), and any method or process so disclosed, may be combined in any combination, except combinations where some of such features are mutually exclusive.
The terms “include,” “have,” and variations thereof, as used herein, have the same meaning as the term “comprise” or appropriate variation thereof. Furthermore, the term “based on”, as used herein, means “based at least in part on.” Thus, a feature that is described as based on some stimulus can be based on the stimulus or a combination of stimuli including the stimulus. In addition, the terms “first” and “second” are used to identify individual elements and may not be meant to designate an order or number of those elements.
The present description has been shown and described with reference to the foregoing examples. It is understood, however, that other forms, details, and examples can be made without departing from the spirit and scope of the present subject matter that is defined in the following claims.
This application is a continuation of PCT/CN2023/124475 filed on Oct. 13, 2023, by LAKSHMIKANTA et al. and titled, “AUTOMATED CELL SITE PROVISIONING IN 5G RADIO-ACCESS NETWORKS,” the entire teachings of which is incorporated herein by reference in its entirety for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2023/124475 | Oct 2023 | WO |
| Child | 18405549 | US |
