CELLULAR NETWORK TEST ENVIRONMENTS FOR EXTERNAL NETWORKS

BACKGROUND

Testing a network and services that use the network to communicate with various devices can be complicated. Testing may be desired to ensure that, under suboptimal conditions, the service, the network, and the devices will continue to function adequately. As an example, under normal operating conditions, latency in communication between the devices and the service may be low. However, if there is a problem with the network, interference, or some other problem, the amount of latency, jitter, or both may increase substantially. The operator of the service, network, and devices may want to perform testing to understand how the system will react under such circumstances.

SUMMARY

Various embodiments are described related to a cellular network test system. In some embodiments, a cellular network test system is described. The system may comprise a cellular network comprising a radio access network (RAN), a cellular network core, and an integrated test system. The system may comprise a plurality of user equipment (UE). Each UE may be configured to communicate with the cellular network using the RAN of the cellular network. Each UE may be configured to communicate with a first external local network that is separate and distinct from the cellular network. The system may comprise the first external local network that hosts a service for the plurality of UE. The integrated test system of the cellular network configures at least a subset of the UE for a test of the service hosted by the first external local network.

Embodiments of such a system may include one or more of the following features: the integrated test system of the cellular network may be configured to perform a set of baseline measurements on the service prior to the test of the service being performed. The test of the service may be selected from a test profile datastore of the integrated test system of the cellular network. The test may be selected at least partially based on a type of the service and a type of the UE. The system may further comprise a second external local network that hosts the service for a second plurality of UE. The integrated test system of the cellular network may configure at least a second subset of the UE for a second test of the service by the second external local network. The second external local network may be separate and distinct from the first external local network. The integrated test system of the cellular network may be further configured to perform a comparison using results of the test with results of the second test. A core of the cellular network and the integrated test system of the cellular network may be hosted on a public cloud computing platform. Configuring the subset of UE may comprise configuring the subset of UE to communicate with the service hosted by the first external local network via the cellular network despite being connected with the first external local network. Configuring the subset of UE comprises configuring the subset of UE to introduce a defined amount of latency when communicating with the service hosted by the first external local network. The cellular network may be a 5G New Radio (NR) cellular network.

In some embodiments, a method for testing a network external to a cellular network. The method may comprise accessing an external local network by a cellular network test system integrated as part of the cellular network. The method may comprise communicating, by the cellular network test system of the cellular network, with a plurality of user equipment (UE) to configure the plurality of UE to perform a test of a service on the external local network. The plurality of UE may be configured to communicate with the cellular network and with the external local network. Performing the test of the service on the external local network using the plurality of UE as configured by the cellular network test system.

Embodiments of such a method may include one or more of the following: the method may further comprise performing, by the cellular network test system of the cellular network, a set of baseline measurements on the service prior to performing the test of the service. The method may further comprise selecting, by the cellular network test system of the cellular network, the test of the service from a test profile datastore of the cellular network. The test may be selected at least partially based on a type of the service and a type of the UE. The method may further comprise communicating, by the cellular network test system of the cellular network, with a second plurality of UE to configure the second plurality of UE to perform a second test of a second service on a second external local network. The second plurality of UE may be configured to communicate with the cellular network and with the second external local network. The method may further comprise performing the second test of the second service on the second external local network using the second plurality of UE as configured by the cellular network test system. The second external local network may be separate and distinct from the external local network. The method may further comprise performing, by the cellular network test system of the cellular network, a comparison using results of the test with results of the second test. A core of the cellular network and the cellular network test system of the cellular network are hosted on a public cloud computing platform. Communicating with the plurality of UE may comprise configuring the plurality of UE to communicate with the service hosted by the external local network via the cellular network despite being connected with the external local network. Communicating with the plurality of UE may comprise configuring the plurality of UE to introduce a defined amount of latency when communicating with the service hosted by the external local network. The cellular network may be a 5G New Radio (NR) cellular network.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of various embodiments may be realized by reference to the following figures. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguish among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

FIG. 1A illustrates an embodiment of a 5G cellular network.

FIG. 1B illustrates an embodiment of a core of a 5G cellular network.

FIG. 2 illustrates an embodiment of a cellular network core implemented using a cloud computing platform.

FIG. 3 illustrates an embodiment of a cellular network providing network connectivity and integrated testing capabilities for external networks.

FIG. 4 illustrates an embodiment of a test system that can operate as part of the cellular network of FIG. 3.

FIG. 5 illustrates an embodiment of a method for testing networks external to the cellular network using the integrated testing capabilities of the cellular network.

DETAILED DESCRIPTION

Various devices, which can be referred to as user equipment (UE), can have the ability to communicate with both a local network (e.g., wireless local area network, wired ethernet network) and a cellular network, such as a 5G New Radio (NR) cellular network. Such an arrangement can be beneficial such that if the UE moves out of range of the local network, the cellular network can be used instead. Additional advantages can also be present. By having the UE able to communicate with multiple networks, the cellular network can be leveraged to perform testing involving the local network, the devices on the local network, and the services provided to the devices via the local network.

The operator of a cellular network may have significant testing experience and data that can be leveraged by entities with less testing experience. The cellular network operator may make its cellular network available for testing purposes to many different unaffiliated entities. By being able to communicate directly with UE operating on various local networks, the cellular network can configure tests to see how a local network and its hosted services behave in response to ordinary and extraordinary operating conditions. Such testing can involve “chaos” testing, which can involve simulating various possible equipment, network, and software failures. Examples of such tests can involve simulating increased traffic volumes, latency, packet loss, jitter, decreased availability of processing resources, offline devices, communication failures (e.g. fiber cuts), etc.

Tests designed and hosted by the cellular network can be reused across many unrelated entities that each have the ability to have their devices communicate with the cellular network. Such an arrangement can help prevent each entity from having to design and create their own test programs and further allows the cellular network to provide comparisons of how disparate entities' systems behave under the same or similar testing. For example, a same service used by multiple entities may behave vastly differently for each entity based on other factors, such as the architecture of each entity's local network.

FIG. 1A illustrates an embodiment of a cellular network system 100 (“system 100”). System 100 can include a 5G New Radio (NR) cellular network; other types of cellular networks, such as 4G LTE, 6G, 7G, etc. are also possible. System 100 can include: UE 110 (UE 110-1, UE 110-2, UE 110-3); base station 115; cellular network 120; radio units 125 (“RUs 125”); distributed units 127 (“DUs 127”); centralized unit 129 (“CU 129”); core 139, and orchestrator 138. FIG. 1A represents a component level view. In a virtualized open radio access network (O-RAN), because components can be implemented as software in the cloud, except for components that need to receive and transmit RF, the functionality of various components can be shifted among different servers, for which the hardware may be maintained by a separate (public) cloud-service provider, to accommodate where the functionality of such components is needed, as detailed in relation to FIG. 2.

UE 110 can represent various types of end-user devices, such as smartphones, cellular modems, cellular-enabled computerized devices, sensor devices, manufacturing equipment, gaming devices, access points (APs), any computerized device capable of communicating via a cellular network, etc. UE can also represent any type of device that has incorporated a 5G interface, such as a 5G modem. Examples include sensor devices, Internet of Things (IoT) devices, manufacturing robots; unmanned aerial (or land-based) vehicles, network-connected vehicles, environmental sensors, etc. UE 110 may use RF to communicate with various base stations of cellular network 120. As illustrated, two base stations 115 (BS 115-1, 115-2) are illustrated. Real-world implementations of system 100 can include many (e.g., hundreds, thousands) of base stations, and many RUs, DUs, and CUs. BS 115 can include one or more antennas that allow RUs 125 to communicate wirelessly with UEs 110. RUs 125 can represent an edge of cellular network 120 where data is transitioned to wireless communication. The radio access technology (RAT) used by RU 125 may be 5G New Radio (NR), or some other RAT, such as 4G Long Term Evolution (LTE). The remainder of cellular network 120 may be based on an exclusive 5G architecture, a hybrid 4G/5G architecture, a 4G architecture, or some other cellular network architecture. Base station equipment 121 may include an RU (e.g., RU 125-1) and a DU (e.g., DU 127-1) located on site at the base station. In some embodiments, the DU may be physically remote from the RU. For instance, multiple DUs may be housed at a central location and connected to geographically distant (e.g., within a couple kilometers) RUs.

One or more RUs, such as RU 125-1, may communicate with DU 127-1. As an example, at a possible cell site, three RUs may be present, each connected with the same DU. Different RUs may be present for different portions of the spectrum. For instance, a first RU may operate on the spectrum in the citizens broadcast radio service (CBRS) band while a second RU may operate on a separate portion of the spectrum, such as, for example, band 71. One or more DUs, such as DU 127-1, may communicate with CU 129. Collectively, RUs, DUs, and CUs create a gNodeB, which serves as the radio access network (RAN) of cellular network 120. CU 129 can communicate with core 139. The specific architecture of cellular network 120 can vary by embodiment. Edge cloud server systems outside of cellular network 120 may communicate, either directly, via the Internet, or via some other network, with components of cellular network 120. For example, DU 127-1 may be able to communicate with an edge cloud server system without routing data through CU 129 or core 139. Other DUs may or may not have this capability.

At a high level, the various components of a gNodeB can be understood as follows: RUs perform RF-based communication with UE. DUs support lower layers of the protocol stack such as the radio link control (RLC) layer, the medium access control (MAC) layer, and the physical communication layer. CUs support higher layers of the protocol stack such as the service data adaptation protocol (SDAP) layer, the packet data convergence protocol (PDCP) layer and the radio resource control (RRC) layer. A single CU can provide service to multiple co-located or geographically distributed DUs. A single DU can communicate with multiple RUs.

Further detail regarding exemplary core 139 is provided in relation to FIG. 1B. Core 139, which can be physically distributed across data centers or located at a central national data center (NDC) as detailed in relation to FIG. 2, can perform various core functions of the cellular network. Core 139 can include: network resource management components 150; policy management components 160; subscriber management components 170; and packet control components 180. Individual components may communicate on a bus, thus allowing various components of core 139 to communicate with each other directly. Core 139 is simplified to show some key components. Implementations can involve additional other components.

Network resource management components 150 can include: Network Repository Function (NRF) 152 and Network Slice Selection Function (NSSF) 154. NRF 152 can allow 5G network functions (NFs) to register and discover each other via a standards-based application programming interface (API). NSSF 154 can be used by AMF 182 to assist with the selection of a network slice that will serve a particular UE.

Policy management components 160 can include: Charging Function (CHF) 162 and Policy Control Function (PCF) 164. CHF 162 allows charging services to be offered to authorized network functions. Converged online and offline charging can be supported. PCF 164 allows for policy control functions and the related 5G signaling interfaces to be supported.

Subscriber management components 170 can include: Unified Data Management (UDM) 172 and Authentication Server Function (AUSF) 174. UDM 172 can allow for generation of authentication vectors, user identification handling, NF registration management, and retrieval of UE individual subscription data for slice selection. AUSF 174 performs authentication with UE.

Packet control components 180 can include: Access and Mobility Management Function (AMF) 182 and Session Management Function (SMF) 184. AMF 182 can receive connection- and session-related information from UE and is responsible for handling connection and mobility management tasks. SMF 184 is responsible for interacting with the decoupled data plane, creating updating and removing Protocol Data Unit (PDU) sessions, and managing session context with the User Plane Function (UPF).

User plane function (UPF) 190 can be responsible for packet routing and forwarding, packet inspection, QoS handling, and external PDU sessions for interconnecting with a Data Network (DN) 195 (e.g., the Internet) or various access networks 197. Access networks 197 can include the RAN of cellular network 120 of FIG. 1A.

While FIGS. 1A and 1B illustrate various components of cellular network 120, it should be understood that other embodiments of cellular network 120 can vary the arrangement, communication paths, and specific components of cellular network 120. While RU 125 may include specialized radio access componentry to enable wireless communication with UE 110, other components of cellular network 120 may be implemented using either specialized hardware, specialized firmware, and/or specialized software executed on a general-purpose server system. In a virtualized arrangement, specialized software on general-purpose hardware may be used to perform the functions of components such as DU 127, CU 129, and core 139. Functionality of such components can be co-located or located at disparate physical server systems. For example, certain components of core 139 may be co-located with components of CU 129.

In a possible O-RAN implementation, DUs 127, CU 129, core 139, and/or orchestrator 138 can be implemented virtually as software being executed by general-purpose computing equipment, such as in a data center. Therefore, depending on needs, the functionality of a DU, CU, and/or 5G core may be implemented locally to each other and/or specific functions of any given component can be performed by physically separated server systems (e.g., at different server farms). For example, some functions of a CU may be located at a same server facility as where the DU is executed, while other functions are executed at a separate server system. In the illustrated embodiment of system 100, cloud-based cellular network components 128 include CU 129, core 139, and orchestrator 138. In some embodiments, DUs 127 may be partially or fully added to cloud-based cellular network components 128. Such cloud-based cellular network components 128 may be executed as specialized software executed by underlying general-purpose computer servers. Cloud-based cellular network components 128 may be executed on a public third-party cloud-based computing platform or a cloud-based computing platform operated by the same entity that operates the RAN. A cloud-based computing platform may have the ability to devote additional hardware resources to cloud-based cellular network components 128 or implement additional instances of such components when requested. A “public” cloud-based computing platform refers to a platform where various unrelated entities can each establish an account and separately utilize the cloud computing resources, the cloud computing platform managing segregation and privacy of each entity's data.

Kubernetes, or some other container orchestration platform, can be used to create and destroy the logical DU, CU, or 5G core units and subunits as needed for the cellular network 120 to function properly. Kubernetes allows for container deployment, scaling, and management. As an example, if cellular traffic increases substantially in a region, an additional logical DU or components of a DU may be deployed in a data center near where the traffic is occurring without any new hardware being deployed. (Rather, processing and storage capabilities of the data center would be devoted to the needed functions.) When the need for the logical DU or subcomponents of the DU no longer exists, Kubernetes can allow for removal of the logical DU. Kubernetes can also be used to control the flow of data (e.g., messages) and inject a flow of data to various components. This arrangement can allow for the modification of nominal behavior of various layers.

The deployment, scaling, and management of such virtualized components can be managed by orchestrator 138. Orchestrator 138 can represent various software processes executed by underlying computer hardware. Orchestrator 138 can monitor cellular network 120 and determine the amount and location at which cellular network functions should be deployed to meet or attempt to meet service level agreements (SLAs) across slices of the cellular network.

Orchestrator 138 can allow for the instantiation of new cloud-based components of cellular network 120. As an example, to instantiate a new DU, orchestrator 138 can perform a pipeline of calling the DU code from a software repository incorporated as part of, or separate from, cellular network 120; pulling corresponding configuration files (e.g., helm charts); creating Kubernetes nodes/pods; loading DU containers; configuring the DU; and activating other support functions (e.g., Prometheus, instances/connections to test tools).

A network slice functions as a virtual network operating on cellular network 120. Cellular network 120 is shared with some number of other network slices, such as hundreds or thousands of network slices. Communication bandwidth and computing resources of the underlying physical network can be reserved for individual network slices, thus allowing the individual network slices to reliably meet particular SLA levels and parameters. By controlling the location and amount of computing and communication resources allocated to a network slice, the SLA attributes for UE on the network slice can be varied on different slices. A network slice can be configured to provide sufficient resources for a particular application to be properly executed and delivered (e.g., gaming services, video services, voice services, location services, sensor reporting services, data services, etc.). However, resources are not infinite, so allocation of an excess of resources to a particular UE group and/or application may be desired to be avoided. Further, a cost may be attached to cellular slices: the greater the amount of resources dedicated, the greater the cost to the user; thus optimization between performance and cost is desirable.

Particular network slices may only be reserved in particular geographic regions. For instance, a first set of network slices may be present at RU 125-1 and DU 127-1, a second set of network slices, which may only partially overlap or may be wholly different from the first set, may be reserved at RU 125-2 and DU 127-2.

Further, particular cellular network slices may include some number of defined layers. Each layer within a network slice may be used to define QoS parameters and other network configurations for particular types of data. For instance, high-priority data sent by a UE may be mapped to a layer having relatively higher QoS parameters and network configurations than lower-priority data sent by the UE that is mapped to a second layer having relatively less stringent QoS parameters and different network configurations.

As illustrated in FIG. 1A, UE 110 may be operating on one or more production slices of cellular network 120. As detailed later in this document, UE that function on a particular entity's local network may be assigned to a slice particular to the entity or a slice that provides a particular QoE for tasks to be performed by the entity's UE.

Components such as DUs 127, CU 129, orchestrator 138, and core 139 may include various software components that are required to communicate with each other, handle large volumes of data traffic, and are able to properly respond to changes in the network. In order to ensure not only the functionality and interoperability of such components, but also the ability to respond to changing network conditions and the ability to meet or perform above vendor specifications, significant testing must be performed.

FIG. 2 illustrates an embodiment of a cellular network core network topology 200 as implemented on a public cloud-computing platform. Cellular network core network topology 200 can represent how logical cellular network groups are distributed across cloud computing infrastructure of cloud computing platform 201. Cloud computing platform 201 can be logically and physically divided up into various different cloud computing regions 210. Each of cloud computing regions 210 can be isolated from other cloud computing regions to help provide fault tolerance, fail-over, load-balancing, and/or stability and each of cloud computing regions 210 can be composed of multiple availability zones, each of which can be a separate data center located in general proximity to each other (e.g., within 100 miles). Further, each of cloud computing regions 210 may provide superior service to a particular geographic region based on physical proximity. For example, cloud computing region 210-1 may have its datacenters and hardware located in the northeast of the United States while cloud computing region 210-2 may have its datacenters and hardware located in California. For simplicity, the details of the cellular network as executed in only cloud computing region 210-1 is illustrated. Similar components may be executed in other cloud computing regions of cloud computing regions 210 (210-2, 210-3, 210-n).

In other embodiments, cloud computing platform 201 may be a private cloud computing platform. A private cloud computing platform may be maintained by a single entity, such as the entity that operates the hybrid cellular network. Such a private cloud computing platform may be only used for the hybrid cellular network and/or for other uses by the entity that operates the hybrid cellular network (e.g., streaming content delivery).

Each of cloud computing regions 210 may include multiple availability zones 215. Each of availability zones 215 may be a discrete data center or group of data centers that allows for redundancy that allows for fail-over protection from other availability zones within the same cloud computing region. For example, if a particular data center of an availability zone experiences an outage, another data center of the availability zone or separate availability zone within the same cloud computing region can continue functioning and providing service. A logical cellular network component, such as a national data center, can be created in one or across multiple availability zones 215. For example, a database that is maintained as part of NDC 230 may be replicated across availability zones 215; therefore, if an availability zone of the cloud computing region is unavailable, a copy of the database remains up-to-date and available, thus allowing for continuous or near continuous functionality.

On a (public) cloud computing platform, cloud computing region 210-1 may include the ability to use a different type of data center or group of data centers, which can be referred to as local zones 220. For instance, a client, such as a provider of the hybrid cloud cellular network can select from more options of the computing resources that can be reserved at an availability zone compared to a local zone. However, a local zone may provide computing resources nearby geographic locations where an availability zone is not available. Therefore, to provide low latency, certain network components, such as regional data centers, can be implemented at local zones 220 rather than availability zones 215. In some circumstances, a geographic region can have both a local zone and an availability zone.

In the topology of a 5G NR cellular network, 5G core functions of core 139 can logically reside as part of a national data center (NDC). NDC 230 can be understood as having its functionality existing in cloud computing region 210-1 across multiple availability zones 215. At NDC 230, various network functions, such as NFs 232, are executed. For illustrative purposes, each NF, whether at NDC 230 or elsewhere located, can be comprised of multiple subcomponents, referred to as pods (e.g., pod 211) that are each executed as a separate process by the cloud computing environment. The illustrated number of pods is merely an example; fewer or greater numbers of pods may be part of the respective 5G core functions. It should be understood that in a real-world implementation, a cellular network core, whether for 5G or some other standard, can include many more network functions. By distributing NFs 232 across availability zones, load-balancing, redundancy, and fail-over can be achieved. In local zones 220, multiple regional data centers 240 can be logically present. Each of regional data centers 240 may execute 5G core functions for a different geographic region or group of RAN components. As an example, 5G core components that can be executed within an RDC, such as RDC 240-1, may be: UPFs 250, SMFs 260, and AMFs 270. While instances of UPFs 250 and SMFs 260 may be executed in local zones 220, SMFs 260 may be executed across multiple local zones 220 for redundancy, processing load-balancing, and fail-over.

FIG. 3 illustrates an embodiment of a system 300 for providing network connectivity and integrated testing capabilities for entity local networks (“ELNs”). System 300 can include: cellular network 120 (such as detailed in relation to FIGS. 1A-2); UEs 320; ELNs 310; service provider systems 330; Internet 340; test system 350; and service systems 360. UE 320 represent embodiments of UE 110 of FIG. 1 which can additionally communicate with ELNs. Examples include smartphones, cellular modems, tablet computers, cellular-enabled computerized devices, sensor devices, factory equipment, security equipment, cameras, Internet of Things (IoT) devices, manufacturing robots; unmanned aerial (or land-based) vehicles, network-connected vehicles, gaming devices, access points (APs), or any computerized device capable of communicating via a cellular network and a wired or wireless local area network that uses a different communication protocol from the cellular network. Therefore, such UE have at least two wireless communication interfaces available for use. UE can also represent any type of device that has incorporated a cellular network interface, such as a 5G modem.

At a high level, cellular network 120, via its RAN, has a large geographic footprint allowing devices over the geographic footprint to connect with and communicate with Internet 340 (and other public and private networks) via cellular network 120. In a much smaller geographic region, the UE has permission to communicate with an ELN, such as ELN 310-1. An ELN is a wired or wireless local area network (“LAN”), such as a WiFi-based wireless LAN (or some other IEEE 802.11 or 802.15 based communication protocol standard network) or an ethernet based wired LAN. When within range of its ELN, the ELN may be preferably used for communication, especially with services hosted by the ELN. However, as detailed below, in certain circumstances, connectivity with cellular network 120 and an ELN can be simultaneously leveraged, such as for testing purposes.

System 300 can allow for a cellular network, such as cellular network 120 detailed in relation to FIGS. 1A and 1B, to function as a “network of networks.” By having an overarching network that allows for communication with the UE of multiple ELNs, various advantages can be realized. A first advantage can be connectivity. When a UE, such as UE 320-1, is outside the range of its ELN (e.g., ELN 310-1) and communication link 324-1 is unavailable, connectivity via communication link 322-2, such as with the Internet 340 and service provider systems 330, can be realized via cellular network 120. Further, by being directly connected with a larger wireless network, UE of an entity can potentially communicate more directly with other service providers (e.g., service provider systems 330), other ELNs, and other UE.

A second advantage can be for testing one or more services hosted by a UE's corresponding ELN, the ELN, and/or the UE itself. The cellular network can provide and coordinate testing to test an entity's UE, network, and services. By having UE connect directly with the cellular network via the cellular network's RAN, the ELN, the entity's UE, and hosted services can be tested in various respects that would otherwise be difficult to perform or at least would require an investment of significant resources (e.g., time, specialized software) by the entity responsible for the ELN.

ELNs 310 represent private wired or wireless networks that are operated by an entity for the entity's devices. Each of ELNs 310 can be located at different locations 380. A possible use case would be a wireless network that communicates with machinery on a factory floor. Another possible use case would be a network that communicates with gambling equipment in a casino. Still another possible use case could be a vehicle (e.g., race car) exchanging operating and location data with a management system. ELNs 310 may operate using various protocols, such as a wireless protocol from the IEEE 802.11 family, WiFi, Zigbee, Thread, Bluetooth, Bluetooth LE, Ethernet, other wired protocols, etc. ELNs 310 may allow for that entity's UE to communicate with other devices on the entity network, services hosted by the ELN, and, possibly, the Internet. For example, UE 320-1, 320-2, and 320-3 can communicate with other devices and systems connected with ELN 310-1 and possibly communicate with other systems and devices via Internet 340. Similarly, UE 320-4, 320-5, and 30-6 communicate with ELN 310-2 and UEs 320-7 and 320-8 communicate with ELN 310-3 to perform similar functions. Each of ELNs 310 may be operated by wholly distinct entities that have no relationship with each other. Rather, the only link between the entities which operate separate ELNs 310 may be that UE of each entity can use cellular network 120 for network access.

In addition to communicating with ELNs 310, UE 320 may have an on-board cellular network interface (e.g., a 5G NR modem) or may be connected with an external cellular network interface or AP that allows for the UE to communicate directly with cellular network 120. To be clear, to communicate “directly” with the cellular network means the UE or the AP with which the UE is connected does not need to communicate via ELN 310 (or some other network) to communicate with cellular network 120.

ELNs 310 can also provide access to various service systems 360 and service provider systems 330. Service systems 360 can represent computer server systems that execute software-based processes or applications that are directly connected with ELNs 310. For example, service system 360-1 can be a server system that provides an application service to UE 320-1, 320-2, and 320-3 on ELN 310-1. Service systems 360 are local to their respective ELNs or may be accessible via the respective ELN and Internet 340.

Service provider systems 330 represent software-based services that are provided by third parties for an entity operating one of ELNs 310. Service provider systems 330 can represent cloud-hosted services or services hosted by a discrete server. For example, a UE, such as UE 320-4, may access service provider system 330-1 via ELN 310-2 to access a particular service. In some circumstances, there may be benefits to accessing a service provider system by a UE via cellular network 120 rather than via the corresponding ELN. Such benefits can include: security, increased uplink or downlink bandwidth, decreased jitter, decreased packet loss, and/or reduction in latency.

Regarding connectivity, UE 320 may use their respective ELN of ELNs 310 when a connection is available. When no connection with the corresponding ELN is available (e.g., out of range, the ELN is offline, the ELN is overloaded), UE can obtain network access to Internet 340 and/or to the associated ELN via cellular network 120. UE may communicate with cellular network 120 via the RAN detailed in relation to base stations 115, RUs 125, DUs 127, CU 129, and 5G core 139 detailed in relation to FIGS. 1A and 1B.

Each UE of ELNs 310 can be assigned to various cellular network slices depending on the level of service needed for the UE while using the cellular network. Assigning a UE to a particular slice to achieve a particular level of service may be significantly more efficient than attempting to configure the corresponding ELN to provide a similar level of service. In some embodiments, a slice of cellular network 120 may be designated for the UE of a particular entity, such as UE 320-1, 320-2, and 320-3. A slice may have defined parameters that satisfy one or more quality of service (QoS) and/or quality of experience (QoE) metrics that have been set by the entity operating an ELN via a service level agreement (SLA) entered between the entity and the operator of the cellular network. As such, the entity may define an SLA that provides UE with cellular services that allow the UE to function at an acceptable performance level when communicating with the cellular network. As an example, UE 320-1 may be particularly vital or may need a particularly high QoS to function properly with other systems for the entity operating ELN 310-1. Therefore, UE 320-1 may be assigned to a slice of cellular network 120 that provides a better QoS than a slice to which UE 320-2 is assigned and used for communication (represented by communication link 322-2).

In addition or in alternative to providing connectivity to Internet 340 and/or ELNs 310, cellular network 120 can be used as a testing platform to test UE 320, ELNs 310, systems and/or service systems 360 connected with entity networks, and communication between UE, entity networks, services, and service provider systems 330. More specifically, test system 350 can be implemented on cellular network 120. Embodiments of test system 350 are detailed in relation to FIG. 4. While FIG. 3 shows only a few UE and up to two service systems per ELN, it should be understood that much greater numbers of UE and services may be present per ELN. Further, the number of ELN for which its UE can communicate with the cellular network may be much greater than three.

FIG. 4 illustrates an embodiment 400 of a test system that can operate as part of the cellular network of FIG. 3. In the illustrated embodiment of FIG. 4, test system 350 is implemented on a public cloud computing platform along with the cellular network core detailed in relation to FIG. 2. As illustrated, test system 350 is implemented in a particular cloud computing region 210-1 of cloud computing platform 201. In other embodiments, rather than implementing on a public cloud computing platform, one or more dedicated server systems of the cellular network operator can be used. Test system 350 can include: test coordinator system 450; baselining engine 460; test monitor 470; and result datastore 480. Each of these components may be executed in the form of one or more pieces of special-purpose software executed by underlying general-purpose hardware. Alternatively, specialized hardware may be used that is hardcoded to perform the functions detailed in relation to the components.

Baselining engine 460 may be used to measure the performance of an ELN, the UE on the ELN, the services provided on the ELN by one or more service systems, and the services provided by service provider systems executed outside of the ELN (e.g., in the cloud). Baselining coordinated by baselining engine 460 can involve accessing UE on an ELN and services systems on the same ELN (or service provider system accessible via the ELN and Internet) to determine for when the ELN is used for communication: 1) latency and/or jitter between individual UE and the service based on timestamped communications; 2) packet loss between individual UE and the service; and 3) uplink and downlink bandwidth between the service system and the UE. Baselining engine 460, by communicating directly with a UE via the cellular network (or via the ELN) can instruct the UE to send one or more messages to a particular service performed by a service system. Baselining engine 460 can then access the service system to obtain information on receipt of the messages, processing time of the messages, and/or timestamps in response to the message. This message or group of messages can be used to assess latency, jitter, packet loss, uplink/downlink bandwidth, etc. Data about such baselining can be stored to result datastore 480 for use in future comparisons. Similar testing can be repeated by baselining engine 460 using the cellular network instead of the ELN for communication between the UE and the service system (and/or service provider system). This data may also be stored to result datastore 480 for use in future comparisons.

As part of test system 350, a test coordinator system is present to coordinate what one or more tests are to be performed on an entity's UE, ELN, and/or services. Test coordinator system 450 can include multiple components, such as: testing tools 452, test wrapper 454, environment controller 456, and test profiles 458. In concert, these components may be used to coordinate and perform various forms of test. Generally, the forms of testing can be divided up to several categories: 1) test of the functions of UE; 3) tests in which data is sent to the UE to, in turn, be sent by the UE to a service of the ELN for processing (e.g., to test latency, jitter, bandwidth, and packet loss in various conditions); 3) test communications among UE via the ELN; 4) tests in which data is sent to the ELN and/or service system to in turn be sent to UE (e.g., to test latency, jitter, bandwidth, and packet loss in various conditions); and 5) tests in which data is to be sent between UE and remote service provider systems. Within each of these categories, variables can be introduced to see how other components and services will respond, such as intentionally introducing latency, jitter, packet loss, decreased bandwidth, loss of computing resources (e.g., insufficient memory, long processing times), etc. These tests can then be repeated wherein the UE communicate using the cellular network instead of the ELN. These tests can also be performed using different cellular network slices to observe performance differences that can be attributed to differences in QoS among slices.

Test coordinator system 450 may be able to perform sets of standardized, customized, and randomized (e.g., chaotic) tests on an entity's network (e.g., ELN 310-1) and the entity's UE (e.g., UE 320-1, UE 320-2, and UE 320-3). Testing tools 352 can represent the software packages used to generate various tests to be performed involving the UE and entity networks. Test wrapper 454 can function as an intermediary between testing tools 452, the UE, and the entity network in order to allow for compatibility between testing tools 452 and the UE and entity network. Environment controller 456 can allow for various aspects of the software environment to be varied, such as parameters involving operating systems, database systems, or specifics of a compiler. For example, a particular test may involve simulating particular limitations or known faults for the operating system operated on the UE and/or on services that use the entity's network. Test profiles 458 can represent a defined test bank of tests that can be executed as-is or used as a template to create a customized test. For example, a defined group of tests may be recommended to be executed on all of an entity's UE, the entity's network, and services connected with the entity's network. The group of tests may be selected based on: the type of UE used; and/or the type of services executed on the ELN. As an example, the types of tests that are run for sensor devices (e.g., focused on packet loss) can differ substantially from the types of tests run for video streaming devices (e.g., focused on bandwidth).

For example, an entity may be interested in testing how a UE will function when its ability to connect with a particular server system has a defined amount of latency. To simulate such a test, the cellular network could be used instead of the ELN. A cellular network slice could be defined via test coordinator system 450 that ensures that the defined amount of latency will be present when the UE uses cellular network 120 to connect with the defined particular server system. Alternatively, the ELN could continue to be used, but the UE could be configured to delay an amount of time before sending communications to simulate the latency. The performance of the UE and other devices that rely on data from the UE can then be monitored. Further, by communicating directly with the UE, test coordinator system 450 can determine whether individual UE respond appropriately to various “chaotic” arrangements. For example, test coordinator system 450 can simulate various negative conditions, which may occur on an entity network, such as unavailability of a server system, decreased availability of bandwidth, an intermittent condition, one or more software-based services going offline, the inability to communicate with a third-party service provider system, etc.

As another example, test coordinator system 450 may be able to test/simulate how an entity's ELN and connected services perform when a large spike in data is received from the entity's UE. Test coordinator system 450 may be able to create simulated traffic (e.g., based on real data from the entity's UE) to test how the entity's network, and/or services connected with the entity's network, will respond. To perform such a test, the test coordinator system may configure the UE to transmit an increased amount of traffic on the ELN or the cellular network may simulate such traffic and send directly to one or more service systems operating on the ELN. In some embodiments, cellular network 120 may be able to send simulated traffic to UEs 320, which is then relayed to the appropriate entity network, such as for load testing.

The tests stored in test profiles 458 may have been developed for specific types of UE, specific services, specific service provider systems, and specific types of ELNs. Therefore, only some of the tests performed on one ELN may be appropriate for another ELN. The specific tests performed from test profiles 458 may be selected by the entity that desires testing. These tests can involve the UE communicating with services hosted by the ELN via the cellular network. By using the cellular network to communicate with the services, test coordinator system 450 can apply the parameters of the test (e.g., simulated increased latency, simulated decreased bandwidth). Alternatively, the cellular network can configure the UE to simulate these parameters and allow the UE to communicate with the service system via the ELN. The service on the ELN can be similarly configured or caused to communicate via the cellular network to simulate a test parameter to see the effect on other services and the UE.

Test coordinator system 450 can set up and conduct a test involving an ELN, its services, its external services, and its UE. Test monitor 470 may gather the results of the test and store the results to result datastore 480. Result datastore 480 may therefore have the results of baselining and the tests that varied particular metrics, such as latency, bandwidth, jitter, packet loss, use of different network slices, etc. Examples of data that may be reflected in result datastore 480 can include: a comparison of latency in communication when ELN traffic is elevated for a UE to communicate with another UE, a service system on the ELN, and/or a service provider system along with latency values for when the cellular network is used for such communication (e.g., via one or more slices) for such communication; and similar comparisons for bandwidth, jitter, and packet loss. In further comparisons, instead of elevated ELN traffic, other suboptimal occurrences may be simulated, such as: a service system having limited memory or limited processing capability, loss of connection between one or more UE or services with the ELN, the ELN experiencing high packet loss, available uplink and/or downlink bandwidth with the ELN being decreased, etc.

Based on the results of the tests, the functionality of an ELN or how one or more UE use an ELN may be adjusted by an entity. For example, the entity may elect to have particular UE communicate using cellular network 120 as the default communication path (even when the ELN is within range) and the ELN used as a backup. For particular tasks being performed by a UE, the cellular network may be used as a preferred communication path (e.g., low latency communication with service provider system 330-1 is required). Components of the ELN may be upgraded or reconfigured to improve performance. One or more UE may be assigned to a particular slice of the cellular network to realize particular QoS.

Tests that are designed and used for a particular ELN may be redeployed by the operator of the cellular network to test another ELN for a wholly unrelated entity. Such an arrangement allows the operator of the cellular network to leverage testing experience from one ELN to another ELN for a wholly unrelated entity. As an example, tests within test profiles 458 may be grouped based on: type of UE; type of business; types of services on ELN; types of external service provider systems used; number of UE; etc.

Various methods may be performed using the systems detailed in relation to FIG. 1A-4. FIG. 5 illustrates an embodiment of a method 500 for using the integrated testing capabilities of the cellular network to test an entity's local network system. Method 500 may be performed using test system 350, which can be implemented as part of cellular network 120 of system 300.

At block 510, an entity allows access for a cellular network test system to communicate with its ELN. For example, the ELN may be a local wireless or wired network used by the entity at a location of the entity, such as a factory, warehouse, etc. The cellular network test system may communicate with services and UE on the ELN of the entity via direct communication using the cellular network's RAN and/or via the Internet.

At block 520, a baselining performance analysis can be performed to understand how the ELN, its services, its external services, and UE perform under normal operating circumstances. Baselining can involve measuring latency, packet loss, bandwidth, and jitter between UE, UE and services, UE and external services, etc. Data obtained via baselining by the cellular network can be stored to a result datastore for later use in comparison with data obtained during tests.

At block 530, some number of predefined tests are selected by the entity and/or cellular network operator. Tests can be selected based on specific aspects of the ELN that the entity desires to test, the type of UE; the type of business of the entity; the types of services on the ELN (as service systems); the types of external service provider systems used; the number of UE; etc. Tests may also be randomized. That is, a chaotic test may simulate a randomized failure or extraordinary event involving the ELN to see how the ELN, UE, and service react.

At block 540, The cellular network, via the RAN of the cellular network, may communicate with the UE operating on the ELN to configure the UE as needed to perform a test. This can involve instructing the UE to use the cellular network, rather than the ELN, to perform communication with each other and services. Alternatively, the cellular network can give the UE configuration information that will be used as part of the test conducted locally on the ELN. For example, the UE may be configured to intentionally have a defined amount of latency when responding to requests.

At block 550, the testing is performed via the ELN or via the cellular network. If via the cellular network, the cellular network test system can cause intentional issues to see how the ELN and its components respond. For example, if UE are caused to communicate via the cellular network with a service resident on the ELN, the cellular network can intentionally introduce latency, packet loss, jitter, reduction in bandwidth, etc. to see how the service and other components on the ELN respond.

At block 560, the test results are stored, such as to a results datastore where baselining data was previously stored.

At block 570, if an additional test is to be performed, method 500 can return to block 540 and further the cellular network and/or UE may be reconfigured for the next test to be performed. For example, if a first test is focused on a defined amount of latency being introduced to communication involving UE, a second test may further increase the amount of latency in an attempt to determine a maximum amount of latency that can be handled before the system fails to perform to specification.

At block 580, if no additional tests are to be performed at block 570, the results of the tests may be analyzed in combination with the baseline data to determine how the ELN, UE, the services on the ELN, external services, or the cellular network should be configured. As previously detailed, for high-priority communication, high bandwidth communication, or low-latency communication, the cellular network may be given preference over the ELN in some or all circumstances for some or all UE. A particular slice of the cellular network may be selected based on the needed QoS.

It should be noted that the methods, systems, and devices discussed above are intended merely to be examples. It must be stressed that various embodiments may omit, substitute, or add various procedures or components as appropriate. For instance, it should be appreciated that, in alternative embodiments, the methods may be performed in an order different from that described, and that various steps may be added, omitted, or combined. Also, features described with respect to certain embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments may be combined in a similar manner. Also, it should be emphasized that technology evolves and, thus, many of the elements are examples and should not be interpreted to limit the scope of the invention.

Specific details are given in the description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, well-known, processes, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the embodiments. This description provides example embodiments only, and is not intended to limit the scope, applicability, or configuration of the invention. Rather, the preceding description of the embodiments will provide those skilled in the art with an enabling description for implementing embodiments of the invention. Various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the invention.

Also, it is noted that the embodiments may be described as a process which is depicted as a flow diagram or block diagram. Although each may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included in the figure.

Having described several embodiments, it will be recognized by those of skill in the art that various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the invention. For example, the above elements may merely be a component of a larger system, wherein other rules may take precedence over or otherwise modify the application of the invention. Also, a number of steps may be undertaken before, during, or after the above elements are considered. Accordingly, the above description should not be taken as limiting the scope of the invention.

CELLULAR NETWORK TEST ENVIRONMENTS FOR EXTERNAL NETWORKS

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

International Classifications

Abstract

Description

Claims

CROSS REFERENCE TO RELATED APPLICATIONS

Provisional Applications (1)