Patent Application
20190059008
In recent years, telecommunication devices have advanced from offering simple voice calling services within wireless communication networks to providing users with many new features. Telecommunication devices now provide messaging services such as email, text messaging, and instant messaging; data services such as Internet browsing; media services such as storing and playing a library of favorite songs; location services; and many others. Thus, telecommunication devices, referred to herein as user devices or mobile devices, are often used in multiple contexts. In addition to the new features provided by the telecommunication devices, users of such telecommunication devices have greatly increased. Such an increase in users is only expected to continue and in fact, it is expected that there could be a growth rate of twenty times more users in the next few years alone.
Wireless communication networks are generally made up of multiple nodes, links, subnetworks, etc. Services, e.g., telephone calls, data transmission, etc., provided to users of the wireless communication network travel between the various nodes and over various links, other nodes, subnetworks, etc. When faults occur within the wireless communication network, it can be difficult to ascertain what is causing the fault. For example, it can be difficult to ascertain if it is a link, a node, a subnetwork, etc., causing the problem. This difficulty can result in delays in fixing the fault, thereby reducing the experience and satisfaction of users of services within the wireless communication network. Such a delay in fixing the fault can also result in wasted resources in attempting to ascertain and fix the fault, as well as wasting resources of users attempting to utilize services within the wireless communication network.
The detailed description is set forth with reference to the accompanying figures, in which the left-most digit of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.
Described herein are techniques and architecture that allow for performance measuring and monitoring of a wireless communication network and developing a prediction model for predicting causes of faults within the wireless communication network. Such techniques allow for gathering of key performance indicator (KPI) performance measurements between points within the wireless communication network. The performance measurements can include evaluating nodes, links, subnetworks, etc., within the wireless communication network. Based upon the performance measurements and historical data, a prediction model can be developed that can be used to predict a likely cause of a future fault within the wireless communication network. Thus, the determination and correction of faults within the wireless communication network can be improved and handled in a more efficient and timely manner. This can save resources within the wireless communication network, e.g., processor time, engineer/technician time, etc., as well as resources of users of the wireless communication network attempting to obtain services within the wireless communication network.
In configurations, point-to-point and point-to-multiple point KPI performance measurements and monitoring among various nodes can be performed within a wireless communication network. The wireless communication network may include various nodes, including, for example, business and engineering functional nodes, including a core network, transport, radio network, small cell nodes, data centers, call centers, regional business offices, retail stores, etc. Performance measurement data may be gathered and correlations among various point-to-point and point-to-multiple point routes within the wireless communication network may be determined.
A prediction model based upon the performance measurement data correlations may be determined. The prediction model may then be verified utilizing historical fault data based upon network root cause fix history, e.g., the history of determining the root cause of faults and fixing the faults within the wireless communication network. In verifying the prediction model, an accuracy may be determined based upon historical performance measurement data and network root cause fix history. In configurations, if the accuracy exceeds a predetermined threshold, then the prediction model may be utilized to predict potential causes of faults within the wireless communication network to thereby increase efficiency and speed of addressing faults within the wireless communication network.
More particularly, in configurations, Ethernet virtual circuits (EVCs) between a mobile switch office (MSO) and a cellular cell site may be measured for various KPI performance measurements including, for example, delay, jitter and frame loss ratio. Bandwidth utilization data from cellular site routers can also be gathered. By considering different locations of cellular sites and some cellular sites proximity, performance measurement data may help identify network performance in vendor core networks or EDGE networks since proximity sites generally share the same EDGE network pipe. This can help determine which vendor services are best by comparing performance measurement data during the same period. The performance measurement data can also be utilized in evaluating vendors that provide network services such as multiple class of service (COS). The performance measurement data can be utilized to determine which vendors to utilize in the wireless communication network. As is known, EDGE generally refers to “enhanced data rates for GSM evolution.” An EDGE device is generally referred to a device that provides an entry point into enterprise or service provider core networks. Examples include, for example, routers, routing switches, integrated access devices (IADs), multiplexors and a variety of metropolitan area network (MAN) and wide area network (WAN) access devices. EDGE devices also provide connections into carrier and service provider networks.
Based on historical performance measurement data and outage (fault) events, a prediction model that uses historical data to train the prediction model with KPI performance measurement data to identify how the faults occurred can be developed. Partial performance measurement data may be used as test data to verify the prediction model. Then with the verified model, the prediction model can be used to forecast the probability of a cause for a fault or outage in the core or transport network.
Access points such as, for example, cellular towers 122, can be utilized to provide access to the wireless communication network 100 for mobile devices 102. In configurations, the wireless communication network 100 may represent a regional or subnetwork of an overall larger wireless communication network. Thus, a larger wireless communication network may be made up of multiple networks similar to wireless communication network 100 and thus, the nodes and networks illustrated in
In configurations, the mobile devices 102 may comprise any appropriate devices for communicating over a wireless communication network. Such devices include mobile telephones, cellular telephones, mobile computers, Personal Digital Assistants (PDAs), radio frequency devices, handheld computers, laptop computers, tablet computers, palmtops, pagers, as well as desktop computers, devices configured as Internet of Things (IoT) devices, integrated devices combining one or more of the preceding devices, and/or the like. As such, the mobile devices 102 may range widely in terms of capabilities and features. For example, one of the mobile devices 102 may have a numeric keypad, a capability to display only a few lines of text and be configured to interoperate with only GSM networks. However, another of the mobile devices 102 (e.g., a smart phone) may have a touch-sensitive screen, a stylus, an embedded GPS receiver, and a relatively high-resolution display, and be configured to interoperate with multiple types of networks. The mobile devices may also include SIM-less devices (i.e., mobile devices that do not contain a functional subscriber identity module (“SIM”)), roaming mobile devices (i.e., mobile devices operating outside of their home access networks), and/or mobile software applications.
In configurations, the wireless communication network 100 may be configured as one of many types of networks and thus may communicate with the mobile devices 102 using one or more standards, including but not limited to GSM, Time Division Multiple Access (TDMA), Universal Mobile Telecommunications System (UMTS), Evolution-Data Optimized (EVDO), Long Term Evolution (LTE), Generic Access Network (GAN), Unlicensed Mobile Access (UMA), Code Division Multiple Access (CDMA) protocols (including IS-95, IS-2000, and IS-856 protocols), Advanced LTE or LTE+, Orthogonal Frequency Division Multiple Access (OFDM), General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Advanced Mobile Phone System (AMPS), WiMAX protocols (including IEEE 802.16e-2005 and IEEE 802.16m protocols), High Speed Packet Access (HSPA), (including High Speed Downlink Packet Access (HSDPA) and High Speed Uplink Packet Access (HSUPA)), Ultra Mobile Broadband (UMB), and/or the like. In embodiments, as previously noted, the wireless communication network 100 may be include an IMS 100a and thus, may provide various services such as, for example, voice over long term evolution (VoLTE) service, video over long term evolution (ViLTE) service, rich communication services (RCS) and/or web real time communication (Web RTC).
In configurations, network devices work as maintenance entity points (MEP) 124 and support PM protocols such as, for example, the TWAMP protocol for testing among various nodes and/or networks of the wireless communication network 100. The testing can involve server-to-client PM or peer-to-peer PM models. A PM server 126 may be included that implements alternate access vendor (AAV) PMs for the mobile backhaul 116, PMs from the data center 114 to the call center(s) 112, PMs from the data center 114 to retail stores 106, etc., as illustrated in
As PM data is gathered based on the TWAMP tests (or other tests), the PM data can be correlated and analyzed. For each PM path, it is assumed that there are KPI metrics defined. If the PM data is within a predefined KPI range, then the performance is regarded as good. Otherwise, the performance is regarded as bad. For example, for AAV mobile backhaul, the KPI matrix may be defined as a frame delay having less than 16 milliseconds (roundtrip); jitter less than four milliseconds (roundtrip); frame loss rate less than 1.0E-6; and service availability 99.99 percent.
Referring to
Referring to Table 3, if a first PM path 402, a second PM path 404 and a third PM path 406 are all good, then Network F is good. If PM path 402 and PM path 404 are good, but PM path 406 is bad, then the subnet with AAV provider EDGE device 400 (node E) is good and Network F is at least partially good. If PM path 402 and PM path 406 are good, but PM path 404 is bad, then Network F is good. Node B may be bad or the link between node B and node E may be bad. If PM path 404 and PM path 406 are good but PM path 402 is bad, then Network F is good and node A may be bad or the link between node A and node E may be bad. If PM path 406 is good but PM path 402 and PM path 404 are bad, then Network F is good and AAV provider EDGE device 400 (node E) is bad. If PM path 404 is good but PM path 402 and PM path 406 are bad, then Network F is partially good and the link between node A and node E is bad. If PM path 402 is good but PM path 404 and PM path 406 are bad, then Network F is partially good and the link between node B and node E is bad. If PM oath 402, PM path 404 and PM path 406 3 are all bad, then Network F is bad.
Thus, in accordance with configurations, the various connections illustrated among the various nodes in
In configurations, referring to
At 502, a network topology is created and defined for all PM paths within the wireless communication network 100. At 504, the PM correlation type may be identified for each PM path. For example, two PM paths may correlate based upon a common node, a common link or a common network/subnetwork located “in the middle,” i.e., a shared component along the PM paths.
At 506, a first portion (X %) of historical PM data is randomly chosen as use for modeling and training data. In configurations, the first portion of historical PM data may be chosen in a manner other than random. In a configuration, 60 percent of the historical PM data is randomly chosen. However, in other configurations, the first portion may comprise a range of 60-80 percent of random historical PM data. In configurations, less than 60% of random historical PM data may be chosen. At 508, based upon the modeling and training data, network fault detection metrics are built utilizing the first portion of the historical PM data and the prediction model is created. For example, the fault detection metrics are built based upon faults or failures within the PM data based upon PM tests along the PM paths as described with respect to
At 510, test data is obtained based upon the remaining portion (1−X %) of the historical PM data to test the prediction model. Thus, if the first portion of the randomly chosen historical data was 60 percent, then the second portion of the randomly chosen historical PM data is 40 percent. In configurations, the second portion of historical PM data may be chosen in a manner other than random. Thus, in configurations, the second portion of the randomly chosen historical data may be in a range of 40-20 percent based upon the amount of the first portion of randomly chosen historical PM data. In configurations, more than 40% of random historical PM data may be chosen. At 512, root cause history data, e.g., history data with respect to the actual root cause and fixes of faults within the wireless communication network is obtained and paired with the test data.
At 514, the prediction model can then be verified using the second portion of randomly chosen historical PM data and the root cause history data. For example, based upon the test data, the prediction model may be utilized to predict the causes of faults within the test data, e.g., the second portion of the historical PM data. Then the root cause history data can be evaluated in order to determine how accurately the prediction model predicted the actual root causes of faults within the test data. For example, if the prediction model predicted that a fault between node A and node B was due to node C on Aug. 1, 2016, then the root cause history can be used to verify that indeed node C caused the fault between node A and Node B. As will be discussed herein, an accuracy of the prediction model may be calculated.
Thus, at 516, performance metrics of the prediction model can be calculated based upon how the prediction model performed with the test data reference to the root cause history. At 518, if the accuracy of the prediction model, based upon the performance metrics, is greater than a predetermined threshold, e.g., 80 percent, 85 percent, 90 percent, etc., then the prediction model is accepted at 520. If not, then the prediction model may be rejected at 522 and the PM data may need to be reanalyzed and reevaluated, or new PM data may need to be obtained.
Thus, when future faults occur within the wireless communication network 100, the prediction model may be used to predict the likely potential causes of the faults. In configurations, when using the prediction model, data may be obtained based upon predictions using the prediction model based upon PM paths and correlations, and then comparing the predictions with the actual root cause of the faults. This data may then be utilized to update the prediction model to thereby allow the prediction model to continue to learn and evolve.
In various implementations, system memory 702 is volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. In some implementations, the processor(s) 704 is a central processing unit (CPU), a graphics processing unit (GPU), or both CPU and GPU, or any other sort of processing unit. System memory 702 may also include applications 716 that allow the server to perform various functions.
The server 700 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in
Non-transitory computer-readable media may include volatile and nonvolatile, removable and non-removable tangible, physical media implemented in technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. System memory 702, removable storage 706 and non-removable storage 708 are all examples of non-transitory computer-readable media. Non-transitory computer-readable media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other tangible, physical medium which can be used to store the desired information and which can be accessed by the server 700. Any such non-transitory computer-readable media may be part of the server 700.
In some implementations, the transceivers 710 include any sort of transceivers known in the art. For example, the transceivers 710 may include wired communication components, such as an Ethernet port, for communicating with other networked devices. Also or instead, the transceivers 710 may include wireless modem(s) to may facilitate wireless connectivity with other computing devices. Further, the transceivers 710 may include a radio transceiver that performs the function of transmitting and receiving radio frequency communications via an antenna.
In some implementations, the output devices 712 include any sort of output devices known in the art, such as a display (e.g., a liquid crystal display), speakers, a vibrating mechanism, or a tactile feedback mechanism. Output devices 712 also include ports for one or more peripheral devices, such as headphones, peripheral speakers, or a peripheral display.
In various implementations, input devices 714 include any sort of input devices known in the art. For example, input devices 714 may include a camera, a microphone, a keyboard/keypad, or a touch-sensitive display. A keyboard/keypad may be a push button numeric dialing pad (such as on a typical telecommunication device), a multi-key keyboard (such as a conventional QWERTY keyboard), or one or more other types of keys or buttons, and may also include a joystick-like controller and/or designated navigation buttons, or the like.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.
