Patent Application
20220413982
Operational data of a computing device provides useful information in analyzing and diagnosing possible causes of incidents that occur on the computing device. The operational data of the computing device may contain event data (e.g., data about hardware; applications; security components; and their daily operations). The operational data may also contain incident data (e.g., data about a degradation in performance; irregular operations; a return to regular operations).
Various examples will be described below referring to the following figures:
As explained above, operational data (e.g., event data, incident data) of a computing device (e.g., laptop, notebook, tablet, smartphone, mobile device, or some other electronic device with an ability to capture data about operations of the device) provides information that is useful in analyzing and diagnosing possible causes of incidents that occur on the computing device. As also explained, event data includes a wide range of information about the computing device and its operations, such as data about hardware, applications, security components, and their daily operations. In one example, the installation of a new peripheral device might generate event data about hardware operations (e.g., new device installed), event data about application operations (e.g., new device driver installed), and event data about security operations (e.g., new device access rights). As mentioned above, the operational data may also contain incident data. An incident is an output of analysis performed on an event or a combination of events. The incident indicates a non-routine change or aberration in functionality of the computing device from a regular state to an irregular state (e.g., a degradation of performance, irregular operations) or from an irregular state to a regular state (e.g., a return to regular operations). For example, event data about battery performance might include statistics on the hour-by-hour operations of the battery and information on what additional hardware components (e.g., mouse, keyboard, graphics processing unit (GPU), central processing unit (CPU)) were utilized during the hour-by-hour operations. After analysis of the information, a computing device may generate an incident about battery degradation or irregular battery operations.
Sorting through the operational data to determine which events are non-routine and may be associated with an incident may be difficult due to the hundreds or thousands of events that occur daily on the computing device. The duration of the timeline may compound the difficulty. For example, if an incident is caused by a cumulative effect of events over a period of days, weeks, or months, then the hundreds or thousands of events are multiplied by the number of days of history that may be examined in a search for events associated with the incident.
This disclosure describes various examples of the creation of an event and incident timeline from filtered operational data. The operational data is filtered to remove event data that is noise (e.g., corrupt, incomplete, or redundant data) or is a usual event. Whether an event is noise or is usual is determined by a parameter or multiple parameters of the filtering. If an event of the event data is not noise or a usual event, the event may be labeled non-routine and stored to an event and incident timeline. To determine if the event is non-routine, the event may be filtered once or multiple times. The number of filters utilized may depend on factors such as a class of events with which the event is associated, an incident of the computing device that occurs during the time period covered by the timeline and that may relate to the event, other events of the computing device, or a result of a previous filtering process. The filter associated with a class of events may perform one round or multiple rounds of filtering on the event. In one example in accordance with the present disclosure, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium stores machine-readable instructions, which, when executed by a processor, cause the processor to: identify an event of a computing device from operational data of the computing device; evaluate the event to determine if the event is a non-routine event; and store the event to a timeline if the event is a non-routine event, where the timeline includes an incident of the computing device.
In another example in accordance with the present disclosure, a system is provided. The system comprises: a network interface; a storage device comprising machine-readable instructions; and a processor coupled to the network interface, the processor to access the storage device. Execution of the machine-readable instructions causes the processor to: collect operational data of a computing device communicatively coupled to the system via the network interface; categorize the operational data as an event; classify the event into a class of events; determine a pattern engine for evaluating the class of events; evaluate the event utilizing the pattern engine to determine if the event is a non-routine event; and store the event to a timeline if the event is a non-routine event, where the timeline includes an incident of the computing device.
In yet another example in accordance with the present disclosure, a method is provided. The method comprises collecting operational data of a computing device; identifying an event in the operational data; classifying the event into a class of events; identifying a machine learning technique for filtering noise within the class of events; utilizing the machine learning technique to determine if the event is noise; and storing the event to a timeline if the event is not noise, where the timeline includes an incident of the computing device.
In some examples, the computer-readable medium 102 may store machine-readable instructions 104, 106, 108. The machine-readable instructions 104, 106, 108 may be machine-readable instructions for execution by the processor 110. The machine-readable instructions 104, 106, 108 may cause the processor to evaluate an event to determine if the event should be stored on an event and incident timeline. (See discussion with respect to
Event data may be collected by a processor (e.g., processor 110) in real-time, daily, at scheduled increments, or on demand from the system 100. Incidents may not occur on a scheduled basis but rather in response to an event or a combination of events. An incident may present in operational data once the condition (e.g., a return to regular operations, a degradation of performance, or operations outside of defined parameters) for the incident is met.
The processor 110 may evaluate an event to determine that the event is not noise and is not a usual event. If the event is not noise or a usual event, then the processor 110 may label the event non-routine and store it to a timeline. As discussed above, the processor 110 may filter the event once or multiple times during the evaluation. A filter may have a single or multiple parameters and may perform one round or multiple rounds of filtering on the event. In some examples, each round of filtering may be considered a level of filtering. In some examples, because of parameters established by a second filter, an event may be considered non-routine despite a first filter indicating the event is noise or a usual event. For example, a first filter may determine that an event about battery performance indicates the battery is operating within parameters, but a second filter may determine that operating within parameters is not noise or a usual event because previous events about battery performance indicate the battery was not operating within parameters. In some examples, an event may be considered non-routine after a first filter but then be classified as noise by a second filter. For example, an event about battery performance may indicate a battery is not operating within parameters, and a first filter may determine the event is non-routine. However, a second filter may classify the event as noise after determining the event is redundant to a non-routine event that is stored on a timeline and indicates the battery is not operating within parameters. (See below with respect to
In various examples, the time period of the non-routine event stored to the timeline may be increased to include the timing of the noisy event. The time period indicates the duration of the non-routine event. For example, the time period of the non-routine event may be expanded from one day to two days. Expanding the duration of the non-routine event, but not storing the noisy event, prevents cluttering the timeline. In some instances, selecting the non-routine event stored to the timeline (e.g., by touching the non-routine event using a touch-sensitive display) may provide a list of any noisy events related to the non-routine event. For example, selecting an event on the timeline presented on a web interface may enable a drop down menu of related noisy events to appear.
In some examples, the storage device 208 may store machine-readable instructions 210, 212, 214, 216, 218, 220. The machine-readable instructions 210, 212, 214, 216, 218, 220 may cause the processor to evaluate an event to determine if the event should be stored on an event and incident timeline. (See discussion with respect to
Execution of the machine-readable instruction 212 may cause the processor 206 to categorize the operational data as an event. Execution of the machine-readable instruction 214 may cause the processor 206 to classify the event into a class of events. Execution of the machine-readable instruction 216 may cause the processor 206 to determine a pattern engine for evaluating the class of events. The pattern engine may be machine-readable instructions (e.g., computer-executable code) implementing a machine learning technique for filtering event data. The filtering may be a function of past data sets. From the past data sets, the pattern engine may establish a parameter or multiple parameters for a class of events to satisfy before an event of the class of events may be classified as non-routine. In some examples, the pattern engine may learn from new data sets and adjust the parameter or multiple parameters.
Execution of the machine-readable instruction 218 may cause the processor 206 to evaluate the event utilizing the pattern engine to determine if the event is a non-routine event. (See discussion below with respect to
In some examples, execution of the machine-readable instruction 212 may cause the processor 206 to determine that the operational data is an event after assimilating and cleaning the operational data. Assimilating and cleaning the operational data may include removing improper values (e.g., data outside actual possible parameters of a component) and partial data (e.g., errors in data collection). The assimilated and cleaned operational data may then be searched for an identifier that indicates an event. In some examples, the identifier may include an event type, a timestamp of the event, and a pseudorandom number.
In some examples, execution of the machine-readable instruction 214 may cause the processor 206 to classify the event into a class of events utilizing a lookup table. (See discussion below with respect to
In some examples, execution of the machine-readable instruction 216 may cause the processor 206 to determine a pattern engine for evaluating the class of events utilizing a lookup table. (See discussion below with respect to
In various examples, a pattern engine may be based on a frequency of occurrence. For example, for one class of events, any occurrence of an event may establish the priority of the event as sufficient to label the event as non-routine (e.g., a single event indicating an incident). For another class of events, an event should occur multiple times within a fixed time window before the priority of the event is established and the event is labeled as non-routine (e.g., an event indicating a degradation in operations of a component is continuing). (See discussion below with regard to
In some examples, a pattern engine may be based on an operational threshold. The operational threshold may be a single parameter threshold or a multi-parameter threshold. For example, for one class of events, if an event fails to satisfy a single threshold, then the parameter to establish a priority of the event as sufficient to label the event non-routine may be met. For another class of events, an event should fail in order to satisfy a parameter for a first threshold as well as a parameter for a second threshold before a priority of the event is established and the event is labeled as non-routine (e.g., an event should fail in order to satisfy a first threshold under a first condition as well as fail to satisfy a second threshold under a second condition). For yet another class, an event should fail in order to satisfy a single threshold multiple times before a priority of the event is established and the event is labeled as non-routine (e.g., to ensure an event does not contain corrupt, incomplete, or inaccurate data). (See discussion below with regard to
In various examples, a pattern engine may be based on a historic trend. The historic trend may indicate whether an event has occurred in the past, has not occurred in the past, or has not occurred within a specific time period. The historic trend may indicate whether an event occurred or did not occur during multiple time periods. For example, for one class of events, if an event has happened multiple times in the past, then the parameter to establish a priority of the event and label the event as non-routine may not be met (e.g., the event may be classified as noise). In another class of events, if an event has never happened in the past, then the parameter to establish a priority of the event and label the event as non-routine may be met. In yet another class of events, if an event does not occur during a specific time period, then the parameter to establish a priority of the event and label the event as non-routine may be met (e.g., if an event has not occurred during a previous two-month time period, then the non-occurrence of the event may indicate a recovery incident (e.g., recovery from a past incident)). (See discussion below with regard to
Turning ahead for a moment to
In some examples, the computer-readable medium 702 may comprise machine-readable instructions 704, 706, 708, 710, 712, 714, 716. The machine-readable instructions 704, 706, 708, 710, 712, 714, 716 may be machine-readable instructions for execution by the processor 718. Execution of machine-readable instruction 704 may cause the processor 718 to determine a class of events from multiple classes of events for every event of operational data of a computing device (e.g., computing device 202). In some examples, the processor 718 may utilize a lookup table to determine a class of events. (See discussion below with regard to
In some examples, if the processor 718 determines a “yes” response to instruction 708 (e.g., a first filter indicates the event is non-routine), then the processor 718 may execute machine-readable instruction 714 (e.g., utilize a second filter). If the processor 718 determines a “yes” response to machine-readable instruction 714 (e.g., the event is non-routine compared to events of a timeline), then the processor, 718 may execute machine-readable instruction 716 and store the non-routine event to the timeline. If the processor 718 determines a “no” response to machine-readable instruction 714 (e.g., a second filter indicates the event is not a non-routine event), then the processor 718 may execute machine-readable instruction 712 and classify the event as noise. If the processor 718 determines a “no” response to machine-readable instruction 708 (e.g., the event is not a non-routine event based on the pattern engine for the class of events), then the processor 718 may execute machine-readable instruction 710 (e.g., utilize a second filter). If the processor 718 determines a “yes” response to machine-readable instruction 710 (e.g., the event is non-routine based on an incident of a timeline), then the processor 718 may execute machine-readable instruction 714 (e.g., utilize a third filter). If the processor 718 determines a “no” response to machine-readable instruction 710 (e.g., the event is not non-routine based on an incident of a timeline), then the processor 718 may execute machine-readable instruction 712 and classify the event as noise.
In some examples, the processor 718 may use a single filter. For example, if the processor 718 determines a “yes” response to machine-readable instruction 708, then the processor 718 may skip machine-readable instruction 714 and execute machine-readable instruction 716 (e.g., store the non-routine event to a timeline). In another example, if the processor 718 determines a “no” response to machine-readable instruction 708, then the processor 718 may skip machine-readable instruction 710 and execute machine-readable instruction 712 (e.g., classify the event as noise). In some examples, the processor 718 may decide to utilize a second filter, but not a third filter. For example, if the processor 718 determines a “yes” response to machine-readable instruction 710, then the processor 718 may skip machine-readable instruction 714 and execute machine-readable instruction 716 (e.g., store the non-routine event to the timeline).
Turning now to
In various examples, the method 300 may be performed by the processor 110, 206, 718 of the system 100, 200, 700. For example, machine-readable instruction 302 may be performed by the processor 110, 206, 718 via a network interface (e.g., network interface 204). Collection of the operational data may occur in real-time, daily, at scheduled increments, or on demand by the system 100, 200, 700. In some examples, machine-readable instruction 304 may be performed by the processor 110, 206, 718 by assimilating and cleaning the operational data and then searching the assimilated and cleaned operational data for an identifier that indicates an event. In some examples, machine-readable instruction 306 may be performed by the processor 110, 206, 718 by utilizing a lookup table. (See discussion below with respect to
In some examples, the descriptors of the event column of the lookup table 400 may indicate a component type. For example, “AppXX” may denote an event associated with machine-readable instructions of an application named “AppXX;” and “OS” may associate to an operating system. Additional description may be provided. For example, for an application, an event may be further described by an error code of the application. The descriptors for the class of events may indicate a type of component—hardware (e.g., HW), machine-readable instructions (e.g., SW), or security. Additionally, a subclass may be indicated—configuration (e.g., Config), operating system (e.g., OS), virus, battery, or error.
For example, in the lookup table 400, a “New Device Installed” event is classified to a “HW Config” class of events, which utilizes a “Frequency of Occurrence” pattern engine with a prioritization parameter of “Prioritize if happens once.” A “Battery Health” event is a first event type classified to a “HW Battery” class of events, which utilizes an “Operational Threshold” pattern engine with a prioritization parameter of “Prioritize if exceeds threshold” and a “Historic Trend” pattern engine with a prioritization parameter of “Prioritize if performance degrading over time.” An “AppXX_Error” event is classified to a “SW Error” class of events, which utilizes a “Historic Trend” pattern engine with a first prioritization parameter of “Prioritize if third occurrence” and a second prioritization parameter of “Prioritize if no repeat occurrence for two months.” A “Virus Detected” error is classified to a “Security_Virus” class of events, which utilizes a “Frequency of Occurrence” pattern engine with a prioritization parameter of “Prioritize if happens once.” An “OS Crash” event is associated with a “SW OS” class of events, which utilizes a “Frequency of Occurrence” pattern engine with a prioritization parameter of “Prioritize if happens once.” A “Battery Performance” event is a second event type classified to a “HW_Battery” class of events, which utilizes an “Operational Threshold” pattern engine with a prioritization parameter of “Prioritize if exceeds threshold” and a “Historic Trend” pattern engine with a prioritization parameter of “Prioritize if performance degrading over time.”
In various examples, upon the processor 110, 206, 718 identifying a “New Device Installed” event of operational data of a computing device (e.g., 202), the processor 110, 206, 718 may utilize the lookup table 400 to determine that the event should be classified to the “HW_Config” class of events. In some examples, this determination may occur for every event of the operational data until all events of the operational data are classified into a class of events. Returning to the “New Device Installed” event, the processor 110, 206, 718 may utilize the lookup table 400 to evaluate the event with a pattern engine that filters the event based on a frequency of occurrence. The processor 110, 206, 718 may utilize the lookup table 400 to determine the “New Device Installed” event should be prioritized if the event happens once.
After the classification of an event into a class of events, the class of events may be associated with zero events, one event, or a plurality of events. The plurality of events may include multiple types of events. For example, in the lookup table 400, the class of events “HW_Battery” includes a first type of event “Battery Health” and a second type of event “Battery Performance.” In some examples, the first type of event may utilize a first pattern engine associated with the class of event and the second type of event may utilize a second pattern engine associated with the class event. For example, in the lookup table 400, a “Battery Health” event may utilize the “Operational Threshold” pattern engine with a prioritization parameter of “Prioritize if exceeds threshold,” and a “Battery Performance” event may utilize the “Historic Trend” pattern engine with a prioritization parameter of “Prioritize if performance degrading over time.”
In some examples, a class of events with multiple types of events and multiple pattern engines may apply every pattern engine to every type of event (e.g., every pattern engine filters the data such that the data may pass through a number of filters). For example, in the lookup table 400, a “Battery Health” event may utilize an “Operational Threshold” pattern engine with a prioritization parameter of “Prioritize if exceeds threshold” and a “Historic Trend” pattern engine with a prioritization parameter of “Prioritize if performance degrading over time.” A first filter utilizes the “Operational Threshold” pattern engine. If the “Battery Health” event satisfies the prioritization parameter for the “Operational Threshold” pattern engine, then a second filter utilizes the “Historic Trend” pattern engine. If the “Battery Health” event satisfies the prioritization parameter for the “Historic Trend” pattern engine, then the processor 110, 206, 718 determines the “Battery Health” event is not noise and stores the “Battery Health” event to a timeline including an incident of the computing device (e.g., 202). If the “Battery Health” event fails to satisfy the prioritization parameter for the first filter or the prioritization parameter for the second filter, then the processor 110, 206, 718 determines that the “Battery Health” event is noise.
In various examples, the method 500 may be performed by the processor 110, 206, 718 of the system 100, 200, 700. The processor 110, 206, 718 may classify an event into a class of events. In some examples, the processor 110, 206, 718 may utilize a lookup table (e.g., 400) to classify the event into a class of events. For example, utilizing lookup table 400, the processor 110, 206, 718 may identify an “AppXX_Error” event and classify it into a “SW Error” class of events. The processor 110, 206, 718 may then determine a priority of the classified event utilizing a lookup table (e.g., 400). For example, the processor 110, 206, 718 may determine the “AppXX_Error” event is prioritized if data of the “AppXX_Error” event exceeds a threshold. If the classified event fails to satisfy a prioritization parameter, then the processor 110, 206, 718 may determine that the classified event is noise. If the classified event satisfies the prioritization parameter, then the classified event is prioritized. After identifying a second event, the processor 110, 206, 718 may determine the second event also belongs to the same class of events as the classified event. For example, utilizing the lookup table 400, the processor 110, 206, 718 may identify a second event as an “AppXX_Error” error and classify the second event in the “SW Error” class of events. The processor 110, 206, 718 may determine that the second “AppXX_Error” event is prioritized because data of the “AppXX_Error” event exceeds a threshold. The processor 110, 206, 718 may then adjust the priority of the second event based on the priority of the first event. For example, the processor 110, 206, 718 may compare the priority of the classified event to the priority of the second event and determine that the second event is noise because the priority of the second event fails to satisfy a parameter. In some examples, the parameter may be that the priority of the second event does not equal the priority of the classified event. For example, if the classified event is prioritized and the second event is prioritized, then the second event is classified as noise. However, in another example, the parameter may be that the priority of the second event is not noise. For example, if the classified event is one type of event of the class of events and the second event is a second type of event of the class of events and both the first and second types are prioritized, then the processor 110, 206, 718 may store both the classified event and the second event to a timeline that includes an incident of a computing device (e.g., 202).
The timeline 600 may be an example of a timeline created by the processor 110, 206, 718 of the system 100, 200, 700 utilizing the method of
The above discussion is meant to be illustrative of the principles and various examples of the present disclosure. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.
In the figures, certain features and components disclosed herein may be shown exaggerated in scale or in somewhat schematic form, and some details of certain elements may not be shown in the interest of clarity and conciseness. In some of the figures, in order to improve clarity and conciseness, a component or an aspect of a component may be omitted.
In the following discussion and in the claims, the terms “including” and “comprising” are used in an open-ended fashion, and thus should be interpreted to mean “including, but not limited to . . . .” Also, the term “couple” or “couples” is intended to be broad enough to encompass both indirect and direct connections. Thus, if a first device couples to a second device, that connection may be through a direct connection or through an indirect connection via other devices, components, and connections. As used herein, including in the claims, the word “or” is used in an inclusive manner. For example, “A or B” means any of the following: “A” alone, “B” alone, or both “A” and “B.” In addition, when used herein including the claims, the word “generally” or “substantially” means within a range of plus or minus 10% of the stated value.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2019/062304 | 11/19/2019 | WO |
