Patent Grant
9658902
Field of the Invention
The field of the invention is data processing, or, more specifically, methods, apparatuses, and computer program products for adaptive clock throttling for event processing in a distributed processing system.
Description of Related Art
The development of the EDVAC computer system of 1948 is often cited as the beginning of the computer era. Since that time, computer systems have evolved into extremely complicated devices. Today's computers are much more sophisticated than early systems such as the EDVAC. Computer systems typically include a combination of hardware and software components, application programs, operating systems, processors, buses, memory, input/output devices, and so on. As advances in semiconductor processing and computer architecture push the performance of the computer higher and higher, more sophisticated computer software has evolved to take advantage of the higher performance of the hardware, resulting in computer systems today that are much more powerful than just a few years ago.
Modern distributed processing systems for intensive computing may have millions of devices with many processes running on each device all of which are capable of error and status reporting for automated error recovery, reporting to a systems administrator, and for other reasons. In many cases, in the case of an error for example, the sheer number of such error reports and status reports are so overwhelming that they cannot be handled in a meaningful manner. For example, a systems administrator receiving a hundred thousand error reports may be overwhelmed by the sheer number of such reports and therefore in the aggregate those reports become more and more unhelpful and irrelevant.
Methods, apparatuses, and computer program products for adaptive clock throttling for event processing are provided. Embodiments include an event processing system receiving a plurality of events from one or more components of the distributed processing system. Embodiments also include the event processing system determining that an arrival attribute of the plurality of events exceeds an arrival threshold. Embodiments also include the event processing system, adjusting, in response to determining that the arrival attribute of the plurality of events exceeds the arrival threshold, a clock speed of at least one of the event processing system and a component of the distributed processing system.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular descriptions of exemplary embodiments of the invention as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts of exemplary embodiments of the invention.
Exemplary methods, apparatuses, and computer program products for adaptive clock throttling for event processing in a distributed processing system according to embodiments of the present invention are described with reference to the accompanying drawings, beginning with
In the example of
In addition to the compute nodes (102), the parallel computer (100) includes input/output (‘I/O’) nodes (110, 114) coupled to the compute nodes (102) through the high speed Ethernet network (174). The I/O nodes (110, 114) provide I/O services between the compute nodes (102) and I/O devices, which in this example is the data storage device (118), the printer (120) and the terminal (122). The I/O nodes (110, 114) are connected for data communications through a local area network (‘LAN’) (130). The parallel computer (100) also includes a service node (116) coupled to the compute nodes (102) through the JTAG network (104). The service node (116) provides service common to the compute nodes (102), such as loading programs into the compute nodes (102), starting program execution on the compute nodes (102), retrieving results of program operations on the compute nodes (102), and so on. The service node (116) runs an event and alert analysis module (124) and communicates with a system administrator (128) through a service application interface (126) that runs on the computer terminal (122).
Many of the components of the distributed processing system of
An incident is a generic term used in this specification to mean an identification or notification of a particular occurrence on a component of a distributed processing system such as events described below, a refined identification of an occurrence often based on events such as an alert described below, or other notifications as will occur to those of skill in the art.
Incidents are administered in pools for event and alert analysis according to embodiments of the present invention. A pool of incidents is a collection of incidents organized by the time of either their occurrence, by the time they are logged in an incident queue, included in the pool, or other time as will occur to those of skill in the art. Such incident pools often provide the ability to analyze a group of time related incidents. Often such incident pools are useful in identifying fewer and more relevant incidents in dependence upon multiple related incidents.
An event according to embodiments of the present invention is a notification of a particular occurrence in or on a component of the distributed processing system. Such events are sent from the component upon which the occurrence occurred or another reporting component to an event and alert analysis module according to the present invention. Often events are notifications of errors occurring in a component of the data processing system. Events are often implemented as messages either sent through a data communications network or shared memory. Typical events for event and alert analysis according to embodiments of the present invention include attributes such as an occurred time, a logged time, an event type, an event ID, a reporting component, and a source component, and other attributes.
An alert according to embodiments of the present invention is a refined identification of an occurrence, such as an error, based upon more than one event and therefore provides an identification of the occurrence in the context of its operation in the distributed processing system. Often an alert may be a notification of a particular error type of occurrence that is identified in dependence upon the plurality of events received from one or more components of the data processing system, such as, for example, a link failure among a plurality of devices each of which are producing many events based upon the single link failure, or a power failure provoking thousands of events, and so on.
Alerts are often implemented as messages to be sent through a data communications network or shared memory. Typical alerts according to embodiments of the present invention have attributes attached to them based upon the attributes of the events received from which they are identified.
The event and alert analysis module (124) includes at least two incident analyzers implemented as an event analyzer and an alert analyzer capable of adaptive clock throttling for event processing in a distributed processing system according to embodiments of the present invention. The event and alert analysis module (124) is also implemented as a monitor and checkpoint manager for managing the checkpoints from the incident analyzers.
Specifically, the event and alert analysis module (124) is implemented as automated computing machinery configured to receive a plurality of events from one or more components of the distributed processing system and determine that an arrival attribute of the plurality of events exceeds an arrival threshold. The event and alert analysis module is also configured to adjust, in response to determining that the arrival attribute of the plurality of events exceeds the arrival threshold, a clock speed of at least one of the event and alert analysis module and a component of the distributed processing system.
The arrangement of nodes, networks, and I/O devices making up the exemplary distributed processing system illustrated in
Adaptive clock throttling for event processing in a distributed processing system in accordance with the present invention is generally implemented with computers, that is, with automated computing machinery. In the system of
Stored in RAM (268) is an event and alert analysis module (124), a module of automated computing machinery for performing adaptive clock throttling for event processing according to embodiments of the present invention. The event and alert analysis module (124) includes two incident analyzers, a monitor (204), and a checkpoint manager (299) according to embodiments of the present invention.
The checkpoint manager (299) performs adaptive clock throttling for event processing according to embodiments of the present invention by processing checkpoints from the incident analyzers. The monitor (204) is configured to perform adaptive clock throttling for event processing in a distributed processing system according to embodiments of the present invention. In the example of
The incident analyzers include an event analyzer (208) and an alert analyzer (218). The event analyzer of
The alert analyzer (218) of
In addition to the general functions described above, the event and alert analysis module (124) may be configured to perform adaptive clock throttling for event processing in a distributed processing system according to embodiments of the present invention. Specifically, the event and alert analysis module (124) is implemented as automated computing machinery configured to receive a plurality of events from one or more components of the distributed processing system and determine that an arrival attribute of the plurality of events exceeds an arrival threshold. The event and alert analysis module is also configured to adjust, in response to determining that the arrival attribute of the plurality of events exceeds the arrival threshold, a clock speed of at least one of the event and alert analysis module and a component of the distributed processing system.
Also stored in RAM (268) is an operating system (254). Operating systems useful for relevant alert delivery according to embodiments of the present invention include UNIX™, Linux™, Microsoft XP™, AIX™, IBM's i5/OS™, and others as will occur to those of skill in the art. The operating system (254), event and alert analysis module (124), the event analyzer (208), the alert analyzer (218) in the example of
The computer (252) of
The example computer (252) of
The exemplary computer (252) of
For further explanation,
An event according to embodiments of the present invention is a notification of a particular occurrence in or on a component of the distributed processing system. Such events are sent from the component upon which the occurrence occurred or another reporting component to an event and alert analysis module according to the present invention. Often events are notifications of errors occurring in a component of the data processing system. Events are often implemented as messages either sent through a data communications network or shared memory. Typical events for event and alert analysis according to embodiments of the present invention include attributes such as an occurred time, a logged time, an event type, an event ID, a reporting component, and a source component, and other attributes. An occurred time is the time at which the event occurred on the component. A logged time is the time the event was included in the event queue (306) and is typically inserted into the event by a monitor. An event type is a generic type of event such as for example, power error, link failure error, errors related to not receiving messages or dropping packets and so on as will occur to those of skill in the art. An event ID is a unique identification of the event. A reporting component is an identification of the component that reported the event. A source component is an identification of the component upon which the event occurred. In many cases, but not all, the reporting component and source component are the same component of the distributed processing system.
The event and analysis module (124) of
In the example of
The system of
As mentioned above, in some embodiments of the present invention, more than one event analyzer may operate in parallel. As such, each event analyzer may maintain one or more events pools for performing adaptive clock throttling for event processing according to embodiments of the present invention. Assigning by the event analyzer the events to an events pool may therefore include selecting only events from one or more particular components. In such embodiments, particular components may be selected for a particular events pool to provide events associated with a particular period of time from a particular set of one or more components.
Assigning by the event analyzer the events to an events pool may also be carried out by selecting only events of a particular event type. In such embodiments, particular events may be selected for a particular events pool to provide events associated with a particular period of time from a particular set of event types. The event analyzer (208) in the example of
Event analyses rules (310) are a collection of predetermined rules for meaningfully parsing received events to identify relevant alerts in dependence upon the events.
The event analysis rules (310) of
An alert according to embodiments of the present invention is a refined identification of an occurrence, such as an error based upon more than one event, and therefore provides an identification of the occurrence in the context of its operation in the distributed processing system. Often an alert may be a notification of a particular error type of occurrence that is identified in dependence upon the plurality of events received from one or more components of the data processing system, such as, for example, a link failure among a plurality of devices each of which are producing many events based upon the single link failure, or a power failure provoking thousands of events, and so on.
Alerts are often implemented as messages to be sent through a data communications network or shared memory. Typical alerts according to embodiments of the present invention have attributes attached to them based upon the attributes of the events received from which they are identified.
The events pool operation rules (332) are configurable predetermined rules for controlling the operations of the events pool. Such rules includes rules identifying the initial predetermined period of time for each events pool, rules dictating the length of time extended to the pool upon the assignment of each new event to the pool, rules dictating the minimum time an event must be in a pool before that event is included in a collection of events when the pool is closed, rules dictating the amount to extend the initial predetermined period of time based on an arrival rate of events assigned to an events pool, rules governing the closing of an events pool, and others as will occur to those of skill in the art. Such rules are flexible and may be tailored to a particular distributed computing system and its functions.
The event suppression rules (334) are configurable predetermined rules for suppressing one or more events in a closed pool of events used in identifying alerts. That is, often events in the closed events pool may be duplicate events, redundant events, or otherwise unnecessary or unhelpful events in identifying alerts. Such suppression rules are typically predetermined to delete, drop, or otherwise ignore those suppressed events. Event suppression rules may for example dictate that more than a threshold number of events of a particular event type or component type are to be suppressed. Such rules are also flexible and may be tailored to a particular distributed computing system and its functions.
The events pool closure rules (336) are configurable predetermined rules for identifying alerts in dependence upon unsuppressed events in the closed events pool and alerts identified by the event arrival rules. That is, events pool closure rules identify new alerts in dependence upon one or more or even all of the unsuppressed events in the closed events pool. The events pool closure rules also identify alerts in dependence upon the alerts identified by the event arrival rules (330) or a combination of the alerts identified by the event arrival rules (330) and one or more of the unsuppressed events in the closed events pool.
The event analyzer (208) in the example of
The alert analyzer (218) of
The alert analyzer (218) of
The alert analysis rules (322) are a collection of rules for suppressing one or more alerts to provide a more relevant set of alerts for transmission to a component of the distributed processing system, such as for example, for display to a systems administrator and to identify additional alerts for transmission to one or more components of the distributed processing system. Alert analysis rules for example may dictate that duplicate alerts are to be suppressed, alerts of a particular type for transmission to a particular component are to be suppressed, alerts of a particular type be transmitted to a particular component are to be suppressed and so on as will occur to those of skill in the art. Such alerts may be more meaningful to a component of the distributed processing system for automated error recovery or for a systems administrator who may otherwise be less informed by a number of raw unanalyzed alerts.
The alert analyzer (218) of
The alert analyzer (218) of
The alert analyzer (218) of
In the system of
The event and alert analysis module (124) is also configured to receive a plurality of events from one or more components of the distributed processing system and determine that an arrival attribute of the plurality of events exceeds an arrival threshold. The event and alert analysis module is also configured to adjust, in response to determining that the arrival attribute of the plurality of events exceeds the arrival threshold, a clock speed of at least one of the event and alert analysis module and a component of the distributed processing system.
As mentioned above, adaptive clock throttling for event processing according to embodiments of the present invention may include assigning events to an events pool and those pools are administered according to embodiments of the present invention. For further explanation,
Events pools according to embodiments of the present invention are typically operated according to events pool operation rules which are themselves often included in event analysis rules. Such events pool operation rules are configurable predetermined rules for controlling the operations of the events pool. Such rules includes rules identifying the initial predetermined period of time for each events pool, rules dictating the length of time extended to the pool upon the assignment of each new event to the pool, rules dictating the minimum time an event must be in a pool before that event is included in a collection of events when the pool is closed, rules dictating the amount to extend the initial predetermined period of time based on an arrival rate of events assigned to an events pool, rules governing the closing of an events pool, and others as will occur to those of skill in the art. Such rules are flexible and may be tailored to a particular distributed computing system and its functions.
Events are often assigned to an events pool according to their logged time. That is, events are typically inserted into the events pool in the order that they are received in the event queue. In the example of
In the example
In typical embodiments of the present invention, events pools may have a maximum duration that can no longer be extended. In such cases, a requirement may exist that an event that has not resided in the events pool for a threshold period of time be moved to a next events pool. In some embodiments, the attributes of such an event that is moved to the next events pool are used for relevant alert delivery with the initial events pool and in other embodiments; the attributes of such an event are used for relevant alert delivery with the next events pool to which that event is moved.
In the example of
In many embodiments, a plurality of events pools may be used in parallel and one or more of such events pools are assigned to a particular events analyzer. In such embodiments, events analyzers may be directed to events in events pools having particular attributes.
As mentioned above, adaptive clock throttling for event processing according to embodiments of the present invention may include assigning alerts to an alerts pool and those pools are administered according to embodiments of the present invention. For further explanation,
In typical embodiments of the present invention, alerts pools may have a maximum duration that can no longer be extended. In such cases, a requirement may exist that an alert that has not resided in the alerts pool for a threshold period of time be moved to a next alerts pool. In some embodiments, the attributes of such an alert that is moved to the next alerts pool are used for relevant alert delivery according to embodiments of the present invention with the initial alerts pool and in other embodiments, the attributes of such an alert are used for relevant alert delivery with the next alerts pool to which that alert is moved.
In the example of
In many embodiments, a plurality of alerts pools may be used in parallel and one or more of such alerts pools are assigned to a particular alerts analyzer. In such embodiments, alerts analyzers may be directed to alerts in alerts pools having particular attributes.
As mentioned above, adaptive clock throttling for event processing according to embodiments of the present invention may include the administration of one or more pools of incidents such as events, alerts or other incidents as will occur to those of skill in the art. For further explanation,
Receiving (602) in an event queue a plurality of events (302) from one or more components of a distributed processing system may be carried out by receiving an event initiated by one or more components of the data processing system and storing the event in the event queue according to the time in which the event occurred or according to the time the event was received. Receiving (602) in an event queue a plurality of events (302) from one or more components of a distributed processing system also may be carried out by polling a component for status and receiving in response an event and storing the event in the event queue according to the time in which the event occurred or according to the time the event was received.
The method of
An events pool according to the method of
The event analyzer includes event analysis rules (310) including, event arrival rules, events pool operation rules, event suppression rules, and events pool closure rules. Event arrival rules are configurable predetermined rules for identifying alerts in dependence upon events in real time as those events are assigned to the events pool. That is, event arrival rules identify alerts in dependence upon events before closing the events pool. Such rules are flexible and may be tailored to a particular distributed computing system and its functions.
An alert according to embodiments of the present invention is a refined identification of an occurrence, such as an error based upon more than one event, and therefore provides an identification of the occurrence in the context of its operation in the distributed processing system. Often an alert may be a notification of a particular error type of occurrence that is identified in dependence upon the plurality of events received from one or more components of the data processing system, such as, for example, a link failure among a plurality of devices each of which are producing many events based upon the single link failure, or a power failure provoking thousands of events, and so on.
Alerts are often implemented as messages to be sent through a data communications network or shared memory. Typical alerts according to embodiments of the present invention have attributes attached to them based upon the attributes of the events received from which they are identified.
Events pool operation rules are configurable predetermined rules for controlling the operations of the events pool. Such rules includes rules identifying the initial predetermined period of time for each events pool, rules dictating the length of time extended to the pool upon the assignment of each new event to the pool, rules dictating the minimum time an event must be in a pool before that event is included in a collection of events when the pool is closed, rules governing the closing of an events pool, and others as will occur to those of skill in the art. Such rules are flexible and may be tailored to a particular distributed computing system and its functions.
Event suppression rules are configurable predetermined rules for suppressing one or more events in a closed pool of events used in identifying alerts. That is, often events in the closed events pool may be duplicate events, redundant events, or otherwise unnecessary or unhelpful events in identifying alerts. Such suppression rules are typically predetermined to delete, drop, or otherwise ignore those suppressed events. Event suppression rules may for example dictate that more than a threshold number of events of a particular event type or component type are to be suppressed. Such rules are also flexible and may be tailored to a particular distributed computing system and its functions.
Events pool closure rules are configurable predetermined rules for identifying alerts in dependence upon unsuppressed events in the closed events pool and alerts identified by the event arrival rules. That is, events pool closure rules identify new alerts in dependence upon one or more or even all of the unsuppressed events in the closed events pool. The events pool closure rules also identify alerts in dependence upon the alerts identified by the event arrival rules or a combination of the alerts identified by the event arrival rules and one or more of the unsuppressed events in the closed events pool.
The method of
The method of
Closing the events pool may be carried out by determining that the initial period of time for the events pool and any particular periods of time for events received in the events pool extended to the initial period of time have expired. In such cases, if no new events are received prior to the expiration of the initial period of time for the events pool and any particular periods of time for events received in the events pool extended to the initial period of time the pool is closed.
Closing the events pool may also be carried out by determining that a maximum duration for the events pool has expired. In such cases, regardless of the number of new events being received after a maximum duration for the events pool has expired the pool is closed. In such embodiments, a maximum duration for the events pool prevents the events pool from including more events than are useful for relevant alert delivery according to embodiments of the present invention.
The method of
The method of
The method of
The method of
The method of
The method of
As mentioned above, alert analysis rules may select additional alerts or suppress alerts in dependence upon events. In such embodiments, determining whether to suppress any alerts includes selecting events and determining whether to suppress any alerts in dependence upon the selected events. The method of
For further explanation,
The method of
An events storm, as that term is used here, is the result of an occurrence in the distributed processing system that causes an overwhelming number of events to be reported by components of the distributed processing system. Such occurrences that may cause an events storm may include the loss of an entire circuit providing power to many components of the data processing system, catastrophic failure of a number of components of the distributed processing system and others as will occur to those of skill in the art. Events storms are often called Reliability, Availability, and Serviceability (RAS) storm. According to embodiments of the present invention, determining (704) that an arrival attribute (780) of the plurality (750) of events exceeds an arrival threshold (760) may serve as an indication of an event storm. Determining (704) that an arrival attribute (780) of the plurality (750) of events exceeds an arrival threshold (760) may be carried out by counting the number of received events over a predetermined period of time and comparing that ratio to the arrival threshold.
The method of
For example, the event processing system may lower the clock speed of a particular component producing events during an event storm. Reducing the clock speed of the particular component may reduce the rate at which the particular component is producing events, thus reducing the rate and number of events that the event processing system receives during the event storm from that particular component. Reducing the number of events that the event processing system receives may allow an event storm to subside and the event processing system to more effectively consume and process the events.
As another example, the event processing system may increase the clock speed of a particular component producing events during an event storm. Increasing the clock speed of the particular component may increase the rate at which the particular component is producing events, thus increasing the rate and number of events that the event processing system receives during the event storm from that particular component. Increasing the rate that events are received from a particular component may increase the number of important events received during an event storm, thus allowing the event processing system to more effectively consume and process the events.
That is, according to embodiments of the present invention, the event processing system, by adjusting the clock speed of a component, may increase or decrease the rate at which events are received from the component. Whether the event processing system wants to increase or decrease the arrival rate of events from a component may depend on a variety of factors, such as the type of events the component is producing.
As another example, the event processing system may increase the clock speed of one or more processors within the event processing system. Increasing the clock speed of the event processing system may increase the rate at which the event processing system is analyzing and processing events, thus allowing the event processing system to more effectively consume and process the events.
For further explanation,
In the method of
In the method of
In the method of
For example, during an event storm, the event processing system may identify components producing a large number of events and send an instruction to reduce the clock speed of the identified components. Reducing the clock speed of the identified components may reduce the rate at which the identified components are producing events, thus reducing the rate and number of events that the event processing system receives during the event storm. Reducing the number of events that the event processing system receives may allow an event storm to subside and the event processing system to more effectively consume and process the events.
For further explanation,
In the method of
In the method of
In the method of
For example, the event processing system may classify an informational type event as a predetermined event type that is a lower priority during an event storm. In this example, during an event storm, the event processing system may identify components producing a large number of informational events and send an instruction to reduce the clock speed of the identified components. Reducing the clock speed of the identified components may reduce the rate at which the identified components are producing events, thus reducing the rate and number of information type events that the event processing system receives during the event storm. Reducing the number of events that the event processing system receives may allow an event storm to subside and the event processing system to more effectively consume and process the events.
As another example, the event processing system may classify a power type event as a predetermined event type that is a higher priority during an event storm. In this example, during an event storm, the event processing system may identify components producing power events and send an instruction to increase the clock speed of the identified components. Increasing the clock speed of the identified components may increase the rate at which the identified components are producing events, thus increasing the rate of events that the event processing system receives during the event storm from the identified components. Increasing the rate that events are received from a particular component may increase the number of important events received during an event storm, thus allowing the event processing system to more effectively consume and process the events.
For further explanation,
The method of
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It will be understood from the foregoing description that modifications and changes may be made in various embodiments of the present invention without departing from its true spirit. The descriptions in this specification are for purposes of illustration only and are not to be construed in a limiting sense. The scope of the present invention is limited only by the language of the following claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4812852 | Bent et al. | Mar 1989 | A |
| 4823261 | Bank et al. | Apr 1989 | A |
| 4965772 | Daniel et al. | Oct 1990 | A |
| 5119377 | Cobb et al. | Jun 1992 | A |
| 5276861 | Howarth | Jan 1994 | A |
| 5515285 | Garrett et al. | May 1996 | A |
| 5535380 | Bergkvist et al. | Jul 1996 | A |
| 5581242 | Arita et al. | Dec 1996 | A |
| 5689636 | Kleber et al. | Nov 1997 | A |
| 5777549 | Arrowsmith et al. | Jul 1998 | A |
| 5991881 | Conklin et al. | Nov 1999 | A |
| 6094681 | Shaffer et al. | Jul 2000 | A |
| 6255943 | Lewis et al. | Jul 2001 | B1 |
| 6314533 | Novik et al. | Nov 2001 | B1 |
| 6373383 | Arrowsmith et al. | Apr 2002 | B1 |
| 6401116 | Okigami et al. | Jun 2002 | B1 |
| 6405250 | Lin et al. | Jun 2002 | B1 |
| 6446136 | Pohlmann et al. | Sep 2002 | B1 |
| 6446224 | Chang et al. | Sep 2002 | B1 |
| 6513129 | Tentij et al. | Jan 2003 | B1 |
| 6539335 | Morris et al. | Mar 2003 | B1 |
| 6594786 | Connelly et al. | Jul 2003 | B1 |
| 6606610 | Gray et al. | Aug 2003 | B1 |
| 6631409 | Watson et al. | Oct 2003 | B1 |
| 6704874 | Porras et al. | Mar 2004 | B1 |
| 6708288 | Ziegler et al. | Mar 2004 | B1 |
| 6754854 | Kurrasch | Jun 2004 | B2 |
| 6801927 | Smith et al. | Oct 2004 | B1 |
| 6871349 | Akhond et al. | Mar 2005 | B1 |
| 6889169 | Kirshenbaum et al. | May 2005 | B2 |
| 6915285 | Gray et al. | Jul 2005 | B2 |
| 6918059 | Galuten et al. | Jul 2005 | B1 |
| 6925586 | Perrella et al. | Aug 2005 | B1 |
| 6988208 | Hrabik et al. | Jan 2006 | B2 |
| 6990601 | Tsuneya et al. | Jan 2006 | B1 |
| 7000138 | Pillay | Feb 2006 | B1 |
| 7043659 | Klein et al. | May 2006 | B1 |
| 7117391 | Hornick et al. | Oct 2006 | B1 |
| 7213179 | Song et al | May 2007 | B2 |
| 7251584 | Perazolo et al. | Jul 2007 | B1 |
| 7251829 | Pagdin et al. | Jul 2007 | B1 |
| 7263553 | Gaspard | Aug 2007 | B2 |
| 7281172 | Chujo | Oct 2007 | B2 |
| 7289988 | Joseph | Oct 2007 | B2 |
| 7299152 | Moritz | Nov 2007 | B1 |
| 7430692 | White et al. | Sep 2008 | B2 |
| 7457805 | Deen et al. | Nov 2008 | B2 |
| 7469239 | Musman | Dec 2008 | B2 |
| 7568027 | Becker et al. | Jul 2009 | B2 |
| 7599359 | Croak et al. | Oct 2009 | B1 |
| 7603711 | Scheidell | Oct 2009 | B2 |
| 7606891 | Shyu et al. | Oct 2009 | B1 |
| 7613741 | Lu et al. | Nov 2009 | B2 |
| 7617074 | Beish et al. | Nov 2009 | B2 |
| 7673335 | Chakravarty et al. | Mar 2010 | B1 |
| 7687066 | Fujino et al. | Mar 2010 | B2 |
| 7702782 | Pai | Apr 2010 | B1 |
| 7756053 | Thomas et al. | Jul 2010 | B2 |
| 7792042 | Golla et al. | Sep 2010 | B2 |
| 7815103 | Timmis et al. | Oct 2010 | B2 |
| 7822848 | Muller et al. | Oct 2010 | B2 |
| 7872982 | Atkins et al. | Jan 2011 | B2 |
| 7904319 | Whear et al. | Mar 2011 | B1 |
| 7954159 | Hrabik et al. | May 2011 | B2 |
| 7979355 | Shah et al. | Jul 2011 | B2 |
| 7984452 | Chakravarty et al. | Jul 2011 | B2 |
| 7996046 | Vargas et al. | Aug 2011 | B2 |
| 8020045 | Morimura et al. | Sep 2011 | B2 |
| 8041799 | Usery et al. | Oct 2011 | B1 |
| 8135863 | Nekovee et al. | Mar 2012 | B2 |
| 8248228 | Hollender | Aug 2012 | B2 |
| 8314694 | Naedele et al. | Nov 2012 | B2 |
| 8321865 | Amini et al. | Nov 2012 | B2 |
| 8347143 | Atkins et al. | Jan 2013 | B2 |
| 8350681 | Bells | Jan 2013 | B2 |
| 8364813 | Atkins et al. | Jan 2013 | B2 |
| 8380838 | Bose et al. | Feb 2013 | B2 |
| 8386602 | Carey et al. | Feb 2013 | B2 |
| 8417809 | Lancaster et al. | Apr 2013 | B1 |
| 8418247 | Sinnaya et al. | Apr 2013 | B2 |
| 8458530 | Kini et al. | Jun 2013 | B2 |
| 8495661 | Carey et al. | Jul 2013 | B2 |
| 8676883 | Atkins et al. | Mar 2014 | B2 |
| 8688769 | Atkins et al. | Apr 2014 | B2 |
| 8737231 | Carey et al. | May 2014 | B2 |
| 8769096 | Carey et al. | Jul 2014 | B2 |
| 8775917 | Bourke et al. | Jul 2014 | B2 |
| 8825836 | Gibson et al. | Sep 2014 | B1 |
| 8832219 | Morgan | Sep 2014 | B2 |
| 8856807 | Khapre et al. | Oct 2014 | B1 |
| 8868986 | Carey et al. | Oct 2014 | B2 |
| 8959063 | Haeberle et al. | Feb 2015 | B2 |
| 8966505 | Neilan | Feb 2015 | B1 |
| 20010055963 | Cloutier | Dec 2001 | A1 |
| 20020016871 | Graf | Feb 2002 | A1 |
| 20020077836 | Elnozahy et al. | Jun 2002 | A1 |
| 20020095595 | Christopherson et al. | Jul 2002 | A1 |
| 20030026525 | Alvarez | Feb 2003 | A1 |
| 20030061514 | Bardsley et al. | Mar 2003 | A1 |
| 20030200187 | Gray et al. | Oct 2003 | A1 |
| 20040030531 | Miller et al. | Feb 2004 | A1 |
| 20040153693 | Fisher et al. | Aug 2004 | A1 |
| 20040181294 | Deitz et al. | Sep 2004 | A1 |
| 20040221025 | Johnson et al. | Nov 2004 | A1 |
| 20040243905 | Merritt | Dec 2004 | A1 |
| 20050010545 | Joseph | Jan 2005 | A1 |
| 20050034134 | Lieblich et al. | Feb 2005 | A1 |
| 20050183093 | Pope et al. | Aug 2005 | A1 |
| 20050193285 | Jeon | Sep 2005 | A1 |
| 20050210331 | Connelly et al. | Sep 2005 | A1 |
| 20050240547 | Gray et al. | Oct 2005 | A1 |
| 20050246288 | Kimura et al. | Nov 2005 | A1 |
| 20060015608 | Becker et al. | Jan 2006 | A1 |
| 20060020942 | Ly et al. | Jan 2006 | A1 |
| 20060036720 | Faulk, Jr. | Feb 2006 | A1 |
| 20060085724 | Merritt | Apr 2006 | A1 |
| 20060129947 | Hamzy et al. | Jun 2006 | A1 |
| 20060168185 | McCall et al. | Jul 2006 | A1 |
| 20060174251 | Pope et al. | Aug 2006 | A1 |
| 20060271784 | Bolosky et al. | Nov 2006 | A1 |
| 20060282419 | Sen et al. | Dec 2006 | A1 |
| 20070002736 | Gade et al. | Jan 2007 | A1 |
| 20070033594 | Allen et al. | Feb 2007 | A1 |
| 20070037521 | Babut et al. | Feb 2007 | A1 |
| 20070073708 | Smith et al. | Mar 2007 | A1 |
| 20070088755 | Nesbitt et al. | Apr 2007 | A1 |
| 20070100959 | Eichstaedt et al. | May 2007 | A1 |
| 20070100960 | Eichstaedt et al. | May 2007 | A1 |
| 20070124437 | Chervets | May 2007 | A1 |
| 20070136237 | Barker et al. | Jun 2007 | A1 |
| 20070174768 | Sen et al. | Jul 2007 | A1 |
| 20070177523 | Nagami et al. | Aug 2007 | A1 |
| 20070180103 | Atkins et al. | Aug 2007 | A1 |
| 20070222576 | Miller et al. | Sep 2007 | A1 |
| 20070294399 | Grossner et al. | Dec 2007 | A1 |
| 20080071403 | Conway et al. | Mar 2008 | A1 |
| 20080080384 | Atkins et al. | Apr 2008 | A1 |
| 20080109683 | Erwin et al. | May 2008 | A1 |
| 20080155360 | Bates et al. | Jun 2008 | A1 |
| 20080196044 | Stanley | Aug 2008 | A1 |
| 20080228787 | Merritt | Sep 2008 | A1 |
| 20080235365 | Bansal et al. | Sep 2008 | A1 |
| 20080284581 | Sheleheda et al. | Nov 2008 | A1 |
| 20090006883 | Zhang et al. | Jan 2009 | A1 |
| 20090070785 | Alvez et al. | Mar 2009 | A1 |
| 20090077224 | Appleton | Mar 2009 | A1 |
| 20090094649 | Patel | Apr 2009 | A1 |
| 20090183037 | Hamaguchi et al. | Jul 2009 | A1 |
| 20090199051 | Jann et al. | Aug 2009 | A1 |
| 20090216881 | Lovy et al. | Aug 2009 | A1 |
| 20090275807 | Sitzman et al. | Nov 2009 | A1 |
| 20090292948 | Cinato et al. | Nov 2009 | A1 |
| 20090327429 | Hughes et al. | Dec 2009 | A1 |
| 20090328044 | Bergheaud et al. | Dec 2009 | A1 |
| 20100019894 | Okada | Jan 2010 | A1 |
| 20100031354 | Hudis et al. | Feb 2010 | A1 |
| 20100042632 | Johnson et al. | Feb 2010 | A1 |
| 20100083382 | Farley et al. | Apr 2010 | A1 |
| 20100180150 | Jeddeloh | Jul 2010 | A1 |
| 20100192163 | Pope et al. | Jul 2010 | A1 |
| 20100211192 | Stluka et al. | Aug 2010 | A1 |
| 20100211952 | Kasravi et al. | Aug 2010 | A1 |
| 20100287615 | Martin et al. | Nov 2010 | A1 |
| 20100332918 | Harnois | Dec 2010 | A1 |
| 20110078519 | Yordanov et al. | Mar 2011 | A1 |
| 20110106941 | Franklin | May 2011 | A1 |
| 20110119372 | Rodrigues et al. | May 2011 | A1 |
| 20110122773 | Kung et al. | May 2011 | A1 |
| 20110145659 | Ikeyama | Jun 2011 | A1 |
| 20110161130 | Whalin et al. | Jun 2011 | A1 |
| 20110167112 | Mazzucco et al. | Jul 2011 | A1 |
| 20110193704 | Harper et al. | Aug 2011 | A1 |
| 20110200304 | Rutledge | Aug 2011 | A1 |
| 20110275356 | Best et al. | Nov 2011 | A1 |
| 20110289433 | Whalin et al. | Nov 2011 | A1 |
| 20120084432 | Soprovich et al. | Apr 2012 | A1 |
| 20120102503 | Meijer | Apr 2012 | A1 |
| 20120110153 | Atkins et al. | May 2012 | A1 |
| 20120110161 | Carey et al. | May 2012 | A1 |
| 20120110600 | Carey et al. | May 2012 | A1 |
| 20120143875 | Sarma et al. | Jun 2012 | A1 |
| 20120144020 | Carey et al. | Jun 2012 | A1 |
| 20120144021 | Carey et al. | Jun 2012 | A1 |
| 20120144243 | Carey et al. | Jun 2012 | A1 |
| 20120144251 | Carey et al. | Jun 2012 | A1 |
| 20120239973 | Walton et al. | Sep 2012 | A1 |
| 20120275456 | Ammireddy | Nov 2012 | A1 |
| 20120303815 | Atkins et al. | Nov 2012 | A1 |
| 20120304012 | Atkins et al. | Nov 2012 | A1 |
| 20120304013 | Atkins et al. | Nov 2012 | A1 |
| 20120304022 | Carey et al. | Nov 2012 | A1 |
| 20120330918 | Carey et al. | Dec 2012 | A1 |
| 20120331332 | Carey et al. | Dec 2012 | A1 |
| 20120331347 | Carey et al. | Dec 2012 | A1 |
| 20120331485 | Carey et al. | Dec 2012 | A1 |
| 20130073726 | Carey et al. | Mar 2013 | A1 |
| 20130074102 | Carey et al. | Mar 2013 | A1 |
| 20130080630 | Carey et al. | Mar 2013 | A1 |
| 20130091386 | Atkins et al. | Apr 2013 | A1 |
| 20130097215 | Atkins et al. | Apr 2013 | A1 |
| 20130097216 | Carey et al. | Apr 2013 | A1 |
| 20130097272 | Atkins et al. | Apr 2013 | A1 |
| 20130097300 | Atkins et al. | Apr 2013 | A1 |
| 20130097310 | Carey et al. | Apr 2013 | A1 |
| 20130097619 | Carey et al. | Apr 2013 | A1 |
| 20130097620 | Carey et al. | Apr 2013 | A1 |
| 20130111502 | Carey et al. | May 2013 | A1 |
| 20130132460 | Atkins et al. | May 2013 | A1 |
| 20130138809 | Carey et al. | May 2013 | A1 |
| 20130144932 | Atkins et al. | Jun 2013 | A1 |
| 20130166743 | Carey et al. | Jun 2013 | A1 |
| 20130179905 | Atkins et al. | Jul 2013 | A1 |
| 20130290554 | Chen et al. | Oct 2013 | A1 |
| 20130305103 | Carey et al. | Nov 2013 | A1 |
| 20130318404 | Carey et al. | Nov 2013 | A1 |
| 20140040673 | Carey et al. | Feb 2014 | A1 |
| 20140047273 | Carey et al. | Feb 2014 | A1 |
| 20140068347 | Carey et al. | Mar 2014 | A1 |
| 20140101307 | Carey et al. | Apr 2014 | A1 |
| 20140172938 | Carey et al. | Jun 2014 | A1 |
| 20140214896 | Hotta et al. | Jul 2014 | A1 |
| 20150033243 | Carey et al. | Jan 2015 | A1 |
| 20150058676 | Boger et al. | Feb 2015 | A1 |
| 20150074164 | Boger et al. | Mar 2015 | A1 |
| 20150074472 | Boger et al. | Mar 2015 | A1 |
| 20150149630 | Atkins et al. | May 2015 | A1 |
| 20150193295 | Boger | Jul 2015 | A1 |
| 20150195165 | Boger | Jul 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 101026494 | Aug 2007 | CN |
| 101739745 | Jun 2010 | CN |
| 2007094997 | Apr 2007 | JP |
| Entry |
|---|
| Cottrell,et al., “Distributed Computing Environment Monitoring and User Expectations,”, CHEP95 Conference, Sep. 1995, pp. 1-29, SLAC, Rio de Janeiro. |
| Sundstrom,et al., “SNA: Current Requirements and Direction,” IBM Systems Journal, vol. 26, No. 1 1987, pp. 13-36, IBM Communication Products Division, Research Triangle Park, North Carolina USA. |
| Carey et al., “A Toolkit for Event Analysis and Logging”, SC'11, Nov. 12, 2011, pp. 1-7, ACM, Seattle, Washington. |
| Interlink Software Services, “Business Enterprise Server User Guide”, Jun. 20, 2012, pp. 1-506, Version 3.5, Interlink Software Services, Ltd., Manchester, UK. |
| Sourceforge, “HowTo—Event Analyzer”,—sourceforge.net (online), Nov. 6, 2012, [accessed Apr. 15, 2013], 4 pages, URL: http://sourceforge.net/apps/mediawiki/pyteal/index.php?title=HowTo—-—Event—Analyzer#Checkpoint—support. |
| Splunk Inc., “Splunk Hadoop Connect”, splunk.com (online), 2 pages, [accessed May 15, 2013], URL: http://www.splunk.com/view/hadoop-connect/SP-CAAAHA3. |
| Brown, “Using Hadoop to Process a Trillion+ Events”, Presentation, Mar. 2012, 29 pages, comScore, Inc., (online), URL: http://www.slideshare.net/Hadoop—Summit/analyzing-14-trillion-events-with-hadoop. |
| Kimball, “Real-time Streaming Analysis for Hadoop and Flume”, Open Source Data Convention (OSCON), Jul. 2011, 24 pages, oscon.com (online), URL: http://cdn.oreillystatic.com/en/assets/1/event/61/Real-time%20Streaming%20Analysis%20for%20Hadoop%20and%20Flume%20Presentation.pdf. |
| Manku, et al., “Approximate Frequency Counts over Data Streams”, Proceedings of the 28th VLDB Conference (28th VLDB), pp. 346-357, Aug. 2002, Morgan Kaufmann, San Francisco, CA. |
| Final Office Action U.S. Appl. No. 13/117,371, May 22, 2014 pp. 1-21. |
| Office Action U.S. Appl. No. 13/166,470, Jul. 21, 2014 pp. 1-24. |
| Office Action U.S. Appl. No. 13/275,530, May 23, 2014 pp 1-22. |
| Office Action U.S. Appl. No. 13/567,601, May 23, 2014 pp. 1-34. |
| Final Office Action U.S. Appl. No. 13/688,603, Jun. 26, 2014 pp. 1-17. |
| Office Action U.S. Appl. No. 13/275,487, Feb. 25, 2014 pp. 1-40. |
| Office Action U.S. Appl. No. 13/710,523, Mar. 3, 2014 pp. 1-38. |
| Office Action U.S. Appl. No. 13/570,819, Apr. 30, 2014 pp. 1-20. |
| Zhao, Q., et al., “Dynamic Memory Optimization using Pool Allocation and Prefetching”, ACM SIGARCH Computer Architecture News , Dec. 2005, pp. 27-32, vol. 33, No. 5, ACM, New York, NY, USA. |
| Notice of Allowance U.S. Appl. No. 12/960,990, Aug. 26, 2013 pp. 1-9. |
| Office Action U.S. Appl. No. 12/962,265, Sep. 20, 2013 pp. 1-18. |
| Final Office Action U.S. Appl. No. 12/961,687, Oct. 7, 2013 pp. 1-17. |
| Office Action U.S. Appl. No. 13/117,371, Sep. 27, 2013 pp. 1-18. |
| Notice of Allowance U.S. Appl. No. 13/114,463, Oct. 10, 2013 pp. 1-15. |
| Final Office Action U.S. Appl. No. 13/117,341 Jul. 16, 2013 pp. 1-16. |
| Notice of Allowance U.S. Appl. No. 13/117,341 Oct. 25, 2013 pp. 1-10. |
| Final Office Action U.S. Appl. No. 13/166,470 Oct. 16, 2013 pp. 1-22. |
| Final Office Action U.S. Appl. No. 13/166,027 Sep. 26, 2013 pp. 1-17. |
| Office Action U.S. Appl. No. 13/166,027, May 3, 2013 pp. 1-15. |
| Office Action U.S. Appl. No. 13/166,397, Jul. 2, 2013 pp. 1-19. |
| Notice of Allowance U.S. Appl. No. 13/116,382, Sep. 16, 2013 pp. 1-9. |
| Office Action U.S. Appl. No. 13/116,382 May 9, 2013 pp. 1-15. |
| Office Action U.S. Appl. No. 13/282,995, Jul. 17, 2013 pp. 1-16. |
| Notice of Allowance U.S. Appl. No. 13/275,467, Aug. 23, 2013 pp. 1-14. |
| Office Action U.S. Appl. No. 13/275,487, Aug. 16, 2013 pp. 1-39. |
| Notice of Allowance U.S. Appl. No. 13/275,500, Aug. 7, 2013 pp. 1-14. |
| Office Action U.S. Appl. No. 13/275,500, May 1, 2013 pp. 1-10. |
| Final Office Action U.S. Appl. No. 13/275,530, Jul. 25, 2013 pp. 1-19. |
| Office Action U.S. Appl. No. 13/663,031, Sep. 20, 2013 pp. 1-17. |
| Notice of Allowance U.S. Appl. No. 16/661,930, Aug. 27, 2013 pp. 1-11. |
| Office Action U.S. Appl. No. 13/676,405, Aug. 30, 2013 pp. 1-29. |
| Notice of Allowance U.S. Appl. No. 13/672,966, Oct. 1, 2013 pp. 1-12. |
| Office Action 13/688,603, Sep. 27, 2013 pp. 1-13. |
| Final Office Action U.S. Appl. No. 13/677,970, Aug. 16, 2013 pp. 1-15. |
| Office Action U.S. Appl. No. 13/672,044, Jul. 5, 2013 pp. 1-17. |
| Notice of Allowance U.S. Appl. No. 13/706,574, Jun. 4, 2013 pp. 1-8. |
| Notice of Allowance U.S. Appl. No. 13/742,325, Aug. 28, 2013 pp. 1-10. |
| Office Action U.S. Appl. No. 13/742,325 Apr. 25, 2013 pp. 1-12. |
| Office Action U.S. Appl. No. 13/710,523 Aug. 20, 2013 pp. 1-29. |
| Notice of Allowance U.S. Appl. No. 13/738,043, Oct. 28, 2013 pp. 1-11. |
| Office Action U.S. Appl. No. 13/738,043, Sep. 5, 2013 pp. 1-10. |
| Final Office Action U.S. Appl. No. 13/708,061 Jul. 25, 2013 pages. 1-26. |
| Office Action U.S. Appl. No. 13/747,895, Oct. 11, 2013 pp. 1-14. |
| Office Action U.S. Appl. No. 13/776,823, Oct. 11, 2013 pp. 1-12. |
| Number | Date | Country | |
|---|---|---|---|
| 20150058657 A1 | Feb 2015 | US |
