The present disclosure relates to an event processing apparatus, an event processing method, and an event processing program for processing an event detected from collected data.
Monitoring and analysis solutions for automatically detecting an important phenomenon (hereinafter, event) using data collected through a number of monitoring cameras and various sensors have been developed for the purpose of safety management and security of public spaces and facilities, and are being introduced in many countries in the world. In the field of the safety management and security, there is a requirement for automatically detecting both types of events, one type of event is an event about which the situation may be recognized from a short-time single scene such as intrusion into an off-limits facility and fire, and the other type of event is an event about which the situation may be recognized by observing the progress of circumstances and scale, such as baggage theft and demonstration march.
The event that can be recognized on the basis of a continuous state or movement of a short time, which can be acquired with a sensor including such as a monitoring camera or a microphone, can be detected by a recognition system that operates according to information supplied from each sensor. For example, regarding the detection of fire, a technology is known, that in the case where a smoke detection sensor is already provided in the interior of each building, the sensor device may be set up as a system that recognizes an event indicating the possibility of occurrence of fire, upon sensing smoke. In addition, PTL 1 discloses a technology of distinguishing between carrying away and leaving of a baggage using images of a monitoring camera, and detecting either event.
Further, a technology called complex event processing (hereinafter abbreviated as CEP) is known that includes detecting a specific complex event by combining a plurality of events recognized on the basis of data acquired through one or a plurality of sensors. A system, that employs general CEP technology, such as the one according to NPL 1 stores a rule that prescribes one or more conditions to be satisfied, according to the case where the order of input data is limited, and where the order of input data is not limited. When certain conditions in the stored rule are satisfied, the system executes a state transition, and applies the rule to the input data until reaching a finished state (where all conditions are satisfied). With the CEP designed like above, the result is provides only after all the conditions prescribed in a unit of rules are satisfied.
PTL 1: International Publication No. WO2010/134241
NPL 1: “Esper-Complex Event Processing”, [online], EsperTech, [Retrieved on Jun. 17, 2013], Internet (URL: http://esper.codehaus.org/)
NPL 2: “Face detection/face comparison engine NeoFace”, [online], NEC, [Retrieved on Jun. 17, 2013], Internet (URL: http://jpn.nec.com/face/)
With the generally employed CEP, conditions that are not expressly defined in the rule are not evaluated. Accordingly, it is difficult to handle a case that the situation of an event makes further progress after detecting the event and providing result of the event. For example, the CEP is able to detect an event of leaving of baggage, when someone leaves a suspicious baggage in a facility. Such an event can be detected by a monitoring camera and an image recognition system.
On the other hand, in the case where another person carries away the baggage that has been left, it is possible that baggage theft has been committed. In such a case, both of the leaving and the carrying away of the baggage can be detected, with the image recognition system (which can be realized for example according to PTL 1) configured to respectively detect the carrying away and the leaving of the baggage. Further, the baggage theft can also be detected if the persons who left the baggage and who carried away the baggage can be determined as different persons, through comparing the features of those persons, with the image recognition system, that is configured to identify a person on the basis of the feature of his/her face, such like, for example, the product according to NPL 2. By specifying such series of condition decision process as a rule, the mentioned events may be automatically detected using the CEP.
Further, in the case where either of the person who left the baggage and the person who carried away the baggage is a person already marked as suspect, or a suspicious character involved with another event, the seriousness of the situation and the measures to be taken may be changed when such other events are associated. However, it is difficult to define in advance the CEP rule that covers the single event about leaving of baggage, events that have occurred before and after the single event, and all the events that has relevance with the single event and includes the relation to surrounding circumstances.
Technologies such as online analytical processing (OLAP) and data mining, designed to discover relevance between data from acquired data set instead of defining in advance all the decision conditions as a rule, are also widely employed. These technologies are, however, designed as batch processing for collectively processing the data accumulated up to a certain time point, rather than stream processing for sequentially processing the data inputted time after time, like the CEP. Therefore such technologies are difficult to be adopted for automatic monitoring that requires immediacy.
Accordingly, an objective of the disclosure is to provide an event processing apparatus, an event processing method, and an event processing program that enable detection of a complex event in which potentially relevant events are combined, while keeping immediacy of notification.
In an aspect, the disclosure provides an event processing apparatus including event detection unit which, when received data satisfies a predetermined condition, detects the data as an event, data storage unit which stores the data received by the event detection unit irrespective of a detection result of the event, relevance information definition storage unit which stores relevance information definition defining relevance between the data, integrated analysis unit which, when data is newly stored in the data storage unit, combines the newly stored data with past data included in the data stored in the data storage unit and a relationship of which with the newly stored data satisfies the relevance information definition, and event display unit which displays the past data and the newly stored data combined with each other.
In another aspect, the disclosure provides an event processing method including detecting, when received data satisfies a predetermined condition, the data as an event, storing the received data in data storage unit irrespective of a detection result of the event, combining, when data is newly stored in the data storage unit, the newly stored data with past data included in the data stored in the data storage unit and a relationship of which with the newly stored data satisfies relevance information definition defining relevance between the data, and displaying the past data and the newly stored data combined with each other.
In still another aspect, the disclosure provides an event processing program configured to cause a computer to execute an event detection process including detecting, when received data satisfies a predetermined condition, the data as an event, and storing the received data in data storage unit irrespective of a detection result of the event, an integrated analysis process including combining, when data is newly stored in the data storage unit, the newly stored data with past data included in the data stored in the data storage unit and a relationship of which with the newly stored data satisfies relevance information definition defining relevance between the data, and a data display process including displaying the past data and the newly stored data combined with each other.
The disclosure enables detection of a complex event in which potentially relevant events are combined, while keeping immediacy of notification.
[Exemplary Embodiment 1]
Hereafter, a first exemplary embodiment of the present disclosure will be described with reference to the drawings.
The event detection means 1 receives data transmitted from an external source. The event detection means 1 detects an event when the received data satisfies a predetermined condition, and transmits the data of the event to the event display means 5. The event detection means 1 also stores the received data in the data storage means 2, irrespective of the detection result of the event.
The data storage means 2 stores therein the data supplied from the event detection means 1.
The integrated analysis means 3 determines the relevance between the data with respect to the data sequentially stored in the data storage means 2, and combines the related data. The integrated analysis means 3 transmits the data combination result to the event display means 5.
The relevance information definition storage means 4 stores therein relevance information definition necessary for the integrated analysis means 3 to determine the relevance between the data and combine the related data.
The event display means 5 is configured to display the event data, and the data combination result, that are transmitted from the event detection means 1 and the integrated analysis means 3, on a screen of a terminal apparatus.
The main memory unit 32 is a main memory constituted of, for example, a random access memory (RAM), and utilized as working area or temporary save area for the data. The input unit 33 includes inputting devices such as a keyboard and a mouse that are used by the user to input data and processing instructions. The output unit 34 may be for example a display apparatus such as an liquid crystal display (LCD) apparatus, or a printing apparatus such as a printer, capable of outputting the data. The communication unit 35 executes input and output of data with peripheral devices through wired or wireless network (information communication network). The auxiliary storage unit 36 may be realized by a hard disk apparatus or other storage device. As shown in
The auxiliary storage unit 36 stores a program for executing the event detection means 1, the data storage means 2, the integrated analysis means 3, the relevance information definition storage means 4, and the event display means 5, that are shown in
The event processing apparatus 30 may also be realized in the form of hardware by implementing a circuit configured by hardware components such as a large scale integration (LSI) in which a program for realizing the functions shown in
The communication unit 35 is connected to the peripheral devices, for transmission and reception of the data. As an example of the peripheral devices, an external storage apparatus 38 may be connected to the event processing device 30 via a network through the communication unit 35. The event detection means 1 may acquire the data stored in the external storage device 38. Further, the event processing apparatus 30 may store the event data supplied by the event display means 5 or the data including combination result of data, in the external storage device 38.
When it is determined that the event is established in the step S20, the event display means 5 displays the event data supplied by the event detection means 1 on the terminal apparatus (step S30). At the same time, the data storage means 2 stores therein the input data received at step S10, irrespective of the result of the decision made at step S20 (step S40).
When the new data is stored in the data storage means 2, the integrated analysis means 3 determines the relevance between the new data and other data stored in the data storage means 2, on the basis of the relevance information definition stored in the relevance information definition storage means 4. The integrated analysis means 3 then combines the data relevant to each other (step S50). The integrated analysis means 3 performs such data combination with respect to each of the events. That is, in the case where the integrated analysis means 3 has determined that the data respectively constituting two events are related with each other, the integrated analysis means 3 combines such two events.
The information stored in the relevance information definition storage means 4, and the determination process performed by the integrated analysis means 3 will now be specifically described.
Regarding the example described hereunder, it will be assumed that the type of detected information of the three items of input data, shown in
The integrated analysis means 3 then determines the relevance between the events in accordance with input data 1 and the input data 2, each regarded as an event, on the basis of the degree of relevance, difference between dates and times of detection, and distance between sensor positions of the respective types of detected information. The integrated analysis means 3 refers to the definition of the degree of relevance stored in the relevance information definition storage means 4 shown in
In the case of input data and input data 2, the types of detected information of the input data 1 and the input data 2 are “person lying down” and “scream”, respectively. And the degree of relevance between those two types of detected information is 0.4 (it will be assumed that the degree of relevance in
The integrated analysis means 3 calculates the difference between the dates and times of detection and the distance between the sensor positions, by measuring a scalar distance expressed numerically. Regarding the date and time of detection, the date and time of detection of the input data 1 and the input data 2 are respectively expressed as “2013/06/14 21:40:14” and “2013/06/14 21:40:26”, and the difference is expressed as 12 seconds, representing the difference in time. Likewise, the integrated analysis means 3 calculates the distance between two positions of the sensors, expressed in latitude and longitude as (34.687549, 135.526848) and (34.687684, 135.526609) respectively, thereby obtaining a value “0.000274492”. Assuming that the threshold of the difference in date and time is set to 60 seconds and the threshold of the difference in sensor position is set to 0.000300, the integrated analysis means 3 determines that the input data 1 and the input data 2 are respectively data of a related event, and combines those data, since the both differences are lower than the respective thresholds.
When the integrated analysis means 3 combines data of another event to the data of the event newly stored in the data storage means 2, the integrated analysis means 3 transmits the combination result to the event display means 5. The event display means 5 then displays the data of the combined events on the screen of the terminal apparatus (step S60). When the input data 3 is additionally stored in data storage means 2, also, the integrated analysis means 3 likewise evaluates the degree of relevance and the difference in date and time of detections and distance of sensor positions, between the types of detected information, of the input data 1 and the input data 3 and the input data 2 and the input data 3. Then the integrated analysis means 3 combines the input data 1 and the input data 3 as related events because the condition is satisfied. In contrast, the degree of relevance between the respective types of detected information (“scream” and a specific utterance “ambulance”) of the input data 2 and the input data 3 is 0.2 as shown in
As described above, an event which is not to be displayed alone may be displayed depending on the degree of relevance with another event. Accordingly, the user of the terminal apparatus can easily recognize the whole aspect of the information detected with respect to the related events.
With the configuration described above, the event processing apparatus according to this exemplary embodiment promptly announces each of the events that have occurred. And in the case where the announced events are determined to be related with each other, the event processing apparatus according to this exemplary embodiment combines those events and sequentially announces the relevance. Further, the event processing apparatus according to this exemplary embodiment can also announce the occurrence of a new complex event composed of the plurality of events already announced, when the predetermined condition is satisfied. Such functions of the event processing apparatus according to this exemplary embodiment enable both immediacy and comprehensiveness of the event detection including combined events to be satisfied.
[Exemplary Embodiment 2]
Hereunder, a second exemplary embodiment of the disclosure will be described.
The duration management means 6 restricts data newly provided to the event detection means 1 from being stored in the data storage means 2, in the case where the new data represents the same event as that represented by the previous data acquired from a transmission source which is the same as the transmission source (for example, a sensor) from which the information source of the new data was acquired. Instead, the duration management means 6 sets a continuation flag (not shown) of the data of the same event that previously occurred, and already stored in the data storage means 2. For example, it is assumed that the property of “continuation flag” is added to the data stored in the data storage means 2, and the value “1” of the continuation flag indicates a state that the flag is set (event is continued), and the value “0” of the continuation flag indicates a state that the flag is unset (event is not continued).
As another example of the case for representing the same event, the duration management means 6 may determine that the data represents the same event when the types of detected information and the sensor positions are the same, in addition to the sensor, in the example shown in
The event processing apparatus 20 according to this exemplary embodiment, with utilizing the duration management means 6, can prevent the data representing the same event from being stored plural times in the data storage means 2, periodically through the event detection means 1, in the case where the event detected by the same sensor is continuously occurring (for example, when an left baggage remains placed for a long time). Therefore, the event processing apparatus 20 prevents the data representing the same event from being combined, as representing a plurality of events, each other. Here, the duration management means 6 may immediately delete the data, of which the value of the continuation flag has been cleared (changed from 1 to 0), from the data storage means 2, when value of the continuation flag is set to 0, unless the data is intended to be subsequently utilized for some purpose.
The comprehensive evaluation means 7 calculates a score of the entire data combined by the integrated analysis means 3. The score is used as information that represents the importance and reliability of the entire data as an event. For example, the comprehensive evaluation means 7 evaluates an event, to which data representing a plurality of events are associated, as an event having a larger scale and a larger impact on the surrounding circumstance. The comprehensive evaluation means 7 may assign a higher score to an event, for example when a larger number of items of data are combined. Otherwise, the comprehensive evaluation means 7 may assign a higher score to an event, for example when a larger number of items of data are associated with each other, because such an event contains a larger amount of circumstantial evidence and highly reliable.
The comprehensive evaluation means 7 may calculate the score, for example, with an equation (1) expressed as follows:
Score=Σ((alert level)×(certainty factor)×(degree of relevance)) (1)
Before applying the equation (1), the comprehensive evaluation means 7 determines the data that represents a primary event, among the data that are subject of the comprehensive evaluation. The comprehensive evaluation means 7 determines such data having a largest value of product of the alert level and the certainty fact, as the data representing the primary event, and regards other related data as the data representing a support event. In the example shown in
With the equation (1), the comprehensive evaluation means 7 calculates the value of “alert level×certainty factor×degree of relevance” with respect to each input data, and the total sum of such values constitutes the score. Here, the degree of relevance refers to the degree of relevance with respect to the primary event, for example expressed as shown in
“Score=(2×0.96×1.0)+(2×0.89×0.4)+(1×0.75×0.3)=2.857”.
The event display means 5 displays the score calculated as above to the user, as value representing the importance and reliability of the primary event, together with the data combination result, through the terminal apparatus. Alternatively, the event display means 5 may display the result of data combination on the terminal apparatus, when the score exceeds a predetermined threshold.
The hardware configuration of the event processing apparatus according to this exemplary embodiment is the same as that shown in
The operation of the event processing apparatus according to this exemplary embodiment will now be described hereunder.
The duration management means 6 determines whether the data newly supplied to the event detection means 1 represents the same event represented by the previous data acquired from the transmission source (for example, a sensor) same as the transmission source of the new data (step S35). In the case where the condition of step S35 is satisfied, the duration management means 6 does not store the newly input data in the data storage means 2, and the process about the data is finished.
The comprehensive evaluation means 7 calculates the score of the entire data combined by the integrated analysis means 3 (step S55).
The event display means 5 displays the combined events and the score (step S60).
The event processing apparatus according to this exemplary embodiment, with utilizing the comprehensive evaluation means 7, enables to quantitatively indicate the importance and reliability of the main event, among the entire data representing the related events. Therefore, the user can utilize the displayed score to decide, for example, whether an immediate action is required or not.
Further, the event processing apparatus according to this exemplary embodiment prevents the data representing the same event from being repeatedly combined redundantly, with utilizing the duration management means 6. Therefore, the event processing apparatus according to this exemplary embodiment can prevent the comprehensive evaluation score from becoming largely deviated from the actual value because of the redundantly combined data.
(1) In the event processing apparatus configured as above, the event detection means may promptly cause the event display means to display the detected event. Such an event processing apparatus enables the immediacy of the notification to be satisfied.
(2) The event processing apparatus may include comprehensive evaluation means (for example, the comprehensive evaluation means 7) which calculates a score indicating the importance and reliability of the entire data, including the newly stored data and the past data that are combined by the integrated analysis means. In addition, in the event processing apparatus, event display means may be configured to displays the score. The event processing apparatus configured as above enables the importance and reliability of the main event to be quantitatively indicated, on the basis of the entire data representing the related events. Therefore, the user can utilize the displayed score to decide, for example, whether an immediate action is required or not.
(3) In the event processing apparatus, the event display means may be configured to display the entire data combined by the integrated analysis means, only when the score calculated by the comprehensive evaluation means exceeds a predetermined threshold.
(4) The event processing apparatus may further include duration management means (for example, the duration management means 6) which determines whether an event detected from new data received by the event detection means is the same event as that represented by the previous data acquired from the same transmission source, and restricts the new data from being stored in the data storage means, when the new data is determined to be the same. Therefore, the event processing apparatus prevents the data representing the same event from being combined as representing a plurality of events.
(5) In the event processing apparatus, the duration management means may set a continuation flag of an event detected from the newly received data, when the event detected from the newly received data is the same event as the one detected from the previous data acquired from the same transmission source, and clear the continuation flag when an event different from the previous one is detected from the data acquired from the transmission source. Further, in the event processing apparatus the integrated analysis means may determine continuation and finish of an event on the basis of the continuation flag.
The present invention has been described with reference to the exemplary embodiments, the present invention is not limited to the foregoing exemplary embodiments. Various modifications obvious to those skilled in the art may be made to the configurations and specific details of the present invention, within the scope of the present invention.
This application claims priority based on Japanese Patent Application No. 2013-144775 filed on Jul. 10, 2013, the entire content of which is incorporated hereinto by reference.
The present disclosure is suitably applicable to event processing apparatuses that are required to promptly detect and notify an event that has occurred, and to associate partial events with each other and notify the occurrence of the event when a predetermined condition is satisfied, even when the occurrence of the event is unable to be immediately detected with a single item of, or a few specific items of sensor information.
1 Event detection means
2 Data storage means
3 Integrated analysis means
4 Relevance information definition storage means
5 Event display means
6 Duration management means
7 Comprehensive evaluation means
10, 20, 30 Event processing apparatus
31 CPU
32 Main memory unit
33 Input unit
34 Output unit
35 Communication unit
36 Auxiliary storage unit
37 System bus
38 External storage apparatus
41 Event detection unit
42 Data storage unit
43 Integrated analysis unit
44 Relevance information definition storage unit
45 Event display unit
Number | Date | Country | Kind |
---|---|---|---|
2013-144775 | Jul 2013 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2014/003243 | 6/17/2014 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2015/004854 | 1/15/2015 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
6825761 | Christ | Nov 2004 | B2 |
20050088295 | Kondo et al. | Apr 2005 | A1 |
20090281839 | Lynn | Nov 2009 | A1 |
20120063641 | Venkatesh et al. | Mar 2012 | A1 |
20120215908 | Tsunoo et al. | Aug 2012 | A1 |
20130067572 | Muramoto | Mar 2013 | A1 |
20130346016 | Suzuki et al. | Dec 2013 | A1 |
20160275776 | Shen | Sep 2016 | A1 |
20160322078 | Bose | Nov 2016 | A1 |
Number | Date | Country |
---|---|---|
2010230857 | Oct 2011 | AU |
1598888 | Mar 2005 | CN |
1983342 | Jun 2007 | CN |
102449660 | May 2012 | CN |
103117884 | May 2013 | CN |
2 415 022 | Feb 2012 | EP |
2 687 998 | Jan 2014 | EP |
2005-065149 | Mar 2005 | JP |
2009-199356 | Sep 2009 | JP |
2012-088855 | May 2012 | JP |
2012-523025 | Sep 2012 | JP |
2013-061794 | Apr 2013 | JP |
20050020712 | Mar 2005 | KR |
174519 | Oct 2011 | SG |
WO 2010111748 | Oct 2010 | WO |
WO 2010134241 | Nov 2010 | WO |
WO 2012111144 | Aug 2012 | WO |
WO 2012124259 | Sep 2012 | WO |
Entry |
---|
International Search Report (ISR) (PCT Form PCT/ISA/210), in PCT/JP2014/003243, dated Aug. 26, 2014. |
“Esper—Complex Event Processing”, [online], EsperTech, [Retrieved on Jun. 17, 2013], Internet (URL: http://esper.codehaus.org/). |
“Face detection/face comparison engine NeoFace”, [online], NEC, [Retrieved on Jun. 17, 2013], Internet (URL: http://jpn.nec.com/face/). |
Number | Date | Country | |
---|---|---|---|
20160163172 A1 | Jun 2016 | US |