Patent Application
20120134540
The present invention claims priority of Korean Patent Application Nos. 10-2010-0120213, filed on Nov. 30, 2010, which is incorporated herein by reference.
The present invention generally relates to a technique for detecting events in an intelligent monitoring system; and, more particularly, to an apparatus and method for creating a surveillance image with event-related information, and an apparatus and method for recognizing an event from the surveillance image by using a supplemental enhancement information (SEI) field.
As is well known, event information, extracted from an image recognition unit or an event detection module of an intelligent monitoring system or intelligent security system, has been used to previously detect and defends various types of dangerous situations, such as the invasion of restricted areas by an outsider, stolen valuables, vehicle accidents, child kidnapping, fire alarms, and flooding, which may occur in the living environment.
Referring to
The encoding unit 104 encodes the received surveillance image in conformity with a preset encoding format, for example, the H.264 video format or the Moving
Picture Experts Group (MPEG) video format. The encoded surveillance image is transmitted to the integrated management server 110 in the form of a bit stream via a transmission channel CH1.
The event information creation unit 106 allows event-related information to store in the event DB 108. Further, the event information creation unit 106 converts the event-related information into the form of a header file or metadata and transmits the converted event-related information along with the bit stream of the encoded surveillance image to the integrated management server 110 via the channel CH1, or transmits them to the integrated management server 110 via a separate transmission channel CH2.
The integrated management server 110 searches for event-related information by performing the conversion of the event-related information and the decoding of the bit stream, recognizes situations related to an event using the event-related information stored in the DB.
As described above, however, the conventional scheme of storing and searching for event-related information has the problem of requiring a separate operating scheme capable of managing metadata because event information extracted from images is included in a separate specific header file and then stored in a storage device and the event information is generated in the form of meta data by combining various types of information. In addition, the conventional scheme has the problem of requiring a large amount of expense and time in order to construct such a system because an integrated management server also should construct an additional operating scheme.
Further, in order to extract the event-related information or to insert additional information required by a user into the event-related information, a format of encoded surveillance image should be converted into another format and only images or files in conformity with a specific compression scheme should be considered to achieve it.
In view of the above, the present invention provides an apparatus and method for creating a surveillance image with event-related information, and an apparatus and method for recognizing an event from the surveillance image by using a supplemental enhancement information (SEI) field.
In accordance with a first aspect of the present invention, there is provided an apparatus for creating a surveillance image with event-related information, the apparatus including:
an event detection unit configured to detect an event in the surveillance image;
an encoding unit configured to encode the surveillance image into a bit stream of the surveillance image;
an event information creation unit configured to create event-related information based on the detected event; and
a parsing unit configured to parse the encoded surveillance image and insert the event-related information into the bit stream of the encoded surveillance image.
In accordance with a second aspect of the present invention, there is provided a method for creating a surveillance image with event-related information, the method including:
detecting an event in a surveillance image using an intelligent image recognition technique;
encoding the surveillance image to produce a bit stream of an encoded surveillance image;
parsing the encoded surveillance image; and
inserting event-related information corresponding to the event into the bit stream of the encoded surveillance image.
In accordance with a third aspect of the present invention, there is provided an apparatus for recognizing an event form a surveillance image, the apparatus including:
a parsing unit configured to parse the surveillance image to determine whether or not there exists a Supplemental Enhancement Information (SEI) field in the surveillance image;
an event recognition unit configured to recognize the event in the surveillance image based on the SEI; and
a display control unit configured to display event-related information corresponding to the event on a monitor together with the surveillance image.
In accordance with a fourth aspect of the present invention, there is provided a method for recognizing an event from a surveillance image, the method including:
receiving a bit stream of a surveillance image;
checking whether or not there exists a SEI field having event-related information in the surveillance image;
when the SEI field exists, extracting the event-related information from the SEI field
to recognize the event that has been detected in the surveillance image based on the event-related information.
The above and other objects and features of the present invention will become apparent from the following description of preferred embodiments given in conjunction with the accompanying drawings, in which:
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that they can be readily implemented by those skilled in the art.
The apparatus 200 of the embodiment of the present invention includes an event detection unit 202, an encoding unit 204, an event information creation unit 206, an event conversion table 208, and a parsing unit 210.
A surveillance image may be obtained at a target area for surveillance using a video camera or DVR (not shown). The event detection unit 202, upon receipt of the surveillance image, detects event(s) from the surveillance image by performing on the surveillance image an image recognition technique, for example, a background modeling algorithm, an object detection and chase algorithm, an object feature parameters extraction algorithm and so on. etc.
The term ‘event(s)’ used herein refers to various types of dangerous situations, for example, such as the invasion of a restricted area by an outsider, stolen valuables, vehicle accidents, child kidnapping, fire alarms, and flooding, that may occur in a life environment.
Information on the event is transmitted to the event information creation unit 206, and the surveillance image is provided to the encoding unit 204.
The encoding unit 204 encodes the surveillance image in conformity with a preset encoding format, for example, the H.264 video format or the MPEG video format, and delivers the encoded surveillance image in the form of a bit stream to the parsing unit 210.
The event information creation unit 206 classifies the event(s) detected by the event detection unit 202 into detailed items, for example, a type of event, a danger level of event, etc based on the property of the event(s), and converts the classified event(s) into event-related information with reference to an event conversion table 208.
For example, an event, “invasion of a restricted area”, may be converted into a tag of 0×AA by the event information creation unit 206 with reference to the event conversion table 208.
The event conversion table may be configured as shown in
In addition, the event information creation unit 206 may encrypt the event-related information before providing it to the parsing unit 208 such that the confidentiality of the event-related information can be secured. For this purpose, the event information creation unit 206 may include an encryption algorithm or encoder well-known in this technical field.
The parsing unit 210 parses the encoded surveillance image from the encoding unit 204, and inserts the event-related information from the event information creation unit 206 into the surveillance image. In accordance with the present invention, the event-related information is inserted into a Supplemental Enhancement Information (SEI) field in the bit stream of the encoded surveillance image, thereby creating the surveillance image with the event-related information.
For H.264/MPEG-4 AVC, the Supplemental Enhancement Information (SEI) in an image frame is extra information that can be inserted into a bit stream of the image frame to enhance the use of the video for a wide variety of purposes.
For example, a bit stream of the event surveillance image may be configured as in
After inserting the event-related information, the parsing unit 210 transmits the surveillance image with the event-related information to an apparatus for recognizing an event from a surveillance image, which will be described with reference with
The apparatus 300 receives a surveillance image from the apparatus 200 of
The apparatus 300 includes a parsing unit 304, an event recognition unit 306, an event conversion table 308, a decoding unit 310, and a display control unit 312.
The surveillance image is provided to both the parsing unit 304 and the decoding unit 310.
The parsing unit 304 parses the surveillance image to determine whether or not there exists SEI field in the surveillance image. When it is determined that there exists the SEI field, the parsing unit 304 checks whether a flag has, for example, a value of “0×11” in the SEI field. When the flag is “0×11”, the parsing unit 304 stores data following the flag within the SEI field in a temporary variable. Thereafter, the parsing unit 304 extracts the hash value from the temporary variable to verify the integrity of the event-related information. When it is determined that the integrity is successfully verified, the parsing unit 304 delivers the event-related information to the event recognition unit 306 together with an integrity success instruction indicating the successful verification for the integrity.
In this case, the event-related information extracted from the SEI field might have been encrypted information in order for securing the confidentiality thereof. In order to manage such a case, the parsing unit 304 may include a decryption algorithm or a decoder capable of decrypting the encrypted event-related information back into original event-related information before encryption thereof.
The event recognition unit 306, upon receipt of the integrity success instruction, recognizes what event has been occurred by performing the conversion of the event-related information with reference to the event conversion table 308, and provides the recognized result to the display control unit 312 together with the event-related information.
In this regard, the event conversion table 308 is the same as the event conversion table 208 of
The decoding unit 310 extracts a surveillance image from the event surveillance image and provides the surveillance image to the display control unit 312.
Finally, the display control unit 312 combines the decoded surveillance image with the event-related information to generate a combined surveillance image in which the event-related information is combined with the surveillance image. The combined surveillance image may be provided to a monitor (not shown) that the event occurrence or warning expression is presented to a manager.
Therefore, the apparatus for recognizing event-related information in accordance with the present invention may effectively recognize what event has been occurred based on the event-related information in the SEI field of the event surveillance image.
Meanwhile, although the detailed description of the present invention has been made such that the apparatus 200 of
Referring to
At step 406, it is determined whether or not any event is detected in the surveillance image. If it is determined that none event is detected, the process goes to step 408, and if it is determined that any event is detected, the process then advances to step 410.
At step 408, the surveillance image is provided to the encoding unit 204 where the surveillance image without the event-related information is encoded. Thereafter, at step 416, the encoded surveillance image without the event-related information will be transmitted to the apparatus of
Meanwhile, at step 410, the event information creation unit 206 classifies the detected event into detailed items, such as a type of event, danger level and so on based on the properties of the event, converts information on the event into an event-related information with reference to the event conversion table 208, and then provides the event-related information to the parsing unit 210.
At step 412, the encoding unit 204 encodes the surveillance image in which the event has been detected. The encoded surveillance image from the encoding unit 204 is then provided to the parsing unit 210.
Next, at step 414, the parsing unit 210 parses the encoded surveillance image and inserts the event-related information from the event information creation unit 208 into a SEI field in the bit stream of the encoded surveillance image, thereby generating the surveillance image with the event-related information. Subsequently, at step 416, the surveillance image with the event-related image will be transmitted to the apparatus 300 of
First of all, at step 502, a surveillance image with/without event-realted information is received by the apparatus 300 shown in
Next, the parsing unit 304 parses the surveillance image at step 504 and determines whether or not an SEI field exists in the bit stream of the surveillance image at step 506.
If the SEI field does not exist in the surveillance image, the process advances to step 514 where the decoding unit 310 decodes the surveillance image back into an original surveillance image before the encoding thereof, and then delivers the decoded surveillance image to the display control unit 312. Thereafter, at step 524, the display control unit 312 provides the decoded surveillance image to a monitor so that the surveillance image without the event-related information is displayed on the monitor.
If, however, it is determined that the SEI field exists in the surveillance image, the process goes to step 508. In step 508, the parsing unit 304 checks whether the value of a flag in the SEI field is, for example, “0×11”.
If, as the result of the check at step 508, it is checked that the value of the flag is “0×11”, the process goes to step 510. In step 510, the parsing unit 304 stores the event-related information in a temporary variable, and verifies the integrity of the event-related information using the hash value stored in the temporary variable.
Thereafter, at step 512, the parsing unit 304 determines whether or not the integrity is successfully verified. If it is determined that the verification of the integrity failed, the process goes to above-described step 514.
If, however, it is determined that the integrity was successfully verified, the process proceeds to step 516 in which the parsing unit 304 notifies the event recognition unit 306 of the successful verification.
Subsequently, at step 518, the event recognition unit 306 recognizes an event that has been occurred in the surveillance image using the event-related information by performing the conversion of the event-related information with reference to the event conversion table 308, and then provided the recognized result to the display control unit 312 along with the event-related information.
Meanwhile, at step 520, the decoding unit 310 decodes the surveillance image with the event-related information and then transmits the decoded surveillance image to the display control unit 312.
After that, at step 522, the display control unit 312 combines the decoded surveillance image with the recognized event to thereby generate the surveillance image with an event-related message, and then transmits the surveillance image with the event-related message to the monitor.
Accordingly, at step 524, the monitor displays the surveillance image with the event-related message at step 524.
In this regard, the event-related message may include information about the event occurrence or warning expression.
As described above, in accordance with the present invention, event-related information is inserted into a SEI field of a surveillance image, without using an additional header file or metadata, thereby realizing an epoch-making reduction in the expense and time required to construct an intelligent monitoring system.
Further, the present invention may also be applied to existing security systems by performing a minimum of modifications on a program without using an additional operating system or database, thereby reducing the time and expense required to implement the security system.
While the invention has been shown and described with respect to the preferred embodiment, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2010-0120213 | Nov 2010 | KR | national |
