Patent Application
20050259149
The present invention relates generally to processing video signals and acoustic signals, and more particularly to augmenting video signals with the acoustic signals in surveillance systems.
In a typical surveillance system, a user, typically a security guard, monitors various locations via monitors connected to cameras. Visual monitoring of the locations provides information about activities at the locations, e.g., the movement of people and vehicles, and conditions of the environment.
In order to perform adequate surveillance and to respond to significant activities, the user typically needs to see some motion at the location. However, that method of surveillance monitoring can be inadequate in various situations.
Due to economic constraints, such systems have a limited range of view of each location because the cameras are either focused at a fixed location, or swivel along predetermined arcs. That can result in ‘blind’ spots, which can cause a misinterpretation, intentionally or unintentionally, of what is happening at a particular location. In addition, just the visual information by itself may not convey sufficient information to trigger intervention in response to unusual events.
To further illustrate the shortcomings of conventional surveillance systems, consider a few examples: A camera is located in a corridor outside an electrical service room. A minor explosion occurs in a transformer in the room. Visual cues are not available until smoke and flames spread from the room to the corridor. At that point, an alert may be too late. Similarly, a camera monitoring a parking lot at night, under snowy conditions, may be unable to detect a break-in or assault.
It is also possible that a camera is deliberately tampered with, making it useless for its intended purpose.
In all of the above examples, additional information, such as audio signals acquired by a microphone near a camera, could alert the user. That solution could suffice for a surveillance system with a single camera. However, for a system with many cameras, for example, tens or hundreds, being monitored by fewer users than cameras, instead of enhancing the surveillance, the multiple overlapping audio signals would result in nothing but an undecipherable cacophony.
Therefore, there is a need for augmenting video signals with acoustic signals that enhance the video signals.
A system and method according to the invention includes visualizing acoustic activity, or its detected features, and displaying the acoustic activity as visual signals alongside videos acquired by cameras.
Either through use of simple signal processing, or through use of more sophisticated audio analysis or sound recognition, useful information can be extracted from acoustic signals. The extracted information can be transformed to visual signals superimposed on the video signals acquired by the cameras.
System Structure
The audio signals 121, from each microphone, are analyzed and transformed by, e.g., a sound recognition module 130, to a visual signal 131. The visual signals 131 is combined with the video signals 111, and displayed on a corresponding monitor 140, to be viewed by a user 150. The visual signal can alter a brightness or color of the display, as indicated by shading in
System Operation
Sound Energy Visualization
In many cases, the level of energy of the acoustic signal is sufficient to indicate an unusual event at a location. Take the case of a secure corridor, as shown in
Specific Sound Detection
It is also possible to train the analysis and transformation module 130 to detect and identify specific acoustic signals, such as doors opening and closing, screams, foot steps, running, etc. Identified acoustic signals can be displayed visually as an intensity level on a monitor, as an icon, or as text. The color of the display can also change from a normal gray scale, to a display that is colored red or yellow.
Spatial Information Visualization
By using an array of microphones, it is possible to perform sound localization to assist the user, as shown in
Effect of the Invention
A system and method visually represents acoustic signals alongside video signals. The acoustic signals are analyzed and transformed to visual signals, which can be superimposed or otherwise displayed along with the video signals. The invention does not require extensive alterations of surveillance systems because most modern cameras are equipped with microphones.
Although the invention has been described by way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.
