This invention relates generally to video monitoring, and, more particularly, to such systems employing means for digitizing camera images for display at a first image size, sampling rate, and frame rate, and for digital storage at a second image size, sampling rate, and frame rate.
Existing systems for monitoring video signals from multiple sources either have relied upon switching means to sequence through each of the sources in a predetermined pattern, or else implement some form of “split screen” display, in which a conventional video monitor displays, for example, the output of four cameras, each occupying a portion of the display screen. Often, the video output is also supplied to signal storage means such as a VCR which is operated in a single-frame recording mode, providing a time-lapse record of the camera outputs. These types of systems are in common usage, and are well known in the art.
More recently, the availability of digital techniques for video signal processing and data compression has opened new video monitoring alternatives. Mathisen, for example, U.S. Pat. No. 4,198,656, describes a system employing a signal selector which is controlled by signal storage means, including a VCR functioning as a time-lapse frame recorder, and disc storage means functioning as a frame-store type device which provides continuity in the display while the selector is advancing through its sequence to the next signal source, or to the next recorded signal from one particular source. Each image is further provided with a digital code, which enables the image source to be identified upon playback.
Katz (U.S. Pat. No. 5,216,502) discloses a system for monitoring and recording transactions at multiple cashier lanes. According to the preferred embodiment, output signals from four cameras are fed to a four-quadrant multiplexer, which contains a frame store having provisions for reducing each camera image to one-quarter size and then displaying it in one of the four quadrants of the video monitor. The combined output signal is then recorded on a Video Cassette Recorder (VCR) to provide a more permanent record of the transaction. No description is given of the size-reduction method, but this kind of display may be implemented by employing a time-compression scheme for the horizontal lines, and eliminating alternate scan lines from the vertical dimension of the video frame. Data from the individual cash registers is encoded digitally, and then either recorded on lines of video in the vertical-interval or else recorded on the audio tracks of the VCR. The monitoring provisions include a video overlay generator, so that transaction information may be displayed concurrently with images of the event. Provisions are also included for selective recording of representative frames of video, as triggered by transaction events at individual cashier lanes. This further reduces the number of frames recorded, and extends the recording duration of the video cassette.
Neither of these two inventions disclose the use of image data compression schemes, nor of image resizing by digital means. Katz does disclose selective recording of frames of video, but only in analog form, and using conventional video recording means, such as a VCR.
Blum et al. (U.S. Pat. No. 5,237,408) disclose a system employing multiple interface circuit boards containing provisions for capturing and digitizing images from a plurality of cameras, further storing a slow-scan sequence of images for each camera in active buffer memory means as RAM on each of the interface boards. The display of these image sequences is preferably controlled by a PC with disk storage capabilities, and individual images selectively may be recorded on the hard disk, under operator control. Storage capacity is limited, however, because no image data compression is provided, and it is not practical to store large quantities or sequences of images. There are no provisions for automatic recording of images, nor any provision for storing these images in a data form differing from the display format image size.
Gormley (U.S. Pat. No. 5,258,837) discloses video compression means as a method for fitting multiple camera images within a single video display screen. However, Gormley clearly indicates that the system relies on fixed image-size compression of the digitized incoming video signal, and does not perform any kind of bandwidth compression. Provisions are included for storing images on a video recorder, but, as in other systems, these images are stored by taking the screen display as a whole, rather than storing independently constituted representations of the individual camera outputs.
In general, these conventional systems, regardless of the subsequent signal processing employed, accept analog video signals from the camera sources for input to the monitoring system. As such, these signals are susceptible to disturbances such as RF interference, ground-loop noise, and high-frequency signal loss due to long runs of coaxial cables. Automatic video switcher units sequence the source signals at a relatively slow rate, requiring as much as 16 seconds delay between successive viewings of a particular source when using a one source-per-second rate to sequence through 16 sources. When such a system is equipped with a time-lapse video recorder, the recording medium is expensive, has low resolution, and is limited in recording capacity.
Other systems, such as video conferencing arrangements, use multiple camera images and audio sources which are coordinated through electronic switching means and image display means, with interconnections between conferencing rooms locally or at remote sites via telephone lines or other communications links. Fabris et al. (U.S. Pat. No. 4,516,156) discloses a sophisticated implementation of such a system, employing satellite links for long-distance communications, a full range of camera remote controls (including lens controls and pan-tilt-head controls), and touch-screen monitor facilities for system controls of cameras and signal switching. This system, however, has no provisions for image storage, or any application of image data compression means coupled with display means or image storage means.
The present invention implements an automated video monitoring system by way of a PC-based platform employing display windowing software, with camera sources being interfaced to an input circuit board which includes provisions for image data compression. Using a basic image size in pixels of 320×240, and optionally including color processing employing a Y-U-V 4:1:1 or 4:2:2 sampling technique, a range of performance standards are established. An economical simultaneous display of 4 sources in a 2×2 configuration on a conventional 10″ VGA-format (640×480 pixels) monitor may be upgraded to a more elaborate 24-source (plus one utility window, which also may be used as a graphical-based input source device for transmitting control commands to the individual cameras) display in a 5×5 configuration on a high-resolution 20″ (1600×1200 pixels) monitor. Other combinations are possible, including arrays of images of size 640×480 pixels or even 800×600 pixels, depending on the screen size of the display monitor and the capabilities of the video display adapter circuit card. In addition, not all image windows need to be of the same size, nor updated at the same rate, but rather they may be mixed and combined based on particular applications. Remote controls for the individual cameras may be implemented by way of windowing software and/or monitor touch-screen devices. Automatic sensing of particular events (representing security alarms, equipment or process disturbances, or a change in the person speaking in a videoconferencing environment) may be employed to cause reconfiguration by way of resizing the image or modifying the update rate of individual windows on the display screen, or by modifying the data format of recording images on a storage device.
Storage of images may be implemented by way of a tape back-up device, such as a DAT or 8-mm tape recorder, which are capable of storing as much as 960 hours of monitoring images, or by way of disk storage devices, preferably including removable disk drives such as magneto-optical disks or PCMCIA-compatible disk-drive modules. Images are preferably stored as a succession of data-compressed representations, corresponding to various window sizes sampled at diverse update rates; however, though the image representations need not be identical to the sizes and rates used for video monitors displaying the various images. In an alternative embodiment, cameras are provided with data compression means at the camera location, with a bi-directional data link providing control signals from the PC, and accepting data-compressed images from the cameras. Depending on the data rates selected, the transmission of these signals may be by means of coaxial cables, twisted-pair lines, or fiber-optic cables. These signals may alternatively be transmitted over internal PBX telephone lines, or over greater distances, including satellite links, as would be the case for remote monitoring of facilities or videoconferencing. The use of digital images allows the application of various useful techniques, such as digital noise reduction and motion-sensing, to trigger security alarms, for example.
It is an object of the invention to provide a more efficient method for monitoring camera outputs by means of a multiple-window display system implemented on a computer platform.
It is another object of the invention to provide an improved recording system for event-logging or security applications, by means of storing data-compressed digital images, including identifying information, which are more representative of events than typical analog time-lapse recordings, and further to implement this system on a computer platform.
It is a further object of the invention to provide extended recording time for event-logging or security applications by means of either tape-based, or disk-based, or both tape-based and disk-based data recording means.
It is yet another object of the invention to provide a system for remote monitoring of cameras by means of transmission of data-compressed digital images over communication links.
It is yet a further object of the invention to provide a convenient method of implementing videoconferencing facilities by means of a PC-based platform.
The present invention implements an automated video monitoring system by way of a PC-based platform employing display windowing software, with camera sources being interfaced to an input circuit board which includes provisions for image data compression. The basic video window size of 320×240 pixels from each camera source can be displayed on a variety of video monitors, in a number of formats, depending on system complexity. The preferred recording medium is a 4-mm helical-scan data cartridge, commonly referred to as a digital audio tape (DAT). Each tape cartridge is capable of storing 10 GB (gigabytes) of data. All recording times in the explanation below are based a data-compression ratio of 100:1, utilizing a 4:2:2 Y/U/V sampling method for color images. Other higher capacity media, such as 8-mm tapes capable of 20 GB of data storage, may be employed when longer times are desired.
In any of the display configurations just described, it is also possible to use a video overlay technique to display the utility controls superimposed on the camera video. Regardless of which display or recording format is used, the PC based monitoring system includes various facilities and features, which will now be described in further detail.
In operation, the user need only observe the various display windows on the monitor screen, rather than concentrate on many monitors simultaneously, thereby reducing the risk of missing an important event. Since no video switcher is used in recording, and images from all sources are continuously recorded at the selected frame rate for each source, more information is recorded than in conventional analog systems, wherein events may be missed due to the sequential switching of input images. The digital format also improves the picture quality, as the signal-to-noise ratio will be higher than for analog systems. In addition, there is no loss of quality during recording or playback, and because the recording technique is digital, other types of information optionally may be recorded along with the camera data, such as audio, time, date, location, etc.
An additional feature is the capability to implement a dual-recording-media option. This facility provides the ability to record simultaneously both on a tape (for high capacity, long term storage) and also on a removable media, such as a removable hard disk (i.e. PCMCIA) or magneto-optical disk (for short-term storage of up to 24 hours of images). These disks facilitate high-speed searching of recorded information without interrupting the tape, as well as providing a back up for the recording on the tape. In addition, the system is capable of simultaneously searching the recorded images on the disk storage unit while continuing to store images on the tape storage unit.
It should be noted that there is no requirement that the image sizes or frame rates utilized for the video display match those utilized for the storage media. In practice, these two specifications may not agree, and will be determined by other factors, such as operator manipulation of the displayed image sizes or changes resulting from the detection of alarm conditions.
By equipping an individual camera input with digital frame store capabilities, it is possible to detect an alarm condition based on deviations from the normal-state image scene. For example, if a camera is monitoring a door exit or an area of a warehouse aisle that is not utilized during evening hours, any change in the image would represent an alarm condition. To allow camera movement, this sensing process would be disabled during pan, scan, tilt, zoom, and other camera positioning controls, and the sensing would be re-enabled after camera positioning ceased, or by manual operator control.
It will be appreciated that in any of these implementations, alarm or sensor signals may be utilized to automatically re-configure the system operating mode, as, for example, increasing the frame rate or image size for an image source associated with the sensor which has initiated the alarm signal condition. As explained above, the displayed windows and image sizes may be reconfigured into an operating mode different from the reconfiguration of the digital storage mode. If desired, the operator may choose to allow the system to automatically adjust the compression ratio utilized for a particular window in response to alarm signal conditions. Alternatively, the compression ratio may be adjusted in response to the selection by the operator of a particular window for closer monitoring, by switching to an image window having larger dimensions in pixels. For some applications, the use of a resolution-independent data compression scheme, such as the “Fractal Compression” method of Iterated Systems, Inc., will be preferred, since images compressed by this method may be resized to fit larger or smaller windows as desired, without loss of apparent resolution. In addition, automatic switching of the audio signals associated with a particular window in response to alarm signal condition will enable the operator to fully investigate such events, and the audio signals may serve to attract the operate's attention to the event. Such switching optionally may be coupled with operator selections for monitoring purposes as well, and in both of these options may be integrated into the system previously described for recording the images, with or without any associated audio signals, onto the mass-storage media.
The application of these PC-based monitoring techniques also may hebe implemented in cases in which the central monitoring area may be located at some distance from the remote site. In this case, the physical hardware and networking facilities will follow the example shown in
The purpose of the main control center is to manage multiple cameras at multiple locations. By monitoring and controlling the remote locations from a central location 220, the system may function without an operator at each location. Optionally, additional computers shown, by way of example, as 222 and 224 at the main location may be implemented to simplify the monitoring and control of the remote locations. The main control center stores the data from remote locations in the main control storage facilities, shown, by way of example, as 238, 240 and 242, using digital tape and removable media, including, by way of example, PCMCIA-based removable media or magneto-optical (MO) media. The images may be stored at various frame rates and resolutions, depending on the requirements for the intended use of the stored images. Intercom provisions as shown as 230, 232 and 234, facilitate the coordination of activities between the main control center and the various remote locations, and video camera units are provided at each of the computers, shown, by way of example, as 244, 246 and 248.
Each remote location includes a PC/monitor 250, with multiple video cameras, shown, by way of example, as 252 and 254, including provisions 256 for image and data storage utilizing digital tape and other removable media. This remote location may be operated under control from either the main location (as described above), or a local operator, or by specific software designed for automatic control. In the case of software-controlled operations, provisions are included for control to be assumed by the main control center, if necessary. As in the case of the main control computers, each of the remote computers is equipped with intercom facilities as 258 and video cameras as 260.
In the traditional videoconferencing situation, only one camera and monitor are used. Unfortunately, this results in an unnatural scene which provides the users with only a single viewing perspective. The use of a wide-angle camera lens which is needed to include many people seated around a table creates a distorted view, especially for those seated furthest from the camera.
Using the PC-based monitoring system, it is possible to create a video conference that presents a much more natural viewing appearance. As shown in
To facilitate the aiming of the individual cameras for the participants, an arrangement such as that implemented at airport gates to assist the parking of airplanes (typically an “I” and an “O” formed of neon light tubes is utilized for airplane gate parking) optionally may be included. In this case, the two indicators are mounted so as to overlap only when viewed directly along the optical axis of the camera lens system. Thus, the camera position need only be adjusted so as to cause the indicators to overlap, thereby assuring that the camera correctly is aimed at the subject. A number of microphones 318 are situated around the conference table in locations suitable for proper audio coverage of the participants' speech.
As shown in
It should be noted that the implementation of the PC-based monitoring system is not limited to these examples of security systems or videoconferencing. Many alternative implementations, such as workplace, factory, production line, and process monitoring, would benefit from this system, and these alternative implementations should be considered to be within the scope of the invention.
This application is a continuation-in-part of U.S. Ser. No. 08/050,861, filed Apr. 21, 1993 now U.S. Pat. No. 5,450,140 and Ser. No. 08/298,104, filed Aug. 30, 1994 now U.S. Pat. No. 5,537,157.
Number | Name | Date | Kind |
---|---|---|---|
2995620 | Burr | Aug 1961 | A |
3617626 | Bluth et al. | Nov 1971 | A |
3811008 | Lee | May 1974 | A |
3935380 | Coutta | Jan 1976 | A |
4051524 | Baxter | Sep 1977 | A |
4074225 | Vandeweghe | Feb 1978 | A |
4120004 | Coutta | Oct 1978 | A |
4138694 | Doi et al. | Feb 1979 | A |
4198656 | Mathisen | Apr 1980 | A |
4237483 | Clever | Dec 1980 | A |
4324401 | Stubben et al. | Apr 1982 | A |
4360827 | Braun | Nov 1982 | A |
4458266 | Mahoney | Jul 1984 | A |
4507683 | Griesshaber et al. | Mar 1985 | A |
4510526 | Coutta et al. | Apr 1985 | A |
4511886 | Rodriguez | Apr 1985 | A |
4516156 | Fabris et al. | May 1985 | A |
4616319 | Peters et al. | Oct 1986 | A |
4618894 | Ichinoi | Oct 1986 | A |
4630110 | Cotton et al. | Dec 1986 | A |
4650929 | Boerger et al. | Mar 1987 | A |
4654703 | Viera | Mar 1987 | A |
4661863 | Ichinoi | Apr 1987 | A |
4663518 | Borror et al. | May 1987 | A |
4673974 | Ito et al. | Jun 1987 | A |
4685003 | Westland | Aug 1987 | A |
4686698 | Tompkins et al. | Aug 1987 | A |
4691248 | Nishimoto | Sep 1987 | A |
4692806 | Anderson et al. | Sep 1987 | A |
4710917 | Tompkins et al. | Dec 1987 | A |
4712103 | Gotanda | Dec 1987 | A |
4729044 | Kiesel | Mar 1988 | A |
4739401 | Sacks et al. | Apr 1988 | A |
4768090 | Camps et al. | Aug 1988 | A |
4772945 | Tagawa et al. | Sep 1988 | A |
4777526 | Saitoh et al. | Oct 1988 | A |
4777527 | Camps et al. | Oct 1988 | A |
4811408 | Goldman | Mar 1989 | A |
4812924 | Fukami et al. | Mar 1989 | A |
4814869 | Oliver, Jr. | Mar 1989 | A |
4823207 | Kobayashi et al. | Apr 1989 | A |
4829389 | Fukuda | May 1989 | A |
4847829 | Tompkins et al. | Jul 1989 | A |
4849486 | Tsuchiya et al. | Jul 1989 | A |
4873573 | Thomas et al. | Oct 1989 | A |
4876597 | Roy et al. | Oct 1989 | A |
4879747 | Leighton et al. | Nov 1989 | A |
4884898 | Magnuson | Dec 1989 | A |
4905262 | Eby | Feb 1990 | A |
4931868 | Kadar | Jun 1990 | A |
4937685 | Barker et al. | Jun 1990 | A |
4942464 | Milatz | Jul 1990 | A |
4943854 | Shiota et al. | Jul 1990 | A |
4951140 | Ueno et al. | Aug 1990 | A |
4961211 | Tsugane et al. | Oct 1990 | A |
4962473 | Crain | Oct 1990 | A |
4965819 | Kannes | Oct 1990 | A |
D311924 | Loyd et al. | Nov 1990 | S |
D312649 | Sakuta | Dec 1990 | S |
4992866 | Morgan | Feb 1991 | A |
5001348 | Dirscherl et al. | Mar 1991 | A |
5012334 | Etra | Apr 1991 | A |
5014267 | Tompkins et al. | May 1991 | A |
5027222 | Shinbo et al. | Jun 1991 | A |
5027413 | Barnard | Jun 1991 | A |
5034986 | Karmann et al. | Jul 1991 | A |
5040067 | Yamazaki | Aug 1991 | A |
D322084 | Tabuchi | Dec 1991 | S |
5095196 | Miyata | Mar 1992 | A |
5097328 | Boyette | Mar 1992 | A |
5103306 | Weiman et al. | Apr 1992 | A |
5111290 | Gutierrez | May 1992 | A |
5117285 | Nelson et al. | May 1992 | A |
5128776 | Scorse et al. | Jul 1992 | A |
5142367 | Hong | Aug 1992 | A |
5144661 | Shamosh et al. | Sep 1992 | A |
5150212 | Han | Sep 1992 | A |
5179449 | Doi | Jan 1993 | A |
5182776 | Suzuki et al. | Jan 1993 | A |
5191645 | Carlucci et al. | Mar 1993 | A |
5202759 | Laycock | Apr 1993 | A |
5206929 | Langford et al. | Apr 1993 | A |
5216502 | Katz | Jun 1993 | A |
5218672 | Morgan et al. | Jun 1993 | A |
5229850 | Toyoshima | Jul 1993 | A |
5235420 | Gharavi | Aug 1993 | A |
5237408 | Blum et al. | Aug 1993 | A |
5237648 | Mills et al. | Aug 1993 | A |
5243418 | Kuno et al. | Sep 1993 | A |
5253062 | Ohta | Oct 1993 | A |
5258837 | Gormley | Nov 1993 | A |
5260783 | Dixit | Nov 1993 | A |
5270811 | Ishibashi et al. | Dec 1993 | A |
5272527 | Watanabe | Dec 1993 | A |
5285273 | James et al. | Feb 1994 | A |
5293313 | Cecil et al. | Mar 1994 | A |
D347839 | Carota | Jun 1994 | S |
5325194 | Natori et al. | Jun 1994 | A |
D349913 | Morris | Aug 1994 | S |
5335013 | Faber | Aug 1994 | A |
5339104 | Hong | Aug 1994 | A |
5339393 | Duffy et al. | Aug 1994 | A |
5341439 | Hsu | Aug 1994 | A |
D354973 | Hisatune et al. | Jan 1995 | S |
5382972 | Kannes | Jan 1995 | A |
5400069 | Braun et al. | Mar 1995 | A |
5406327 | Guarnotta | Apr 1995 | A |
5406328 | Chodos et al. | Apr 1995 | A |
D358812 | Boyd | May 1995 | S |
5422989 | Bell et al. | Jun 1995 | A |
5446002 | Kukes et al. | Aug 1995 | A |
5481297 | Cash et al. | Jan 1996 | A |
5490247 | Tung et al. | Feb 1996 | A |
5491508 | Friedell et al. | Feb 1996 | A |
5491511 | Odle | Feb 1996 | A |
5491797 | Thompson et al. | Feb 1996 | A |
5502576 | Ramsay et al. | Mar 1996 | A |
5509009 | Laycock et al. | Apr 1996 | A |
D370010 | Clapp et al. | May 1996 | S |
5515099 | Cortjens et al. | May 1996 | A |
5517236 | Sergeant et al. | May 1996 | A |
5521634 | McGary | May 1996 | A |
5526037 | Cortjens et al. | Jun 1996 | A |
5526133 | Paff | Jun 1996 | A |
5548324 | Downs et al. | Aug 1996 | A |
5568183 | Cortjens et al. | Oct 1996 | A |
5583565 | Cortjens et al. | Dec 1996 | A |
5589878 | Cortjens et al. | Dec 1996 | A |
5598208 | McClintock | Jan 1997 | A |
5598209 | Cortjens et al. | Jan 1997 | A |
5604341 | Grossi et al. | Feb 1997 | A |
5612734 | Nelson et al. | Mar 1997 | A |
5619995 | Lobodzinski | Apr 1997 | A |
5642498 | Kutner | Jun 1997 | A |
5648814 | Munson | Jul 1997 | A |
5649255 | Schieltz | Jul 1997 | A |
5657096 | Lukacs | Aug 1997 | A |
5659692 | Poggio et al. | Aug 1997 | A |
5689442 | Swanson et al. | Nov 1997 | A |
5729471 | Jain et al. | Mar 1998 | A |
5737011 | Lukacs | Apr 1998 | A |
5764277 | Loui et al. | Jun 1998 | A |
5767897 | Howell | Jun 1998 | A |
5774674 | Gutmann et al. | Jun 1998 | A |
5818514 | Duttweiler et al. | Oct 1998 | A |
5821990 | Rudt et al. | Oct 1998 | A |
5825413 | Mullis | Oct 1998 | A |
5841763 | Leondires et al. | Nov 1998 | A |
5872923 | Schwartz et al. | Feb 1999 | A |
5896128 | Boyer | Apr 1999 | A |
5926611 | Yang et al. | Jul 1999 | A |
6037977 | Peterson | Mar 2000 | A |
6292204 | Carleton et al. | Sep 2001 | B1 |
6469746 | Maida | Oct 2002 | B1 |
6700625 | Fujii | Mar 2004 | B1 |
20020131505 | Vidunas | Sep 2002 | A1 |
Number | Date | Country |
---|---|---|
0494752 | Jul 1992 | EP |
4184761 | Jul 1992 | JP |
4373386 | Dec 1992 | JP |
5168760 | Jul 1993 | JP |
5227532 | Sep 1993 | JP |
5346773 | Dec 1993 | JP |
6004731 | Jan 1994 | JP |
6006804 | Jan 1994 | JP |
6044470 | Feb 1994 | JP |
686289 | Mar 1994 | JP |
6070040 | Mar 1994 | JP |
6078269 | Mar 1994 | JP |
H670277 | Mar 1994 | JP |
6096378 | Apr 1994 | JP |
6141310 | May 1994 | JP |
6266774 | Sep 1994 | JP |
H6303651 | Oct 1994 | JP |
7067098 | Mar 1995 | JP |
WO-9625710 | Aug 1996 | WO |
Number | Date | Country | |
---|---|---|---|
Parent | 08050861 | Apr 1993 | US |
Child | 08418823 | US | |
Parent | 08298104 | Aug 1994 | US |
Child | 08050861 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 08418823 | Apr 1995 | US |
Child | 09301656 | US |