Patent Application
20130021433
1. Field of the Invention
The present invention relates to surveillance cameras, and, more particularly, to surveillance cameras that are able to pan, tilt and zoom.
2. Description of the Related Art
Surveillance camera systems are commonly used by retail stores, banks, casinos and other organizations to monitor activities within a given area. The cameras are often provided with the capability to pan and tilt in order to acquire images over a wide domain. The tilt of the camera generally refers to the pivoting of the camera about a horizontal axis that is parallel to the floor, such that the lens of the camera may tilt between an upwardly pointing position and a downwardly pointing position. The pan of the camera refers to the rotation of the camera about a vertical axis that is perpendicular to the floor, such that the lens may scan from side to side. The cameras may also be able to zoom in order to reduce or enlarge the field of view.
PTZ cameras have a varying field of view and have relevant features. However, the configuration methods for these cameras are mere extensions of the methods used for configuring fixed cameras.
In comparing a PTZ camera and a fixed camera, the field of view of a fixed camera is determined at the time of installation and does not change thereafter. On the other hand, a PTZ camera finds a reference position upon power up and then the PTZ camera can move in pan, tilt and zoom directions. Thus, any point within three-dimensional space may be in the field of view of the PTZ camera, assuming that the panning, tilting and zooming mechanisms have no limitations.
Most of the features of a PTZ camera relate to the video at specific pan, tilt and zoom coordinates. Such features may include presets, tours, record/playback, privacy masks, alarms and video analytics settings. As used herein, the term “preset” may refer to a particular pan, tilt, zoom position of the camera. The camera may stop at a preset at during each tour of the camera. The camera may dwell at the preset for some period of time and continue to capture images at the preset. A playback of a recording of the camera may involve the camera undergoing a predetermined path of pan, tilt and zoom movements. A tour may include panning components of the camera movements spanning 360 degrees or more.
Certain, pan, tilt, zoom locations can be saved as presets. The presets can be recalled on demand or the camera sequentially moves between various presets.
A tour involves the PTZ device repetitively moving to predefined positions in sequence. There may be predefined or fixed time intervals in-between the positions on the tour.
In record/playback, the user records a path involving specific pan, tilt and zoom movements at specific time intervals. Once recorded, the path may be played back once or repetitively.
Privacy masks are areas within a field of view that the system does not allow a viewer to see. That is, privacy masks are used to block out the video from chosen pan, tilt, zoom locations. For example, a window in a house may be covered by a privacy mask.
Alarm inputs can be physical inputs such as sensors, and alarm outputs can be switches and relays. Although alarms may not be related to video, alarms may be attached to specific pan, tilt, zoom positions such as at doors, gates, etc.
Video analytics settings or parameters such as trip wires or sensitive areas may relate to specific pan, tilt, zoom locations. The user may be interested in monitoring certain sensitive areas for activity, such as at doors, gates, etc. Video analytics settings or parameters may be conjointly referred to herein as a “video analytics profile” or an “intelligent video analysis (IVA) profile.”
In currently known methods such as described above for configuring the various features of a PTZ camera, only the video at the current PTZ position is shown to the user. For example, when configuring the privacy masks 16 of
These above-described configuration methods, which work well for fixed cameras, however ignore and fail to take advantage of the fact that the PTZ camera can move to any point in the entire three-dimensional space and is not limited to the current field of view. Thus, with the above-described configuration methods, the user does not receive a comprehensive outlook of the entire field of view. For example, he does not know where all the presets are configured; where all the sensitive areas are; or where the alarms are located, for example. Also, with the above-described configuration methods, the user does not know the positions of the parameters relative to each other. It may be beneficial to know the positions of the parameters relative to each other because the number of configurable parameters is generally large. For example, there may be ninety-nine presets, twenty-four masks, and ten analytics profiles.
A problem is that currently known configuration methods of PTZ cameras restrict the user to the current field of view (FOV). This can be constraining for various features of the PTZ cameras. In many cases, the user would like to see where these features are physically located in the larger scene and how the features are separated from each other. For example, the user would like to see where he has defined the FOV presets within the larger scene, and which presets are on a given tour.
What is neither disclosed nor suggested by the prior art is a surveillance camera arrangement that enables the user to see the location of the present FOV, as well as features such as presets, privacy masks, recordings, and video analytics profiles within a panoramic view of the surrounding area. Nor has the idea of being able to control the camera via a panoramic view been suggested by the prior art.
The present invention is directed to an overview configuration method for a PTZ camera wherein the method employs a panorama (plain or circular) image. The invention provides a method for controlling, configuring and viewing the features of a typical PTZ device. The user is provided with a representation of the complete possible field of view by a panoramic image and the selected features are mapped to it. The user can view/configure the parameters such as presets, tours, pattern record and playback via the overview configuration method. The user can control the pan, tilt, and zoom movement of the camera via a FOV projection overlaid on the panoramic image. The panoramic image is governed by moving the camera to each required pan, tilt, zoom position; acquiring the image at each position; and stitching all the acquired images together. The various PTZ-dependent features are mapped to this image. The user can then select and modify the features of his interest, such as presets, privacy masks, recordings, video analytics profiles, etc. The PTZ-dependent features can be represented on the panoramic image as a scaled FOV projection.
In one embodiment of an overview configuration method of the present invention for PTZ cameras, the user is presented with a visual representation of the complete possible field of view of the camera. The representation may be in the form of a panoramic image, and the selected features may be mapped to the panoramic image. While viewing the selected features and the panoramic image, the user can view and/or configure the parameters. The panoramic image may be generated by moving the PTZ camera to each required pan, tilt, zoom position; acquiring an image at each of these positions; and stitching together all of the acquired images to form a composite image.
The panoramic image could be created at any time. The user would be able to determine if the image is to be used in the configuration process or saved to a location of their choosing on the computer. Saving images of this type would be useful to get a snapshot of the entirety of the camera's view at the time of the image's creation.
The various features which are dependent upon the pan, tilt, zoom positions may be mapped to the composite image. The user may then be able to select and modify the features he is interested in, such as presets, privacy masks, recordings, video analytics profiles, etc.
In one embodiment of a configuration method of the invention, the user initiates the configuration process, such as by clicking on an on-screen configuration icon, or by pressing a configuration pushbutton, for example. Next, images covering the entire scene within the space are captured at respective panning intervals. These individual images may be stitched together to form a panoramic 360 degree image. On the panoramic 360 degree image, the user may view and/or edit features such as by editing a mask, removing a preset, etc. After the user has viewed and/or edited the features, the user may save the features and their corresponding locations. The user may initiate the saving process by clicking on an on-screen save icon, or by pressing a save pushbutton, for example. Upon the saving process being initiated, the locations corresponding to the saved features may be converted or mapped from the panoramic image to the pan, tilt, zoom coordinates of the PTZ device's coordinate system. Thus, the features may be saved along with their corresponding pan, tilt, zoom coordinates. During subsequent tours of the PTZ device (e.g., PTZ camera), the PTZ device may stop at these saved pan, tilt, zoom coordinates and then execute the corresponding feature (e.g., a preset). Alternatively, the PTZ device may not stop at saved pan, tilt, zoom coordinates, but rather may place a feature at the saved pan, tilt, zoom coordinates (e.g., a privacy mask).
After the configuration is complete, the user may use the panoramic view to control the camera's pan, tilt, and zoom position via a FOV projection on the view. This type of control would be available from hereinafter.
The invention comprises, in one form thereof, a method of operating a surveillance camera arrangement, including panning a PTZ camera about a pan axis. First images are captured with the camera throughout the panning. A composite panoramic or circular second image is created by stitching together the first images captured during the panning. The user is enabled to select and modify presets, recordings, and/or video analytics profiles within the composite panoramic or circular second image.
The invention comprises, in another form thereof, a method of operating a surveillance camera arrangement, including, in response to the user initiating a configuration process, capturing a plurality of discrete images with at least one camera. Each of the discrete images correspond to a different respective field of view. The captured images are combined to thereby form a panoramic image. The panoramic image is displayed. The user is enabled to view, establish and/or edit an operating feature of the at least one camera. The operating feature corresponds to at least one location in the panoramic image. After the enabling step, information defining the feature and the at least one corresponding location is saved.
The invention comprises, in yet another form thereof, a method of configuring a surveillance camera arrangement, including receiving an initiation of the configuration from the user. The camera is used to perform a scanning movement. A plurality of images are captured with the camera. Each of the images is captured at a respective one of a plurality of substantially evenly-spaced locations within the scanning movement. The captured images are merged together to thereby generate a panoramic composite image. The user is enabled to view the composite image. Received from the user is a modification of a feature associated with the panoramic composite image. The feature is saved in memory as modified by the user.
The present invention may take into account that the PTZ camera can move to any point in the entire three-dimensional space, and is not limited to the current field of view. This would be controlled by the user via a FOV projection overlaid on the panoramic image. The camera may move fluidly as the user drags the FOV projection. The camera could also move directly to a pan and tilt position by clicking the FOV to a different location on the panoramic view.
The user may receive a comprehensive outlook of the entire field of view, and may be informed of where all the presets are configured, where all the sensitive areas are, and where the alarms are located.
The user may be informed of the positions of the parameters relative to each other. This may be especially beneficial in view of the number of configurable parameters being generally large, e.g., ninety-nine presets, twenty-four masks, and ten analytics profiles.
The invention may provide a method for faster configuration for setting up all the relevant features in a single session. For example, all the privacy masks can be defined at once.
The invention may provide a high level of flexibility in programming the PTZ device as the user is not restricted to predefined primitives, such as a preset tour.
A still further another advantage is that the method of the invention has a lot of potential for future expansions, and can provide a basis for developing new and useful features such as intelligent video analysis (WA) tour, etc.
The above mentioned and other features and objects of this invention, and the manner of attaining them, will become more apparent and the invention itself will be better understood by reference to the following description of embodiments of the invention taken in conjunction with the accompanying drawings, wherein:
Corresponding reference characters indicate corresponding parts throughout the several views. Although the exemplification set out herein illustrates embodiments of the invention, in several forms, the embodiments disclosed below are not intended to be exhaustive or to be construed as limiting the scope of the invention to the precise forms disclosed.
In accordance with the present invention, a surveillance camera arrangement 20 is shown in
Arrangement 20 also includes a head end unit 32. Head end unit 32 may include a video switcher or a video multiplexer 33. For example, the head end unit may include an Allegiant® brand video switcher available from Bosch Security Systems, Inc., such as a LTC 8500 Series Allegiant® Video Switcher which provides inputs for up to sixty-four cameras and may also be provided with eight independent keyboards and eight monitors. Head end unit 32 includes a keyboard 34 and joystick 36 for operator input. Head end unit 32 also includes a display device in the form of a monitor 38 for viewing by the operator. A twenty-four volt a/c power source 40 is provided to power both camera 22 and a processing device 50 that is operably coupled to both camera 22 and head end unit 32.
The illustrated arrangement 20 is a single camera application, however, the present invention may be used within a larger surveillance system having additional cameras which may be either stationary or moveable cameras or some combination thereof to provide coverage of a larger or more complex surveillance area. One or more analog or digital recording devices may also be connected to head end unit 32 to provide for the recording of the video images captured by camera 22 and other cameras in the system.
Camera 22 may include an image-capturing device such as a charge coupled device (CCD) that acquires a four-sided (e.g., rectangular) CCD video image. Processing device 50 may identify or select at least a portion of the CCD image to be displayed on a screen of monitor 38 for viewing by an operator of system 20.
The hardware architecture of processing device 50 is schematically represented in
Via another analog video line 56, a video decoder/scaler 58 receives video images from camera 22, converts the analog video signal to a digital video signal, and separates the luminance (Y) component from the chrominance (U, V) components of the composite, color video signal. Video decoder/scaler 58 sends a full resolution (unscaled) digital video signal 59 to a video capture port of the VCA DSP 62. Video decoder/scaler 58 also sends a scaled (sub-sampled horizontally by four and vertically by four) QCIF image 61 produced by its scaler function to a second video capture port of VCA DSP 62. SDRAM memory 60 connects directly to VCA DSP 62 and provides volatile memory to store and execute the VCA software after boot, and to provide temporary memory storage. This temporary storage includes, but is not limited to, the storage of video buffers. The video stabilization algorithm described above with reference to
In the illustrated embodiment, video input to system controller board 64 is limited to about 1.1 Volt peak-to-peak. If the video signal exceeds 1.1 Volt peak-to-peak without a proportional increase in synchronization level, then the signal may be clipped to about 1.1 Volt peak-to-peak. If the video signal including the synchronization level is increased, then the video decoder/scaler 58 will attempt to compensate by reducing the video gain in order to regulate the synchronization level. However, alternative embodiments having a greater or lesser capacity may also be employed with the present invention. Processor 62 may be a TMS320DM642 programmable Video/Imaging Fixed-Point Digital Signal Processor available from Texas Instruments. At start up, processor 62 loads a bootloader program. The boot program then copies the VCA application code from a memory device such as flash memory 78 to SDRAM 60 for execution. In the illustrated embodiment, flash memory 78 provides four megabytes of memory and SDRAM 60 provides thirty-two megabytes of memory. In the illustrated embodiment, at most four MBytes of the thirty-two MBytes of SDRAM will be required to execute code, and the remaining twenty-eight MBytes of SDRAM is available for video buffers and other use.
In the embodiment shown in
System controller board 64 is connected to an RJ-45 compatible Ethernet transceiver 109 via RJ-45 Ethernet cable 111. A line 51, which can be in the form of an RJ-45 Ethernet cable 111, communicates signals from head end unit 32 to processing device 50. The signals on cable 51 can include signals that can be modified by processing device 50 before being sent to camera 22. Such signals may be sent to camera 22 via line 48 in communication with microprocessor 112. Camera 22 may have Ethernet capability, and the capturing of the images and control may be performed via Ethernet.
System controller board 64 may also include a field programmable gate array 116 including a mask memory 118, a character memory 120, and an on screen display (OSD) memory 122. Similarly, VCA components 114 may include a mask memory 124, a character memory 126, and an on screen display (OSD) memory 128. These components may be used to mask various portions of the image displayed on screen 38 or to generate textual displays for screen 38. Finally, system controller board 64 can include a parallel data flash memory 130 for storage of user settings.
In the illustrated embodiment, the only necessary commands conveyed to processing device 50 that are input by a human operator are on/off commands and PTZ commands. However, even these on/off commands and PTZ commands may be automated in alternative embodiments. Such on/off commands and other serial communications are conveyed via bi-phase line 46 between head end unit 32 and camera 22, and between processing device 50 and camera 22 via line 48.
In the illustrated embodiment, processing device 50 is mounted proximate camera 22. However, processing device 50 may also be mounted employing alternative methods and at alternative locations. Alternative hardware architecture may also be employed with processing device 50. Such hardware should be capable of running the software and processing at least approximately five frames per second for good results. It is also noted that by providing processing device 50 with a sheet metal housing, the mounting of processing device 50 on or near a PTZ camera is facilitated, and system 20 may thereby provide a stand alone embedded platform which does not require a personal computer-based image stabilization system. If desired, however, the present invention may also be employed using a personal computer based system.
Processing device 50 can perform several functions, including capturing video frames acquired by camera 22, identifying a stationary feature in the video frames, determining the intended change in the camera FOV based upon signals sent to or received from camera 22, identifying a stationary feature and determining the actual change in the camera FOV, comparing the intended and actual change in the camera FOV to determine the magnitude of the image translations resulting from the unintentional motion of the camera and selecting display image coordinates to counteract the translations resulting from the unintentional motion of the camera. Processing device 50 may also be used to perform an automated tracking function. For example, processing device 50 may also provide an automated tracking system wherein processing device 50 is used to identify moving target objects in the FOV of the camera and then generate control signals which adjust the pan, tilt and zoom settings of the camera to track the target object and maintain the target object within the FOV of the camera.
Arrangement 20 as described above may be used in conjunction with an overview configuration method of the present invention. The user may be presented with a representation of the complete possible field of view by a panoramic image and the selected features are mapped to the panoramic image. The user can view and/or configure the parameters.
The panoramic image may be generated by moving the camera to each required pan, tilt, zoom position, acquiring an image at each position, and stitching together all of the acquired images into a composite panoramic or circular image. The various features that are dependent upon pan, tilt, zoom position may be mapped to the composite image. The user can then select and modify the features of his interest, such as presets, privacy masks, recordings, video analytics profiles, etc.
One embodiment of a configuration method 500 of the present invention is illustrated in
In a next step 504, the camera performs scanning movement. That is, the camera may undergo a panning motion that may also include components of tilting motion and zoom motion. In one embodiment, the panning motion may span approximately 360 degrees about a vertical axis.
Next, in step 506, the camera captures images at evenly-spaced locations throughout the scanning movement. The images may be captured over the Ethernet interface. In one embodiment, the distance or space between the evenly-spaced locations may depend upon the width of the field of view of the camera. For example, assume that in a very specific embodiment the width of the field of view of the camera is sixty degrees. In this case, the camera may undergo sixty degrees of panning motion between the locations at which the camera captures images. Thus, after capturing the first image and panning sixty degrees five separate times, capturing an image after each sixty degree panning motion, the camera encompasses 360 degrees of panning movement. Each of the six captured images is sixty degrees wide in the panning direction, and together cover the entire 360 degrees surrounding the camera. Each of the captured images may be horizontally adjacent two other ones of the six captured images.
In step 508, the captured images are stitched or merged together to thereby generate a panoramic image. To continue the example given above in association with step 506, assuming that each captured image has two opposing vertically-oriented edges, each of the six adjacent pairs of vertically-oriented edges may be merged together to thereby form a continuous, seamless, and uninterrupted 360 degree panoramic composite image. It is possible for each of the captured images to be substantially triangular such that each of the captured images meet at a point directly above the camera. In this case, the composite image is hemispherically-shaped, or, more generally, frusto-spherically-shaped.
In a next step 510, the user views and/or edits features associated with the panoramic image. For example, the user may edit a mask, remove a preset, modify a video recording, and/or modify a video analytics profile.
In a final step 512, the user selects to save the features. For example, the user may use a computer mouse or joystick 36 to click on an on-screen icon for saving the features as edited in step 510. Alternatively, the user may initiate the saving of modified features by navigating through an on-screen menu of options, or by pressing a dedicated pushbutton on keyboard 34. In one embodiment, the features represented on the panoramic image may be converted back to the pan, tilt, and zoom coordinates of the PTZ device's coordinate system and saved in memory.
In a specific embodiment, method 500 may further include performing a tour with the camera including panning, tilting and/or zooming movements. The tour of the camera may be stopped at a set of pan, tilt, and zoom coordinates. After the stopping step, the feature is placed at the set of pan, tilt, and zoom coordinates.
In
One embodiment of a method 1100 of the present invention for operating a surveillance camera arrangement is illustrated in
In a next step 1104, first images are captured with the camera throughout the panning. For example, the image of
Next, in step 1106, a composite panoramic or circular second image is created by stitching together the first images captured during the panning. That is, the composite panoramic image of
Next, in step 1108, the user is enabled to select and modify presets, recordings, and/or video analytics profiles within the composite panoramic or circular second image. For example, as shown in
In a final step 1110, the user is enabled hereinafter to control the camera's pan, tilt, and zoom position via a FOV projection on the composite image as shown in
Another embodiment of a method 1200 of the present invention for operating a surveillance camera arrangement is illustrated in
In a next step 1204, the captured images are combined together to thereby form a panoramic image. That is, the panoramic image of
Next, in step 1206, the panoramic image is displayed. That is, the panoramic image may be displayed on a monitor or display screen as shown in
In step 1208, the user is enabled to view, establish and/or edit an operating feature of the at least one camera. The operating feature corresponds to at least one location in the panoramic image. For example, as shown in
In a final step 1210, after the enabling step, information defining the feature and the at least one corresponding location is stored in memory. For example, information defining the user-created features shown in
According to one embodiment of the invention, the panoramic image is completed within a few seconds after the user clicks the configuration button. In able to facilitate such responsiveness, the camera may be restricted to only a few zoom positions.
There may be memory constraints involved with the invention. Where the panoramic image is generated may determine the memory limitation. In one embodiment, the panoramic image is generated in the personal computer (PC) of the surveillance camera arrangement.
As the FOV projection 1500 is moved, the camera may physically move in correspondence with the projection's location and size on the panoramic image. The camera may also zoom in and out to follow the zooming in and out of projection 1500 on the display screen. Thus, FOV projection 1500 may enable full pan, tilt, and zoom movements to be conveyed to the camera. In contrast to known camera control methods that include a physical or digital representation of a joystick, FOV projection 1500 may enable instant movement to a pan and tilt position. In another embodiment, camera movement may be initiated by, and may follow, a single mouse click that may instantly move the FOV projection to the screen location clicked in the panoramic view. In another embodiment, zoom control may be implemented by clicking and dragging one of edges 1502, 1504, 1506, 1508 of FOV projection 1500 radially inward and outward, similar to resizing an application's window on a computer.
In another embodiment, a FOV projection may be used as a way of representing a preset, tour path and stopping positions, record/playback path, alarm inputs, and/or video analytics positions on the panoramic view. The FOV projection may show or indicate the associated actual FOV of the camera. For example, if a preset is recalled, or the position in which an alarm input is triggered, the FOV projection may show the actual FOV of the camera at the preset or at the position in which the alarm input is triggered.
While this invention has been described as having an exemplary design, the present invention may be further modified within the spirit and scope of this disclosure. This application is therefore intended to cover any variations, uses, or adaptations of the invention using its general principles.
