The present invention relates to a display control apparatus that displays a video captured by a monitoring camera, a display control method, a camera system, a control method for a camera system, and a storage medium.
In the field of monitoring systems, a technique to detect a moving object in a video using a video analysis technique is known. Furthermore, a technique to enable constant capturing and recognition of a detected moving object is known as a moving object tracking technique.
Using the foregoing techniques, Japanese Patent Laid-Open No. 2002-374522 discloses a monitored region setting method for setting regions monitored by a plurality of installed monitoring cameras while checking videos of the monitoring cameras.
However, in conventional monitoring systems, during display of a video captured by a certain camera that is currently selected, if detection such as passage detection and intrusion detection is performed by the selected camera, such detection can be displayed on a screen together with a passage line on which passage was detected and a region in which intrusion was detected; however, if such detection is performed by another camera, such detection cannot be displayed on the screen. This gives rise to the possibility that a detected event is overlooked, and even if the occurrence of the detection is acknowledged, it is not possible to specifically check where the site of the detection is on a video.
In view of the above problems, the present invention provides a technique to, when monitoring a region with a plurality of cameras, allow display of an event detected by a certain individual camera on a currently-displayed video together with a detection setting that has contributed to the detection, even if the currently-displayed video is captured by another camera.
According to one aspect of the present invention, there is provided a display control apparatus comprising: a reception unit configured to receive a detection notification indicating detection of an event by a first camera; an image obtainment unit configured to obtain a video from a second camera; and a display control unit configured to, in a case where the detection notification is received from the first camera that corresponds to detection of the event in which an occurrence position is included in the video from the second camera, display a predetermined image corresponding to the detection of the event together with the video from the second camera.
Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings).
An exemplary embodiment(s) of the present invention will now be described in detail with reference to the drawings. It should be noted that the relative arrangement of the components, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.
The detection setting terminal 1000 configures various types of detection settings, such as setting related to moving object detection functions of monitoring cameras, setting of a type of detection performed by the monitoring cameras (passage detection, intrusion detection, and the like), setting of a detection line segment for detecting passage in the case of passage detection, and setting of a closed region for detecting intrusion in the case of intrusion detection. A user configures detection settings via a touchscreen operation while viewing a capture screen displayed on the detection setting terminal 1000. The detection setting terminal 1000 functions as an information processing apparatus that controls a plurality of monitoring cameras monitoring predetermined regions.
The monitoring cameras 3001 to 3003 have a moving object detection function and a passage detection function. The monitoring cameras 3001 to 3003 transmit monitoring videos that have been captured to the monitoring video recording server 4000 via the LAN 2000. The monitoring cameras 3001 to 3003 are also connected to the detection setting terminal 1000 via the LAN 2000, receive detection settings from the detection setting terminal 1000, and store therein the received detection settings. Furthermore, at the time of installation of the monitoring cameras 3001 to 3003, they receive, from the detection setting terminal 1000, detection setting conversion information determined by angles of view of the cameras, zoom settings of the cameras, and positions and angles at which the cameras are installed, and also store therein the received detection setting conversion information. The monitoring video recording server 4000 receives monitoring videos in a compressed state from the monitoring cameras 3001 to 3003, and records the received monitoring videos in a hard disk drive. The detection notification display terminal 5000 receives detection notifications from the monitoring cameras 3001 to 3003, and displays camera videos with the result of detection superimposed thereon. While it is assumed in the present embodiment that the detection notification display terminal 5000 is separate from the detection setting terminal 1000, they may be configured as the same apparatus.
The detection setting obtainment unit 101 obtains detection settings and detection setting conversion information set in the monitoring cameras from the monitoring cameras. The detection setting obtainment unit 101 is connected to the monitoring cameras via the LAN 2000, and can obtain the detection settings and the detection setting conversion information set in the monitoring cameras in accordance with a predetermined setting information obtainment protocol.
The detection setting management unit 102 manages the detection settings and the detection setting conversion information obtained from the monitoring cameras, and newly updates the detection settings in accordance with user input.
The image obtainment unit 103 obtains captured images from the monitoring cameras. More specifically, it obtains compressed video data from the monitoring cameras via the LAN 2000, similarly to the detection setting obtainment unit 101.
The detection setting display unit 104 displays, on a screen, the videos obtained from the monitoring cameras with the detection settings superimposed thereon. In the present embodiment, the detection setting display unit 104 is a liquid crystal monitor for display.
The user operation input unit 105 accepts user input for setting detection settings. In the present embodiment, the user operation input unit 105 is described as, but not limited to, a touchscreen that is arranged to overlie the liquid crystal monitor.
The detection setting unit 106 sets generated detection settings for the monitoring cameras. It has functions of connecting to the monitoring cameras via the LAN 2000, similarly to the image obtainment unit 103 and the detection setting obtainment unit 101, and of setting detection settings for the monitoring cameras in accordance with a predetermined setting protocol.
The detection setting conversion unit 107 executes coordinate conversion between: detection settings obtained from the monitoring cameras and detection settings set for the monitoring cameras; and detection settings on a two-dimensional plane. For example, it executes coordinate conversion between coordinates on a planar map and coordinates on videos captured by the monitoring cameras in relation to a detection line set for passage detection.
The map information storage unit 108 stores planar map information. The planar map will be described later in greater detail with reference to
A description is now given of the operations of the above-described monitoring system.
The monitoring camera 3001 is installed so as to face the direction of a door to some degree with a passageway situated substantially therebelow, and the monitoring camera 3002 is installed so as to capture the direction of the door from a rear side of the passageway at a shallow oblique angle. The monitoring camera 3003 is installed to capture people who pass by the door from a lateral direction. Regions included in capturing ranges of the monitoring cameras 3001 to 3003 overlap at least partially.
The operations of the detection setting conversion unit 107 shown in
The monitoring cameras 3001, 3002 and 3003 capture videos in the state where the four points A, B, C, D have been marked on the actual floor surface as indexes. The image obtainment unit 103 obtains the video of the monitoring camera 3001, and the detection setting display unit 104 displays the video. The marked points A, B, C, D on the camera video displayed on a setting screen are selected through a user operation on the screen; as a result, coordinates of these four points on the screen are obtained.
Next, positions of these four points on the corresponding planar map of
In the above-described method, correspondence between four points A, B, C, D on the image captured by the monitoring camera 3001 and four points A′, B′, C′, D′ on the planar map is obtained, thereby making it possible to obtain projection conversion between points on the floor surface captured by the monitoring camera 3001 and points on the planar map, as well as inverse conversion therebetween, as conversion parameters.
At the time of installation of the monitoring camera 3001, the aforementioned measurement is carried out, and the obtained conversion parameters are stored in the monitoring camera 3001 as setting conversion information. Similarly, with regard to the monitoring camera 3002, a capture screen shown in
Similarly, with regard to the monitoring camera 3003, projection conversion and inverse conversion are obtained as conversion parameters from a corresponding relationship between the four points on the floor surface, and the conversion parameters are stored in the monitoring camera 3003 as setting conversion information. With the setting conversion information obtained in the above manner, a corresponding relationship between the monitoring cameras can be identified. Note that it is sufficient to execute the aforementioned processing only once at the time of installation of the monitoring cameras, and it is not necessary to carry out the measurement again unless the angles of view and orientations of the monitoring cameras change.
The following describes the operations executed by the detection setting terminal 1000 to set detection settings for the monitoring cameras. First, a description is given of a method for setting detection settings for the first camera, that is to say, the monitoring camera 3001 with reference to a flowchart of
In step S801, the detection setting management unit 102 connects to the monitoring camera 3001 in accordance with a setting protocol. In step S802, the image obtainment unit 103 connects to the monitoring camera 3001 in accordance with a video obtainment protocol and obtains images from the monitoring camera 3001.
In step S803, the user selects a detection type to be set by operating the user operation input unit 105 (for example, a touchscreen).
In step S804, the detection setting management unit 102 determines the detection type selected in step S803. If the detection type selected in step S803 is passage detection, processing proceeds to step S805. On the other hand, if the detection type selected in step S803 is intrusion detection, processing proceeds to step S806.
In step S805, through an operation on the user operation input unit 105, the user inputs a detection line by selecting a start point and an end point of a line on the floor surface targeted for detection while viewing a captured video displayed on a screen. Consequently, in accordance with the user operation, the detection line is rendered and superimposed on the video of the monitoring camera on the screen. In the example of the screen shown in
In step S806, through an operation on the user operation input unit 105, the user inputs a closed region for intrusion detection on the floor surface while viewing the captured video displayed on the screen. Consequently, in accordance with the user operation, the closed region is rendered and superimposed on the video of the monitoring camera on the screen.
In step S807, the detection setting management unit 102 determines whether or not detection settings for the target monitoring camera (in the present example, the monitoring camera 3001) have been completed. If detection settings have not been completed (NO of step S807), processing returns to step S803, and input of detection settings is repeated. On the other hand, if detection settings have been completed (YES of step S807), processing proceeds to step S808.
In step S808, the detection setting unit 106 transmits the detection settings to the monitoring camera 3001. In the present embodiment, passage detection is input as a detection type, start coordinates and end coordinates of the passage detection line L1 are input as a site targeted for detection, and these detection settings are transmitted to the monitoring camera 3001 via the detection setting unit 106. Note that these coordinates are plotted on a capture screen of the monitoring camera 3001.
In step S809, once the settings for the monitoring camera 3001 have been completed, connection with the monitoring camera 3001 is terminated, and detection setting processing is ended.
The following describes a method for setting detection settings for the second and subsequent cameras with reference to a flowchart of
The user selects settings for the second camera, e.g. the monitoring camera 3002 (second monitoring camera) by operating the detection setting terminal 1000. It will be assumed that the monitoring camera 3001 is the first monitoring camera.
In step S901, the detection setting management unit 102 connects to the monitoring camera 3002 in accordance with a setting protocol. In step S902, the image obtainment unit 103 connects to the monitoring camera 3002 in accordance with a video obtainment protocol and obtains images from the monitoring camera 3002.
In step S903, the detection setting management unit 102 connects to the monitoring camera 3001 in accordance with a setting protocol. In step S904, the detection setting management unit 102 checks setting information of the monitoring camera 3001 and determines whether or not there is any detection setting that has not been obtained yet. If there is any detection setting that has not been obtained yet in the monitoring camera 3001 (YES of step S904), processing proceeds to step S905.
In step S905, the detection setting obtainment unit 101 obtains setting information from the monitoring camera 3001. In step S906, the detection setting conversion unit 107 converts the setting information of the monitoring camera 3001 obtained in step S905 into detection line setting values normalized on a planar map.
In step S907, the detection setting conversion unit 107 converts the normalized setting values on the planar map into setting information for the monitoring camera 3002 (second monitoring camera), and superimposes the resultant setting information on an image displayed on a detection setting screen for the monitoring camera 3002.
In this way, based on information showing correspondence between coordinates on a video of the monitoring camera 3001 and coordinates on a video of the monitoring camera 3002 (setting conversion information of conversion from the monitoring camera 3001 to the planar map, and setting conversion information of conversion from the planar map to the monitoring camera 3002), detection settings for the monitoring camera 3001 are converted into detection settings for the monitoring camera 3002, and then displayed together with a video. In the example of
If the user wishes to modify detection settings that have been automatically generated (YES of step S908), the user modifies the detection settings through a user operation. On the other hand, if no further modification is applied to the settings (NO of step S908), processing proceeds to step S910.
In step S909, the detection line is modified by selecting an edge point of a tentative detection line and dragging the selected edge point on the screen using the user operation input unit 105. If the user applies no modification (NO of step S908), a detection line that has been automatically generated by the detection setting conversion unit 107 is used as-is.
In step S910, the detection setting unit 106 transmits detection line information set for the monitoring camera 3002 to the monitoring camera 3002, and accordingly configures detection settings for the monitoring camera 3002. The detection setting management unit 102 then terminates connection with the monitoring camera 3002.
Thereafter, in step S911, the detection setting management unit 102 terminates connection with the monitoring camera 3001, and ends processing. In the above manner, detection settings can be configured for a plurality of monitoring cameras.
As described above, detection setting information that has been set for one camera is displayed as default setting information when configuring detection settings for another monitoring camera; therefore, the detection settings can easily be configured with reference to the default setting information. As a result, detection settings for a plurality of cameras can be configured with ease.
The following describes the operations executed by a detection display terminal when passage detection is performed by the monitoring camera 3001 and information to that effect is notified while the user is viewing a video of the monitoring camera 3002.
The detection notification obtainment unit 109 receives, from the monitoring cameras, notifications of passage detection and intrusion detection performed by the monitoring cameras. The detection notification management unit 110 manages detection notifications received from the plurality of cameras. The detection notification display unit 111 displays the results of detection notifications together with camera videos.
A procedure of processing executed by the detection notification display terminal 5000 will now be described with reference to a flowchart of
In step S1101, the detection notification obtainment unit 109 connects to the monitoring cameras 3001 to 3003 in the network in accordance with a detection notification setting protocol, and sets these monitoring cameras to transmit detection notifications when they perform passage detection or intrusion detection.
In step S1102, the user operation input unit 105 accepts a user operation for selecting a monitoring camera whose video is to be displayed on a screen of the detection notification display terminal 5000. It is assumed here that the user inputs a selection of the monitoring camera 3002 through a user operation, so that a video thereof is displayed on the screen of the detection notification display terminal 5000.
In step S1103, the image obtainment unit 103 connects to the monitoring camera selected in step S1102 and starts obtaining the video thereof. It is assumed here that the image obtainment unit 103 connects to the monitoring camera 3002 and starts obtaining a video thereof. Then, the detection notification display unit 111 displays the video obtained from the monitoring camera 3002 on the screen of the detection notification display terminal 5000.
In step S1104, the detection notification obtainment unit 109 determines whether or not a detection notification has been received. If the detection notification has been received (YES of step S1104), processing proceeds to step S1107. On the other hand, if the detection notification has not been received (NO of step S1104), processing proceeds to step S1105.
In step S1105, the detection notification obtainment unit 109 determines whether or not to change the monitoring camera whose video is to be displayed. Note that whether or not to change the monitoring camera whose video is to be displayed can be determined based on whether or not the user has selected a monitoring camera from a drop-down menu and pressed the “SWITCH CAMERA” button as shown in
In step S1106, the image obtainment unit 103 terminates connection with the monitoring camera from which the video is currently received. Thereafter, processing returns to step S1103, in which the image obtainment unit 103 connects to the monitoring camera that has been selected as a result of the change, and newly starts obtaining a video thereof.
In step S1107, the detection notification management unit 110 determines whether or not the received detection notification has been issued from the monitoring camera whose video is currently displayed (in the present example, the monitoring camera 3002). If it is determined that the received detection notification has been issued from the monitoring camera whose video is currently displayed (YES of step S1107), processing proceeds to step S1108. On the other hand, if it is determined that the received detection notification has not been issued from the monitoring camera whose video is currently displayed (NO of step S1107), processing proceeds to step S1111.
In step S1108, the detection setting obtainment unit 101 obtains detection settings for the monitoring camera whose video is currently displayed (in the present example, the monitoring camera 3002) from that monitoring camera. In step S1109, the detection notification display unit 111 displays detection information that has been obtained in step S1108 and contributed to the detection together with the video. In step S1110, the detection notification display unit 111 displays the detection performed by the monitoring camera whose video is currently displayed.
In step S1111, the detection setting obtainment unit 101 receives and hence obtains detection settings of the monitoring camera from which the detection notification has been received (for example, the monitoring camera 3001) from that monitoring camera. In this way, if the detection notification is received from the monitoring camera 3001 (first monitoring camera) while the video of the monitoring camera 3002 (second monitoring camera) is being displayed, the detection setting obtainment unit 101 obtains detection settings set for the monitoring camera 3001.
In step S1112, the detection setting obtainment unit 101 obtains perspective projection conversion information from the monitoring cameras 3001 and 3002. The detection setting conversion unit 107 also converts detection settings that have contributed to the detection into settings for the monitoring camera whose video is currently displayed based on the perspective projection conversion information.
In step S1113, the detection notification display unit 111 displays the detection settings converted for the monitoring camera whose video is currently displayed (in the present example, the monitoring camera 3002) together with the video thereof.
In step S1114, the detection notification display unit 111 displays, on the screen, information showing that the detection was performed by another monitoring camera (in the present example, the monitoring camera 3001), that is to say, information showing the contents of the detection notification.
As an event detected by a certain camera is superimposed on a displayed video of another camera in the above manner, the result of detection can easily be checked even while the video of another camera is being viewed.
In the first embodiment, regions captured by a plurality of cameras overlap one another, and detection settings are configured with respect to the overlapping regions. However, the present invention is not limited in this way. For example, in the case where cameras are installed in similar positions on different but identically-designed floors of an office building such that each camera detects passage of people who come in and out of a door, the identical design allows mapping on the same planar map, and therefore the present invention can be applied.
Furthermore, while detection settings are configured for the second camera continuously after the configuration of detection settings for the first camera in the first embodiment, a time interval may be set between the initial configuration of detection settings for the first camera and the configuration of detection settings for the next camera. In this case, if information of the first camera for which detection settings have already been configured is not left in a setting terminal, a camera that satisfies the following conditions is searched for: a capturing range overlaps, and a detection line is set in an overlapping region. If a camera that satisfies these conditions is found, it is sufficient to obtain detection line information of the camera and then display the detection line information on a setting screen of a camera for which a detection line is to be set.
While the setting conversion information provided to the detection setting conversion unit 107 is stored in the cameras in the first embodiment, this information is not limited to being stored in the cameras. For example, the setting conversion information may be stored in the monitoring video recording server 4000 connected to the network, and the setting terminal may obtain the setting conversion information from the monitoring video recording server 4000. Also, the setting conversion information of all installed cameras may be prestored in the setting terminal.
While the setting conversion information is generated by obtaining a corresponding relationship between four points on images captured by the cameras and four points on a planar map in the first embodiment, the setting conversion information is not limited to being generated in this way. For example, the setting conversion information may be obtained from angle-of-view information, which is determined by lens characteristics of the cameras and settings of zoom positions of the cameras, and from information of the positions, heights and angles at which the cameras are installed.
While it is assumed that all cameras have a detection function in the first embodiment, the present invention is not limited to this configuration. For example, one wide-angle camera may capture an extensive panoramic view, and videos of four divided sections of the extensive panoramic view may be respectively captured by four telescopic cameras with a passage detection function. In this case, the following configuration may be adopted: the wide-angle camera does not store detection settings shown in
Embodiments of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions recorded on a storage medium (e.g., non-transitory computer-readable storage medium) to perform the functions of one or more of the above-described embodiment(s) of the present invention, and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more of a central processing unit (CPU), micro processing unit (MPU), or other circuitry, and may include a network of separate computers or separate computer processors. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2013-054136 filed on Mar. 15, 2013, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2013-054136 | Mar 2013 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
6737970 | Wuestefeld et al. | May 2004 | B2 |
8797403 | Suzuki et al. | Aug 2014 | B2 |
9384642 | Suzuki et al. | Jul 2016 | B2 |
20070035627 | Cleary | Feb 2007 | A1 |
20080018737 | Suzuki et al. | Jan 2008 | A1 |
20080219509 | White | Sep 2008 | A1 |
20100246901 | Yang | Sep 2010 | A1 |
20110157368 | Jo | Jun 2011 | A1 |
20130162838 | Huang | Jun 2013 | A1 |
20140185877 | Suzuki et al. | Jul 2014 | A1 |
20140267751 | Kitagawa | Sep 2014 | A1 |
Number | Date | Country |
---|---|---|
2002-374522 | Dec 2002 | JP |
2005-136557 | May 2005 | JP |
2006-339733 | Dec 2006 | JP |
2008-016898 | Jan 2008 | JP |
2009-017179 | Jan 2009 | JP |
2010-161632 | Jul 2010 | JP |
9842131 | Sep 1998 | WO |
Entry |
---|
By Lee, Chao-Yang; Lin, Shou-Jen; Lee, Chen-Wei; Yang, Chu-Sing. In Special Issue on Trusted Computing and Communications, Journal of Network and Computer Applications. May 2012 35(3):1067-1073. |
Eiichiro Kitagawa U.S. Appl. No. 14/187,569, filed Feb. 24, 2014. |
Chao-Yang Lee, et al., “An efficient continuous tracking system in real-time surveillance application,” Journal of Network and Computer Applications, vol. 35, Issue 3, May 2012, pp. 1067-1073. |
Number | Date | Country | |
---|---|---|---|
20140267737 A1 | Sep 2014 | US |