Patent Application
20200234060
The present application claims the priority benefit of Japanese Patent Application No. 2019-6218 filed on Jan. 17, 2019, the subject matter of which is hereby incorporated herein by reference.
The present invention relates to an image processing device and an image processing method.
Recently, many imaging devices configured to capture monitored objects (e.g. persons, products, or properties) have been located at many places. Users doing monitoring objects (e.g. watchmen, inspectors, or observers) may observe images captured by imaging devices to confirm the state of a monitored object. For example, a monitoring system is introduced for the purpose of users (e.g. parents who need to watch children) to monitor children left in nursery with caregivers. Users may use terminal devices (e.g. smartphones) to communicate with a monitoring system so as to display images capturing children on screen, thus confirming the state of children.
Patent Document 1 discloses a person image searching method configured to share and search images in consideration of privacy protection. This method is configured to search for a database holding personal characteristic information using a search key having a facial photograph of a person while taking privacy protection measures for other facial images of other persons. Patent Document 2 discloses an image conversion method configured to enlarge or reduce part of panorama pictures using different enlargement ratios at desired regions of panorama pictures. Patent Document 3 discloses an automatic tracking method configured to capture objects in an enlarged manner while automatically tracking objects. Patent Document 4 discloses a technique to process captured images derived from selected target objects in a field of view combined with a zoom-in close-up lens. Patent Document 5 discloses a method configured to allow users to preserve privacy and control the use of individual images and/or personal information, wherein if not authorized the use of individual images, a privacy control server may generate modified images instead of individual images.
Patent Document 1: Japanese Patent Application Publication No. 2014-89625
Patent Document 2: Japanese Patent Application Publication No. 2013-12893
Patent Document 3: Japanese Patent Application Publication No. 2008-219467
Patent Document 4: US 2007/0109411 A1
Patent Document 5: US 2016/0381013 A1
A monitoring system may be given a user instruction to enlarge and display part of images reflecting monitored objects. To generate an enlarged image reflecting a monitored object, it is possible to carry out zoom-in control to reduce a viewing angle by increasing a focal distance. Due to the narrowed viewing angle, it is difficult for the monitoring system to generate images each normally covering a prescribed wide imaging range, and therefore it is impossible to record captured images of objects each normally covering a prescribed wide imaging range irrespective of zoom-in control. However, none of the related arts provides any solutions to this problem.
The present invention is made under the aforementioned circumstances, and therefore the present invention aims to provide an image processing device and an image processing method to solve the problem.
In a first aspect of the invention, an image processing device including a captured-image recorder configured to record a captured image acquired from an imaging device; a monitor request detector configured to detect a monitor request made by a user with respect to a monitored object reflected in the captured image; a monitored-object specifying part configured to specify the monitored object in the captured image according to the monitor request; and an output part configured to generate an output image including the monitored object such that the user can monitor the monitored object in the output image.
In a second aspect of the invention, an image processing method includes the steps of: recording a captured image acquired from an imaging device; detecting a monitor request made by a user with respect to a monitored object reflected in the captured image; specifying the monitored object in the captured image according to the monitor request; and generating an output image including the monitored object such that the user can monitor the monitored object in the output image.
According to the present invention, it is possible to provide users with captured images of objects designated by users in an enlarged manner and normally covering a prescribed wide imaging range, thus recording desired images of objects on storage media.
The present invention will be described in detail by way of embodiments with reference to the drawings.
The controller 11 is configured to control other functional parts. The acquisition part 12 is configured to acquire or receive images captured by the camera 2 and to further receive information transmitted from other devices such as the terminal 3. The captured-image recorder 13 is configured to record on the database 104 the captured images acquired by the acquisition part 12. The monitor request detector 14 is configured to detect a monitor request from a user who needs to monitor a monitored object reflected in the captured images. The monitored-object specifying part 15 is configured to specify the position of a monitored object in the captured image being monitored by the user according to the monitor request. The output part 16 is configured to generate an output image including a monitored object and thereby output it to the user such that the user can monitor the output image. For example, the output part 16 transmits the output image to the terminal 3 held by the user.
According to the present embodiment, the image processing device 1 generates the output image in which a display ratio of a monitored object will be higher than a display ratio of the captured image. For example, the monitored object is a predetermined person being monitored by the user. Based on characteristics of a person specified according to the monitor request, the image processing device 1 specifies the monitored object being monitored by the user. In this connection, when the output image includes a predetermined monitored object different than the monitored object being monitored by the user, the image processing device 1 may carry out a process to obscure the display area of the predetermined monitored object. In addition, the image processing device 1 may generate replication data of the captured image for each user so as to generate the output image from the replication data. Alternatively, the image processing device 1 may divide the captured image into a plurality of divided regions so as to specify a divided region (i.e. any one of divided regions) specifying a monitored object, thus generating the output image based on the image data of the divided region.
Accordingly, the image processing device 1 is able to provide the user with the output image displaying the monitored object designated by the user in an enlarged manner and to record the captured image normally covering a prescribed wide imaging range.
When a person is selected as the monitored object or another predetermined monitored object, the image processing device 1 is able to generate the output image including an enlarged image of a person (i.e. a monitored object designated by the user) in an understandable manner and to ensure privacy protection due to the process of obscuring the other person when reflected in the output image.
Since the output image is configured of image data replicating the captured image, it is possible to directly record the captured image including a prescribed imaging range on recording media, thus implementing a process to prevent an unrecorded region within the prescribed imaging range.
When the camera 2 has a viewing angle of 180 degrees of more like a fisheye lens, it is possible to record the captured image normally covering a prescribed wide imaging range using one or few cameras 2 located at predetermined positions.
The acquisition part 12 of the image processing device 1 acquires the captured image (step S101). The acquisition part 12 outputs the captured image to the captured-image recorder 13. The captured-image recorder 13 sequentially acquires configuration data of captured images to generate and record moving images (based on captured images) on the database 104 (step S102).
The image processing device 1 has a web-server function, according to which the image processing device 1 may receive a request of information from the terminal 3 to deliver the requested information to the terminal 3. In the present embodiment, a monitor request may represent the request of information transmitted from the terminal 3 to the image processing device 1 while the output image reflecting a monitored object such as a person designated by the user (which is generated based on the captured image of the camera 2) may represent the delivered information.
Upon receiving an access from the terminal 3, the image processing device 1 establish communication with the terminal 3 (step S103). The user operates the terminal 3 to make a monitor request for monitoring the user's child in the nursery, and therefore the terminal 3 transmits the monitor request to the image processing device 1. The monitor request includes at least the user ID. When the image processing device 1 receives the monitor request, the acquisition part 12 acquires and sends the monitor request to the monitor-request detector 14. According to the monitor request, the monitor request detector 14 detects that the monitor request is made by the user of the terminal 3 (step S104). The monitor request detector 14 retrieves the user ID from the monitor request so as to send the user ID to the monitored-object specifying part 15.
The monitored-object specifying part 15 acquires the characteristic information representative of a face of a user's child and the privacy-protection flag which are stored on the database 104 in connection with the user ID (step S105). In addition, the monitored-object specifying part 15 replicates the captured image stored on the database 104 (step S106). Due to sequential accumulation of moving images added with new images, the monitored-object specifying part 15 sequentially replicate newly-added captured images to produce replication data.
The monitored-object specifying part 15 determines whether the characteristic information of a child designated by the user is included in each of still images constituting the newest moving image among moving images representing the replication data of captured images (step S107). The monitored-object specifying part 15 sequentially specify still images constituting the newest moving image captured in a predetermined period so as to repeat the process to determine whether the characteristic information of a child designated by the user is included in each still image. When the characteristic information of the user's designated child is included in a still image, the monitored-object specifying part 15 specifies the position of the user's designated child in the still image (step S108). Subsequently, the monitored-object specifying part 15 outputs to the output part 16 the still image including the characteristic information of the user's designated child, the position (or coordinates) indicating the characteristic information of the user's designated child, and the privacy-protection flag. The monitored-object specifying part 15 determines whether a predetermined period has elapsed after the step S107 for determining that the still image does not include the characteristic information of the user's designated child (step S116). Upon determining a lapse of the predetermined period, the flow proceeds to step S115.
The output part 16 sequentially acquires still images including the characteristic information of the user's designated child from the monitored-object specifying part 15 as well as the position (or coordinates) indicating the child characteristic information in each still image. In the still image, the output part 16 detects the position indicating the child characteristic information and then sequentially extracts small areas of images displaying the child based on the detected position (step S109).
The output part 16 determines whether the privacy-protection flag is “ON” (step S110). When the privacy-protection flag is “ON”, the output part 16 determines whether the output image (e.g. a moving image) includes the characteristic information of other person's faces different from the characteristic information of the user's designated child (step S111). When the output image includes the characteristic information of other' persons faces (e.g. the facial characteristics of the child 6b in
Specifically, the output part 16 generates an output image (i.e. a moving image) sequentially connecting still images (step S113), and therefore it is possible to generate the moving image including an enlarged image of the child's face. When the captured image is generated according to the 8K UHDTV (Ultra-High-Definition Television) specification and divided into sixteen small areas, for example, each small area represents a moving image having the same quality as the Full HD (High Definition) specification. The output part 16 transmits the output image (i.e. the moving image) to the terminal 3 (step S114). Upon receiving the output image, the terminal 3 displays the output image on screen. Accordingly, the user is able to monitor the state of the user's child using the terminal 3.
The controller 11 determines whether to exit the process (step S115). Upon receiving a user's logout request from the terminal 3, the controller 11 determines to exit the process. Alternatively, the controller 11 may exit the process when a predetermined time has elapsed after step S116 for determining that the output image does not include the child's characteristic information. When the controller 11 does not exit the process, the controller 11 repeats a series of steps starting with step S106.
Upon receiving a plurality of monitor requests from different users in the above process, the monitored-object specifying part 15 should carry out the same process for each user. That is, the monitored-object specifying part 15 generates a plurality of replication data in correspondence with the number of users so as to carry out the same process using a plurality of replication data representing a plurality of captured images in parallel. Accordingly, it is possible for each user to monitor the user's child according to the user's own monitor request even when other users operate their terminals 3 to make monitor requests to access the image processing device 1.
According to the above process, it is possible to record captured images without being processed at all on the database 104. Therefore, the image processing device 1 is able to provide users with output images and to record captured images each normally covering a prescribed wide imaging range.
According to the aforementioned process in which an output image (e.g. a moving image) includes the characteristic information representing other faces than the characteristic information of the user's designated child, it is possible to randomly convert pixel values and to thereby obscure regions of other faces in the output image. That is, it is possible to obscure other faces than the user's designated face, and therefore it is possible to ensure privacy protection with respect to other children than the child whom the user needs to monitor.
When the camera 2 has a fisheye lens having a viewing angle of 360 degrees, it is possible to record a captured image normally covering a prescribed wide imaging range using a single or few cameras 2. In addition, it is possible for each user to confirm an output image (e.g. a moving image) including an enlarged image of the user's child, thus monitoring the state of the user's child.
The foregoing descriptions refer to the monitoring system 100 installed in a nursery, however, it is possible to set up the monitoring system 100 in other places such as stores. In addition, it is possible to set up the monitoring system 100 at a predetermined position to monitor a plurality of stores. In this connection, the image processing device 1 may specify other monitored objects than persons in captured images. When the user designated a monitored object representing a bag held by the user, for example, the image processing device 1 generates an output image (e.g. a moving image) configured to enlarge an image of an object such as a bag designated by the user according to a monitor request, thus sending the output image to the terminal 3. According to the process, for example, the image processing device 1 may output to a certain device designated by the user an output image (e.g. a moving image) used to monitor a predetermined object such as a bag designated by the user in an event venue.
In the foregoing embodiment, the image processing device includes a computer system therein, and therefore programs achieving the foregoing functions and processes are stored on computer-readable storage media; hence, the computer system reads and execute programs to implement the foregoing functions and processes. Herein, computer-readable storage media may refer to magnetic disks, magneto-optical disks, CD-ROM, DVD-ROM, semiconductor memory or the like. In addition, it is possible to deliver computer programs to computers through communication lines, and therefore computers may execute programs.
The foregoing programs may achieve part of the foregoing functions. Alternatively, the foregoing programs may be differential files (or differential programs), which can be combined with pre-installed programs of a computer system to achieve the foregoing functions.
The present embodiment relates to a monitoring system configured to monitor an object (e.g. a person) using an image captured by a camera located at a predetermined position such as a nursery room. However, the monitoring system can set up in various sites such as airports, railway stations, distribution centers, stores, public facilities, private facilities, convention halls and the like. In addition, the monitoring system can monitor various types of objects such as visitors, passengers, customers, luggage, packets, products and the like. Therefore, the monitoring system, the image processing device, and the image processing method can be modified in various ways according to their locations, needs and requirements.
Lastly, the present invention is not necessarily limited to the present embodiment, and therefore the present invention may embrace various design changes and modifications within the scope of the invention as defined in the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2019-006218 | Jan 2019 | JP | national |
