Patent Grant
11854288
The present invention relates to a security system using an image determining device.
There have been video monitoring devices that were developed for capturing images of elderly people living alone with a camera and determining presence or absence of a problem. Also, to protect the privacy of a person to be imaged by a camera, adding a mosaic pattern to each captured video image has been suggested (Patent Document 1, for example).
However, a conventional video monitoring device constantly captures images with a camera to detect a problem in a subject person, and monitors the subject person with another monitoring device. Therefore, complicated processes need to be performed.
An image determining device of the present invention comprises: an imaging unit that captures an image from a first direction, the image including a subject person; a first detector that detects size information from the image, the size information being about the subject person in the first direction; a second detector that detects position-related information, the position-related information being different from the information detected by the first detector; and a determining unit that determines a state of the subject person, based on a result of the detection performed by the first detector and a result of the detection performed by the second detector.
In this case, the second detector may detect position information with respect to a second direction, the second direction being different from the first direction. In this case, the second detector may detect position information about the subject person or a location near the subject person in the second direction. Also, the first detector may detect a size of the head of the subject person, and the determining unit may determine the state of the subject person by associating the second-direction position information with the size of the head of the subject person. Further, the image determining device may further comprise a storage unit that stores a reference value for the size information, in accordance with the second-direction position information.
Also, the image determining device of the present invention may be configured so that the first detector detects a size of the head of the subject person. Also, the determining unit may determine the state of the subject person, based on a plurality of images captured at different times. Also, the imaging unit may image the subject person at first predetermined intervals, and, in accordance with a result of the determination made by the determining unit, may image the subject person at second predetermined intervals, the second predetermined intervals being shorter than the first predetermined intervals. Further, the imaging unit may capture a still image.
The image determining device of the present invention may further comprise a determining sensor that determines whether the subject person exists. In this case, the imaging unit may perform the imaging in accordance with a result of the determination made by the determining sensor.
The image determining device of the present invention may further comprises a communicating unit that communicates with another device. In this case, when the communicating unit sends a result of the determination on the subject person to the another device, the determining unit may prohibit at least sending of the image captured at first predetermined intervals. Also, in this case, the determining unit may determine whether to allow sending of the image captured at second predetermined intervals, the second predetermined intervals being shorter than the first predetermined intervals.
Also, the image determining device of the present invention may further comprise a second power source that is different from a first power source supplying power to the imaging unit.
Also, the image determining device of the present invention may be configured so that the second detector is a relative position detector that detects information about a relative position between the imaging unit and the subject person. In this case, the imaging unit may include an optical system, and the relative position detector detects information about a relative position between the optical system and the subject person. Also, the relative position detector may detect an anticipated angle of the optical system with respect to the subject person. Further, the imaging unit may include an imaging element, and the relative position detector may detect the information about the relative position between the imaging unit and the subject person, based on a position of an image of the subject person captured by the imaging element.
The image determining device of the present invention may further comprise a correcting unit that corrects the size information about the subject person detected by the first detector, in accordance with a result of the detection performed by the relative position detector. Also, the image determining may be configured so that the imaging unit images a structural object and the subject person, and based on the imaged structural object and the imaged subject person, the determining unit determines the state of the subject person.
An image determining device of the present invention comprises: an imaging unit that captures an image from a first direction; an acquiring unit that acquires height information about a first subject person and height information about a second subject person; and an identifying unit that identifies the first subject person and the second subject person in accordance with the height information from the acquiring unit, when the captured image includes at least one of the first subject person and the second subject person.
In this case, the identifying unit may determine a state of at least one of the first subject person and the second subject person, in accordance with the height information from the acquiring unit. Also, the imaging unit may image the heads of the first subject person and the second subject person. Also, the imaging unit may perform imaging when at least one of the first subject person and the second subject person moves from an imaging area of the imaging unit to a non-imaging area.
The image determining device may be configured so that the acquiring unit acquires respective heights of the first subject person and the second subject person as the height information. Also, the acquiring unit may acquire the height information in accordance with a result of the imaging performed on the first subject person and the second subject person by the imaging unit. Also, the image determining device may be configured so that the imaging unit images a structural object, and based on the imaged structural object, the acquiring unit acquires the height information about at least one of the first subject person and the second subject person.
The image determining device of the present invention may be configured so that the imaging unit simultaneously images the first subject person and the second subject person. Further, the image determining device may be configured so that the imaging unit images the first subject person and the second subject person when the first subject person and the second subject person are in the same posture.
According to the present invention, the privacy of a subject person can be secured, and the state of the subject person can be determined with a simple structure.
An image determining device of this embodiment can be placed in a room of a house or office or the like, and a case where the image determining device is placed in a living room is described in the following.
The image determining device 1 is also connected to a lithium-ion cell 13 as a backup power source in case power from the primary source (a household power source in this embodiment) is not supplied due to a power failure or the like. The backup power source is not necessarily a lithium-ion cell. A household solar power system may be used, or a capacitor may be used in place of the lithium-ion battery.
A CPU 14 controls the entire image determining device 1, and is electrically connected to the respective components. It goes without saying that the CPU 14 and the setting unit 7 are electrically connected to each other even if the setting unit 7 is a remote controller.
(Imaging Unit 2)
The imaging unit 2 is placed in the center of the ceiling of the living room, and mainly captures images of the head of the subject person. In this embodiment, the height of the ceiling of the living room is 2.6 m (see
As shown in
The above wide-angle lens system includes: a first group including two negative meniscus lenses; a second group including a positive lens, a cemented lens, and an infrared cutoff filter; and a third group including two cemented lenses. A diaphragm is provided between the second group and the third group. The wide-angle lens system of this embodiment has a focal length of 6.188 mm in the entire system, and has a maximum view angle of 80 degrees.
The wide-angle lens system is not limited to the above three-group structure, and it is possible to arbitrarily change the number of lenses in each group, the lens structure, the focal length, and the angle of view.
Where the wide-angle lens system of this embodiment has a focal length of 6.188 mm as described above, and the diameter of the head of the subject person is 200 mm, the size of the head of the subject person imaged in the imaging element of the imaging unit 2 is 1.238 mm when the distance from the front-side focal point of the wide-angle lens system to the position of the head of the subject person is 1000 mm (when the subject person is standing). When the head of the subject person is lowered from that position by 300 mm, and the distance from the front-side focal point of the wide-angle lens system to the position of the head of the subject person becomes 1300 mm, the size of the head of the subject person imaged in the imaging element of the imaging unit 2 becomes 0.952 mm, and there is a 0.286 mm (23.1%) change in the image size.
Likewise, when the distance from the front-side focal point of the wide-angle lens system to the position of the head of the subject person is 2000 mm (when the subject person is in a semi-crouching position), the size of the head of the subject person imaged in the imaging element of the imaging unit 2 is 0.619 mm. When the head of the subject person is lowered from that position by 300 mm, the size of the head of the subject person imaged in the imaging element of the imaging unit 2 becomes 0.538 mm, and there is a 0.081 mm (13.1%) change in the image size. From that, it is apparent that as the distance from the front-side focal point of the wide-angle lens system to the head of the subject person becomes longer, the change in the image size becomes smaller.
As described above, with the image determining device 1 of this embodiment, the distance from the front-side focal point of the wide-angle lens system to the subject person is determined from the size of an image of the head of the subject person when the size of the head of the subject person is known. Accordingly, it is possible to determine the posture of the subject person (standing, semi-crouching, or lying on the floor) and a change in the posture while maintaining the privacy of the subject person.
When the distance from the front-side focal point of the wide-angle lens system to the position of the head of the subject person is increased from 1000 mm by 100 mm, the rate of change in the image size is as high as 9.1%. Therefore, with the image determining device 1 of this embodiment, two or more subject persons can be identified from the difference in height, if the height difference is at least approximately 100 mm.
When the distance from the front-side focal point of the wide-angle lens system to the position of the head of the subject person is increased from 2000 mm by 100 mm, the rate of change in the image size becomes as low as 4.8%. With the image determining device 1 of this embodiment, however, the change in the posture of the subject person can be detected even if the distance from the front-side focal point of the wide-angle lens system to the position of the head of the subject person becomes longer, which will be described later in detail.
Luminous flux incident on the wide-angle lens system enters the imaging element via the low-pass filter, and the circuit board converts the output of the imaging element into a digital signal. An image processing control unit formed with an ASIC performs image processing, such as white balance adjustment, sharpness adjustment, gamma correction, and gray level correction, on the image signal converted into the digital signal, and also performs image compression using JPEG or the like. The still image compressed using JPEG is recorded on the flash memory 11. To record a video image, the image signal should be subjected to MPEG processing, and should be recorded on the flash memory 11.
When there is something wrong with the subject person and the subject person is lying on the floor or the like, a zoom lens added to the above described wide-angle lens system continuously captures images of the eye and mouth, so that a check can be made to determine whether the eye is open based on pattern matching, whether the eyeball is moving, whether the mouth is moving, or whether the subject person is breathing based on movement of the mouth.
Further, the high-frequency component is extracted from the signal supplied from the imaging element, and the lens position in which the high-frequency component become largest is detected. In this manner, focus detection is performed, and this is known as contrast AF. Contrast AF is also applied to this embodiment, and part of the wide-angle lens system may be adjusted to an optimum position.
Normally, the imaging by the imaging unit 2 only involves capturing 48 still images per day if the imaging is performed once every 30 minutes. Accordingly, the power supplied to the image determining device 1 and the recording area in the flash memory 11 can be saved. Also, in accordance with date information supplied from the calendar unit 12, the imaging interval may be varied with the time of day. Specifically, the imaging interval may be 30 minutes in the daytime, and may be 20 minutes in the nighttime. If the imaging element is a CMOS, and an electronic shutter (a rolling shutter) is used, instead of a mechanical shutter, no noise would be generated at the time of imaging, and the sleep of the subject person would not be interrupted.
Also, instead of still images, short video images each lasting 3 to 5 seconds may be captured.
When the CPU 14 detects a problem from a captured image, the imaging unit 2 captures about a hundred successive images per minute, or captures about a hundred images at 15-second intervals, in accordance with an instruction from the CPU 14. The imaging interval may be set by using the setting unit 7.
The imaging time and ISO sensitivity at the time when the imaging unit 2 captures still images are determined by the illuminance detected by the illuminometer 4. When the living room is dark, the ISO sensitivity should be made higher, and the imaging time should be prolonged. In a case where the image determining device 1 is placed in a bedroom, and the subject person puts out all the lights when asleep, the LED 5 may illuminate the subject person, or the interface unit 8 cooperates with the lighting fixture in the bedroom, to illuminate the subject person for a few seconds (approximately 3 seconds) and secure the brightness necessary for imaging. The pyroelectric sensor 6 checks the position of the subject person, and the position of the LED 5 is adjusted by a drive unit (not shown) so as to illuminate the checked position with illuminating light. In this manner, the subject person can be efficiently illuminated. The LED 5 may be attached to the ceiling, or may be placed at any location in the bedroom. A light source other than a LED may be used. If the lighting fixture in the bedroom suffices, the LED 5 may not be provided. As the lighting fixture in the bedroom, a LED lamp may be used.
Further, captured still images of the subject person are only required to be clear enough to detect a problem of the subject person, and therefore, those still images differ from conventional still images for viewing. When imaging is performed in a dark-lit room, the infrared cutoff filter provided in the wide-angle lens system may be retracted by a retracting mechanism (not shown), so as to secure a sufficient amount of light.
(Communicating Unit 3)
The communicating unit 3 is formed with an antenna, a transmitter, a receiver, and the like. In this embodiment, the communicating unit 3 can be placed in a corner of a living room. The communicating unit 3 reports the occurrence or non-occurrence of a problem of the subject person to a designated other device (a portable telephone of a member of the family of the subject person or a personal computer at a security company), and receives audio information and the like from the other device. When there is nothing wrong with the subject person, the communicating unit 3 transmits a message indicating no problem, together with date information, to the designated other device, but does not transmit any captured image. In this manner, the privacy of the subject person can be protected. As will be described later, when there is something wrong with the subject person, images can be transmitted under predetermined conditions.
(Pyroelectric Sensor 6)
The pyroelectric sensor 6 is an infrared sensor. The pyroelectric sensor 6 detects infrared rays generated from a person, and outputs the detection result to the CPU 14. In this embodiment, the pyroelectric sensor 6 is placed so as to detect infrared rays via the wide-angle lens system. Pyroelectric sensors 6 may be provided in the living room.
Upon detecting the subject person entering the living room based on the output of the pyroelectric sensor 6, the CPU 14 instructs the imaging unit 2 to perform imaging, and may cause the imaging unit 2 to perform imaging thereafter at 30-minute intervals. If the subject person moves out of the living room and returns a few minutes later, there is no need to reset the time counter.
Upon detecting absence of the subject person from the living room based on the output of the pyroelectric sensor 6, the CPU 14 controls the imaging unit 2 not to perform imaging even at a scheduled time for imaging. In a case where the subject person does not enter the living room (and is not present in any other room) for a few days, and there is not a reported plan of absence from home in the schedule stored in the calendar unit 12, the CPU 14 reports the absence of the subject person and a possibility of a problem to the designated other device via the communicating unit 3.
(Calendar Unit 12)
As described above, the calendar unit 12 automatically updates calendar information indicating the year, month, day, hour, minute, and second, and stores plans of the subject person. A plan of the subject person may be input through the setting unit 7, or a date (duration) and destination may be input through the microphone 9. In that case, the calendar unit 12 is equipped with a voice recognition function. However, only a simple phonetic dictionary for dates and destinations and the like needs to be prepared, and accordingly, the system does not become larger. A plan may be voice-input through the microphone 9, in response to an inquiry (as to when and where) from the speaker 10.
The calendar unit 12 notifies the subject person of the next day's plan via the speaker 10 at night, and again notifies the next morning.
In accordance with a plan that is input to the calendar unit 12, the CPU 14 may set an alarm clock or display the plan on a television screen via the interface unit 8. Also, where a request to watch a television program is input, the CPU 14 may cause the television set to be be automatically turned on or cause the requested program to be recorded.
(Calibration)
In step 1, a communication device to which presence or absence of a problem of the subject person is to be reported is designated. Here, a mobile device such as a portable telephone or a personal computer of a family member, a personal computer of a security company, or the like can be designated. Also, privacy levels of the family member and a security company can be set. For example, in a case where the privacy level of the family member may be lowered, transfers of images captured when a problem occurs are allowed, or the later described mask processing can be cancelled or can be given a lower mask processing level. In a case where the privacy level of a security company is made higher than that of a family member, transfers of images captured when a problem occurs can be prohibited, and only transfers of images masked through the later described mask processing can be allowed.
Based on the designation made in step 1, the CPU 14 reports presence or absence of a problem of the subject person to the designated device via the communicating unit 3.
In step 2, a connection device such as a television set, an audio device, or a clock is designated. Specifically, a wired or wireless connection with a television set is made via the interface unit 8, for example, so that the activity schedule stored in the calendar unit 12 is displayed on the television set, the television set is turned on a date and a time designated in the calendar unit 12, or a requested program is recorded. Likewise, an audio device is turned on or an alarm clock is started on a date and a time stored in the calendar unit 12.
In step 3, the CPU 14 stores the relationship between a typical position in the living room and the position of the head of the subject person into the flash memory 11.
The CPU 14 captures an image of the head of the subject person with the imaging unit 2 when the subject person is standing in the living room as shown in
When the subject person is near the closet, the subject person is considered to be standing or in a crouching position. Therefore, the CPU 14 captures an image of the head of the subject person in a crouching position with the imaging unit 2, as shown in
Even in a case where the subject person is standing in the living room as shown in
As shown in
As described above, the image determining device 1 of this embodiment can detect the positional relationship between the imaging unit 2 and the subject person (the distance to the subject person and the anticipated angle with respect to the subject person). Accordingly, the posture of the subject person can be accurately detected, no matter which part of the living room the subject person is in.
In a case where there are two or more subject persons, the heights and the head sizes vary. Therefore, head sizes of each subject person in accordance with positions of the head in the Z-direction, and the position information in the X- and Y-directions are stored into the flash memory 11. In that case, images of the subject persons may be simultaneously captured by the imaging unit 2, or may be captured separately from one another.
Further, the heights and the head sizes (the diameters of the heads) of the subject persons may be input through the setting unit 7, and be stored into the flash memory 11. When a first subject person enters the living room through the door, and the head of the first subject person is imaged by the imaging unit 2, for example, the CPU 14 can recognize the first subject person entering the living room based on the size of the captured image and the height information stored in the flash memory 11, as the heights and the head sizes of the subject persons are stored in the flash memory 11. In this manner, the heights of two or more subject persons are stored into the flash memory 11, so that the subject persons can be identified based on the height differences even though there are no differences in the head size, while the privacy of the subject persons is maintained.
However, when the subject persons are in semi-crouching positions or are lying on the floor, there are no substantial differences in height, and the subject persons cannot be identified. Even in such a situation, one of particularly serious problems occurs when one of the subject persons is left in the living room.
If images of subject persons leaving the living room are captured near the door, the CPU 14 can determine which ones have left the living room and which one remains in the living room, using the output of the pyroelectric sensor 6. In this manner, the imaging unit 2 of this embodiment captures images of subject persons before the subject persons move out of the imaging area of the imaging unit 2, so that not only presence or absence of a person in the living room but also which one remains in the living room can be determined.
Also, a template image of a subject person (an image of the head of the subject person captured by the imaging unit 2) may be stored into the flash memory 11, and template matching may be performed, to identify the subject person who is lying on the floor.
When a subject person enters the living room, an image of the subject person standing immediately below the imaging unit 2 is captured, so that the subject person can be accurately identified based on the height.
Instead of inputting the height and the head size of a subject person, images of a structural object (such as the door) in the living room and the subject person may be captured, and, if the size of the structural object (the height of the door being 2 m, the width being 1 m, for example) is known, the height and the head size of the object may be estimated from the structural object. In this case, the size of the structural object should be recorded on the flash memory 11, and the imaging unit 2 preferably captures an image of each person in the same posture (a standing posture).
In a case where a subject person is a child, the height and the head size increase. Therefore, a reference still image of the child should be taken once a year, for example. When the layout of the room is changed, the relationship between head sizes corresponding to positions of the head of each subject person in the Z-direction and the position information in the X- and Y-directions should be updated.
Further, based on the output of the pyroelectric sensor 6, the CPU 14 stores the times of the day at which the subject person spends time in the living room on weekdays and holidays are stored into the flash memory 1.
In step 4, masking is performed on the captured image. If the subject person does not request the masking, this step may be skipped. The masking is performed by lowering the resolution of the entire captured image, or adding a mosaic pattern to the parts other than the head or lowering the resolution of the parts other than the head, or detecting the flesh-colored portions from the captured image and adding a mosaic pattern to the flesh-colored portions or lowering the resolution of the flesh-colored portions, for example.
In step 5, the settings, such as the time intervals at which an image is captured, and the volume of sound from the speaker 10, are checked. When sound is emitted from the speaker 10 or the voice of the subject person is collected by the microphone 9, the CPU 14 controls the interface unit 8 to lower the volume of sound emitted from the television set and the audio device.
(Operation Flowchart)
Based on the output of the pyroelectric sensor 6, the CPU 14 determines whether the subject person is present in the living room (step 1). If the subject person is present in the living room, the CPU 14 moves on to step 2. If the subject person is not present in the living room, the CPU 14 repeats step 1.
The CPU 14 controls the calendar unit 12, to start the timing (step 2). Step 2 is the step for setting the intervals at which a still image is captured when the subject person is in a normal state, in conjunction with step 8, which will be described later.
The CPU 14 determines whether the number of times a still image has been captured is a predetermined number of times (step 3). Step 3 is the step for taking measures to plot the next move when the state of the subject person cannot be determined even after continuous still images of the subject person are captured in a case where it is difficult to determine the state of the subject person as described later. In this embodiment, the predetermined number of times a still image is to be captured is 300 to 500, for example, and the imaging time is approximately 3 to 5 minutes.
The CPU 14 checks the position of the subject person, based on the output of the imaging element of the imaging unit 2 (step 4). This is to correct for the influence of an aberration of the wide-angle lens system in accordance with the position of the subject person. In the following, the influence of an aberration of the wide-angle lens system is described.
As described above, the wide-angle lens system of this embodiment is designed to have a view angle of 80 degrees, and a focal length of 6.188 mm. Therefore, where the height of the ceiling of the living room is 2.6 m, and the height of the subject person is 1.6 m, a circle (approximately 100 m2) having a radius of 5.67 m from a point immediately below the wide-angle lens system is the imaging area of the imaging unit 2. Accordingly, as long as the room is a standard-sized room, the head of the subject person can be imaged by the wide-angle lens system of this embodiment, no matter which part of the room the subject person is in.
When the subject person is present almost immediately below the wide-angle lens system (when the angle of view is low), the image size of the head of the subject person is proportional to f·tan θ (f being the focal length). Therefore, the relationship between the height of the head from the floor and the size (the diameter) of the image is as shown in
Next, a case where the subject person is in an intermediate view-angle position (at a view angle of approximately 70 degrees) is described.
Lastly, a case where the subject person is in an outermost peripheral view-angle position (at a view angle of approximately 80 degrees) is described. As shown in
If two or more wide-angle lens systems are provided, the accuracy of detection performed when the subject person is present in a corner of the room can be made higher, and it is possible to cope with a larger living room.
In a case where the number of times imaging has been performed is 300 to 500, which has been determined in step 3, a still image is captured in step 5, and the amount of image characteristics is calculated in step 6. Specifically, the size of the head in the Z-direction is associated with the X- and Y-directions, and is calculated from the captured still image.
The CPU 14 determines whether the state of the subject person can be determined from the position of the subject person in the X- and Y-directions and the calculated head size (step 7).
In such a case, the CPU 14 determines that a check cannot be made to determine whether the subject person is in a normal state, and returns to step 3. A still image is then captured several hundred times in several minutes as described above, and attention is paid to the head of the subject person as indicated by the dotted-line circles. A check is then made to determine whether the head is moving.
If the CPU 14 determines that the state of the subject person can be determined after the predetermined number of imaging operations, the CPU 14 determines the state of the subject person in step 8.
If the CPU 14 determines that the subject person is in a normal state, the CPU 14 stands by until a predetermined period of time (15 minutes since the start of the timing, for example) passes in step 9. After that, the CPU 14 returns to step 1. Even when the subject person is in a normal state, the CPU 14 notifies the designated device that there is no problem, at intervals (at 8-hour intervals; 7 am, 3 pm, and 11 pm, for example) set in the setting unit 7.
If the CPU 14 determines that the subject person is in an abnormal state, the CPU 14 moves on to step 10.
In step 10, the CPU 14 notifies the designated device that the subject person is in an abnormal state.
The CPU 14 determines whether there is a request for an image from a portable telephone or a personal computer of a family member with a low privacy level that is set in advance (step 11). If there is a request for an image, an image is transferred in step 12, and an instruction from the designated device is awaited (step 13). Even in transferring an image in step 12, the CPU 14 prohibits a transfer of any still image captured while the subject person is determined to be in a normal state, and transfers an image captured when the subject person is determined to be in an abnormal state. In this manner, the privacy of the subject person is secured.
If there is not a request for an image, the CPU 14 moves on to step 13, and awaits an instruction from the designated device. When the image determining device of this embodiment transfers an image, either a still image or a video image can be transferred.
Meanwhile, if the CPU 14 determines, in step 3, that the number of imaging operations performed exceeds the predetermined number, the CPU 14 moves on to step 14. The CPU 14 then uses the speaker 10, to talk to the subject person, “Are you all right?” or “How are you feeling?.” At this point, the CPU 14 controls the interface unit 8 to lower the volume of the sound source device such as an audio device (or to mute the audio).
The CPU 10 collects the voice of the subject person via the microphone 9 (step 15), and uses an audio dictionary (not shown) to carry out an audio analysis (step 16). In this case, the phrases the subject person would use are limited to rather typical ones, such as “I'm all right,” “What?,” “I feel ill,” and “Help me.” Therefore, the audio dictionary does not become large (complicated), and there is no need to carry out a sophisticated audio analysis such as breaking down a conversation into individual words.
The speaker 10 and the microphone 9 are directed toward the position of the subject person by a drive unit (not shown) such as a stepping motor, so that the inquiry through the speaker 10 and the voice collection by the microphone 9 can be efficiently conducted.
If the CPU 14 recognizes a phrase indicating a normal state from the subject person, the CPU 14 returns to step 1. If there is a phrase indicating an abnormal state from the subject person or there is no response from the subject person, the CPU 14 determines that the subject person is in an abnormal state, and moves on to step 10 (step 17).
Prior to the notification of an abnormal state in step 10, an inquiry through the microphone 9 may be made as shown in step 14. In this manner, a more specific notification indicating not only that the subject person is having a problem but also that the subject person is having a headache or is unable to speak or the like can be sent to the designated device.
In the above described embodiment, the imaging unit 2 is attached to the ceiling, and images of the head of a subject person are captured from above. However, the imaging unit 2 may be installed in a living room or a bedroom so that images of a subject person can be captured from the side. By capturing images of the subject person from the side, movement of the eyes, the mouth, the nose, and the like can be easily determined. Further, still images captured by the imaging unit 2 attached to the ceiling and still images of the subject person captured from the side can be used at the same time in determining whether the subject person is in a normal state. In this manner, determination with higher precision can be performed.
Also, in the above described embodiment, a check is made to determine whether the subject person is present in a living room by the pyroelectric sensor 6. However, the intervals at which the imaging unit 2 performs imaging may be shortened, and, based on the results of the imaging, a check may be made to determine whether the subject person is present in the living room. In this case, the pyroelectric sensor 6 may not be provided.
In the above described embodiment, the image determining device of this embodiment is installed in a living room or a bedroom. However, the image determining device may of course be installed in a hallway, a restroom, a bathroom, or the like, and can also be used in an office or a school.
While the exemplary embodiments of the present invention have been illustrated in detail, the present invention is not limited to the above-mentioned embodiments, and other embodiments, variations and modifications may be made without departing from the scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-001901 | Jan 2010 | JP | national |
| 2010-001902 | Jan 2010 | JP | national |
| 2010-001903 | Jan 2010 | JP | national |
This application is a Continuation Application of application Ser. No. 16/274,631 filed on Feb. 13, 2019, which is a Continuation Application of application Ser. No. 15/727,271 filed on Oct. 6, 2017, which is a Divisional of application Ser. No. 14/700,960 filed on Apr. 30, 2015, which is a Divisional of application Ser. No. 13/513,704 filed on Jun. 4, 2012, which is a National Phase Application of International Application No. PCT/JP2010/073748 filed on Dec. 28, 2010, which claims the benefit of Japanese Patent Application No. 2010-001903 filed on Jan. 7, 2010, Japanese Patent Application No. 2010-001902 filed on Jan. 7, 2010, and Japanese Patent Application No. 2010-001901 filed on Jan. 7, 2010. The disclosure of each of the prior applications is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3425387 | Mitchell | Feb 1969 | A |
| 20030058111 | Lee et al. | Mar 2003 | A1 |
| 20030117279 | Ueno et al. | Jun 2003 | A1 |
| 20040179736 | Yin | Sep 2004 | A1 |
| 20040211883 | Imagawa et al. | Oct 2004 | A1 |
| 20050289197 | Kan et al. | Dec 2005 | A1 |
| 20060291694 | Venetianer et al. | Dec 2006 | A1 |
| 20070265533 | Tran | Nov 2007 | A1 |
| 20080285797 | Hammadou | Nov 2008 | A1 |
| 20100145998 | Poce | Jun 2010 | A1 |
| 20120025989 | Cuddihy et al. | Feb 2012 | A1 |
| 20130184592 | Venetianer | Jul 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 1332840 | Jan 2002 | CN |
| 1503194 | Jun 2004 | CN |
| 1647535 | Jul 2005 | CN |
| 101404758 | Apr 2009 | CN |
| 1 148 316 | Oct 2001 | EP |
| H08-150125 | Jun 1996 | JP |
| H09-65325 | Mar 1997 | JP |
| H11-73582 | Mar 1999 | JP |
| 2000-099863 | Apr 2000 | JP |
| 2000-134611 | May 2000 | JP |
| 2001-052270 | Feb 2001 | JP |
| 2001-148011 | May 2001 | JP |
| 2001-325677 | Nov 2001 | JP |
| 2002-109666 | Apr 2002 | JP |
| 2002-142214 | May 2002 | JP |
| 2004-129280 | Apr 2004 | JP |
| 2008-052626 | Mar 2008 | JP |
| 2008-052631 | Mar 2008 | JP |
| 2008-092538 | Apr 2008 | JP |
| 3143307 | Jul 2008 | JP |
| 2008-282181 | Nov 2008 | JP |
| 2009-140407 | Jun 2009 | JP |
| Entry |
|---|
| Feb. 15, 2011 International Search Report issued in International Patent Application No. PCT/JP2010/073748. |
| Aug. 25, 2014 Office Action issued in Japanese Application No. 2010-001901. |
| Jan. 28, 2014 Office Action issued in Japanese Application No. 2010-001901. |
| Jan. 28, 2014 Office Action issued in Japanese Application No. 2010-001902. |
| Jan. 28, 2014 Office Action issued in Japanese Application No. 2010-001903. |
| May 16, 2014 Office Action issued in Chinese Application No. 2010-80055107.6. |
| Sep. 16, 2013 Office Action issued in Chinese Application No. 2010-80055107.6. |
| Jun. 30, 2014 Office Action issued in U.S. Appl. No. 13/513,704. |
| Oct. 3, 2016 Office Action issued in U.S. Appl. No. 14/700,960. |
| May 8, 2017 Office Action issued in U.S. Appl. No. 14/700,960. |
| Nov. 30, 2017 Office Action issued in U.S. Appl. No. 15/727,271. |
| Yuan-Fang Wang, Edward Y. Chang, Ken P. Cheng, “A Video Analysis Framework for Soft Biometry Security Surveillance”, Nov. 11, 2005, VSSN'05, ACM 1-59593-242-0/05/0011, pp. 71-78. |
| Apr. 24, 2018 Office Action issued in Indian Application No. 6007/DELNP/2012. |
| Jul. 9, 2018 Office Action issued in U.S. Appl. No. 15/727,271. |
| Jun. 20, 2018 Office Action issued in Chinese Patent Application No. 201510594449.X. |
| Antonio Criminisi; Andrew Zimmerman; Luc van Gool; Simon Bramble; and David Compton, “A New Approach to Obtain Height Measurements from Video”, Jan. 1, 21998, Proceedings of SPIE—The International Society of Optical Engineering. |
| Shaou-Gang Miaou; Pei-Hsu Sung;, and Chia-Yuan Huang, “A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information”, Apr. 2-4, 2006, Proceedings of the 1st Distributed Diagnosis and Home Healthcare (D2H2) Conference, Arlington, Virginia, pp. 39-42. |
| Erno Jeges; Istvan Kispal; and Zoltan Hornak, “Measuring Human Height Using Calibrated Camera”, 2008, IEEE, Krakow, Poland, May 25-27, 2008, pp. 755-760. |
| Jan. 14, 2019 Office Action issued in Chinese Patent Application No. 201510594449.X. |
| Jul. 23, 2019 Office Action issued in Chinese Patent Application No. 201510594449.X. |
| Jul. 30, 2020 Office Action issued in U.S. Appl. No. 16/274,631. |
| Jan. 4, 2021 Hearing Notice issued in Indian Patent Application No. 6007/DELNP/2012. |
| Mar. 2, 2021 Notice of Allowance issued in U.S. Appl. No. 16/274,631. |
| Number | Date | Country | |
|---|---|---|---|
| 20210286987 A1 | Sep 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14700960 | Apr 2015 | US |
| Child | 15727271 | US | |
| Parent | 13513704 | US | |
| Child | 14700960 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16274631 | Feb 2019 | US |
| Child | 17336550 | US | |
| Parent | 15727271 | Oct 2017 | US |
| Child | 16274631 | US |
