Patent Grant
9298258
The present invention relates to a display apparatus and a control method.
Recently, with the expansion of a flat-screen television set market, there has been an increase in a demand for an image displaying apparatus, such as a large-screen television set, which is to be installed in a living room. In such a situation, there has been proposed an image displaying apparatus including various functions.
Because a user can view an image displayed on the image displaying apparatus at any position, sometimes the user views the image displayed on the image displaying apparatus at an improper position.
For example, sometimes a child approaches a television screen because children tend to overdose easily on a broadcast content. When the child continuously views the image while approaching the television screen, a focus is fixed, which causes a risk of weakening eyesight or generating epilepsy. When the child excessively approaches the television screen, the television set possibly falls, causing injury to the child. Because children hardly recognize such risks, it is necessary to keep them away from the television screen. Further, possibly they approach the television screen to break a display portion of the television set, and they are injured by the display portion of the broken television set, it is necessary to keep them away from the television screen.
In light of the foregoing, an object of the present invention is to provide novel and improved display apparatus and control method capable of preventing the user from viewing the image at the improper viewing position.
According to an aspect of the present invention, in order to achieve the above-mentioned object, there is provided a display apparatus including: an imaging unit that captures a moving image in a predetermined range with respect to an image display direction; an image analyzer that analyzes the moving image captured by the imaging unit, and calculates a position of a target that should be guided to a proper viewing position; and a display controller that causes a display unit to perform display in order to guide the target to the proper viewing position when the target position calculated by the image analyzer is at an improper viewing position.
The display controller may cause the display unit to display a message that guides the target to the proper viewing position.
The display controller may cause the display unit to display a graph illustrating a distance between the target and the display unit.
The display controller may lower luminance of the display unit.
The display apparatus may further include a sound controller that causes a sound output unit to output a tone in order to guide the target to the proper viewing position when the target position detected by the image analyzer is at the improper viewing position.
When a determination of whether the target should be guided to the proper viewing position is made in analyzing the moving image captured by the imaging unit, the image analyzer may make a determination by a combination of a determination of whether the target should be guided to the proper viewing position and a determination of whether the target is not needed to be guided to the proper viewing position.
The image analyzer may make a determination using a past determination history when a determination of whether the target should be guided to the proper viewing position is made in analyzing the moving image captured by the imaging unit.
The image analyzer may calculate the position using a past calculation history when a position of the target that should be guided to the proper viewing position is calculated in analyzing the moving image captured by the imaging unit.
According to another aspect of the present invention, in order to achieve the above-mentioned object, there is provided a control method including: capturing a moving image in a predetermined range with respect to an image display direction; analyzing the captured moving image to calculate a position of a target that should be guided to a proper viewing position; and causing a display unit to perform display in order to guide the target to the proper viewing position when the calculated target position is at an improper viewing position.
As described above, according to the present invention, the novel and improved display apparatus and control method capable of preventing the user from viewing the image at the improper viewing position can be provided.
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the appended drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is omitted.
The explanation will be made in the following order.
<1. An Embodiment of the Invention>
[1-1. Structure of image displaying apparatus]
[1-2. Configuration of controller]
[1-3. Child approach preventing processing]
[1-1. Structure of Image Displaying Apparatus]
A configuration of an image displaying apparatus according to an embodiment of the present invention will be described below.
As illustrated in
The image displaying apparatus 100 of the embodiment of the present invention includes a sensor 106 in a lower center portion of the display panel 102. The sensor 106 detects presence or absence of a human in front of the image displaying apparatus 100.
In
Although not illustrated in
The appearance of the image displaying apparatus 100 has been described above with reference to
As illustrated in
The controller 110 is configured to include an image input unit 112, an image processor 114, a viewing state analyzer 116, a viewing state recorder 118, a system optimization processor 120, and a system controller 122.
The display panel 102 is an example of the display unit of the present invention, and displays the still image or the moving image based on a panel driving signal. In the embodiment, in the display panel 102, the still image or the moving image is displayed on a liquid crystal display panel. Needless to say, the display panel 102 is not limited to the liquid crystal display panel. The display panel 102 may display the still image or the moving image using a self-luminous display device, such as an organic EL (Electro Luminescence).
As described above, the imaging units 104 are included in the upper center portion and right and left center portions of the display panel 102 that displays the still image or the moving image. The imaging units 104 capture the moving image with respect to the direction in which the image displaying apparatus 100 displays the moving image on the display panel 102 when the panel driving signal is supplied to the display panel 102 and the moving image is displayed on the display panel 102. The imaging unit 104 may capture the moving image using a CCD (Charge Coupled Device), or capture the moving image using a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The moving image captured by the imaging unit 104 is transmitted to the controller 110.
As described above, the sensor 106 is included in the lower center portion of the display panel 102 that displays the still image or the moving image. For example, the sensor 106 detects the presence or absence of a human in front of the image displaying apparatus 100. The sensor 106 can detect a distance between the image displaying apparatus 100 and the human when the human is present in front of the image displaying apparatus 100. The detection result and distance information from the sensor 106 are transmitted to the controller 110. The speaker 108 is an example of the sound output unit of the invention, and outputs the sound based on a sound output signal.
The controller 110 controls an operation of the image displaying apparatus 100. Each unit of the controller 110 will be described below.
The image input unit 112 receives the moving image captured by the imaging unit 104. The moving image received by the image input unit 112 is transmitted to the image processor 114, and used in image processing performed by the image processor 114.
The image processor 114 is an example of the image analyzer of the present invention. The image processor 114 performs various pieces of image processing to the moving image, which is captured by the imaging unit 104 and transmitted from the image input unit 112. The pieces of image processing performed by the image processor 114 include processing of detecting a dynamic body included in the moving image captured by the imaging unit 104, processing of detecting the number of humans included in the moving image, and processing of detecting a face and a facial expression that are included in the moving image. Results of various pieces of image processing performed by the image processor 114 are transmitted to the viewing state analyzer 116, and used to analyze the presence or absence of a person who views the image displaying apparatus 100 and a viewing state and a viewing position of the person who views the image displaying apparatus 100.
In the image processor 114, for example, a technology disclosed in Japanese Patent Application Laid-Open 2007-65766 (JP-A) or JP-A 2005-44330 can be used as the processing of detecting the face included in the image. The face detecting processing will briefly be described below.
In order to detect the user's face from the image, a position of the face, a size of the face, and a direction of the face are detected in the supplied image. When the position and size of the face are detected, a portion of the face image can be cut out from the image. Facial characteristic portions (facial characteristic positions) such as an eyebrow, an eye, a nose, and a mouth are detected from the cut-out face image and the information on the face direction. In order to detect the facial characteristic positions, for example, a method called AAM (Active Appearance Models) can be adopted to detect the characteristic position.
When the facial characteristic positions are detected, a local characteristic amount is calculated with respect to each detected facial characteristic position. The local characteristic amount is calculated, and the calculated local characteristic amount is stored along with the face image, which allows the face to be identified from the image captured by the imaging unit 104. For example, a technology disclosed in JP-A 2007-65766 or JP-A 2005-44330 can be used in the face identifying method. Therefore, the detailed description is omitted herein. Whether the face of the supplied image is a male or female, or how old the person is can be determined by the face image and the facial characteristic position. When the face information is previously recorded, the person of the supplied image is searched from the recorded and an individual can be identified.
The viewing state analyzer 116 is an example of the image analyzer of the present invention. The viewing state analyzer 116 receives the results of various pieces of image processing performed by the image processor 114 as well as the detection result and distance information detected by the sensor 106, and analyzes whether the person who views the image displayed by the image displaying apparatus 100 is a child or an adult and the viewing state and viewing position of the person who views the image using the results of various pieces of image processing performed by the image processor 114 as well as the detection result and distance information detected by the sensor 106. The viewing state analyzer 116 analyzes whether the person who views the image displayed by image displaying apparatus 100 is a child or an adult and the viewing state and viewing position of the person who views the image, which allows the image displaying apparatus 100 to lower the luminance of the display panel 102, control display contents of the display panel 102, and controls sound contents based on whether the person who views the image displaying apparatus 100 is a child or an adult and the viewing position of the person. The analysis result of the analysis processing performed by the viewing state analyzer 116 is transmitted to the viewing state recorder 118 and the system optimization processor 120.
The viewing state analyzer 116 can detect the dynamic body from the detection result and distance information detected by the sensor 106. Alternatively, the dynamic body may be set out of the detection target when the distance between the sensor 106 and the dynamic body is longer than a predetermined distance.
The viewing state recorder 118 records the analysis result that is obtained through the analysis processing of the viewing state analyzer 116. The analysis result of the viewing state analyzer 116, which has been recorded in the viewing state recorder 118, is used in system optimization processing performed by the system optimization processor 120. The analysis result of the viewing state analyzer 116, which has been recorded in the viewing state recorder 118, may be transmitted to an external information-gathering server 200.
The system optimization processor 120 is an example of the image analyzer of the present invention. Using the analysis result that is obtained through the analysis processing performed by the viewing state analyzer 116, the system optimization processor 120 calculates system control information in order to perform the system optimization processing to each unit of the image displaying apparatus 100. Examples of the system optimization processing performed to each unit of the image displaying apparatus 100 include luminance control of the display panel 102, control of the display contents of the display panel 102, control of the sound contents output from the speaker 108, and volume control of the sound.
The image displaying apparatus 100 can perform child approach preventing processing based on the system control information calculated by the system optimization processor 120. The system control information calculated by the system optimization processor 120 is transmitted to the system controller 122.
The system controller 122 is an example of the display controller and the sound controller of the present invention, and performs system optimization processing to each unit of the image displaying apparatus 100 based on the system control information calculated by the system optimization processor 120. Specifically, based on the system control information calculated by the system optimization processor 120, the system controller 122 performs the luminance control of the display panel 102, the control of the display contents of the display panel 102, the control of the sound contents output from the speaker 108, the volume control of the sound, and the like.
The configuration of the image displaying apparatus 100 according to the embodiment of the present invention has been described above with reference to
[1-2. Configuration of Controller]
As illustrated in
The user direction/distance calculator 132 receives the results of various pieces of image processing performed by the image processor 114 as well as pieces of optical information such as the angle of view and the resolution of the imaging unit 104, and calculates a relative position (direction [φ1, θ1] and distance d1) of the user with respect to an optical axis of the imaging unit 104 using the results of various pieces of image processing performed by the image processor 114 as well as the pieces of optical information on the imaging unit 104.
As to the direction [φ1, θ1], the horizontal direction: φ1=φ0*a1 and the vertical direction: θ1=θ0*b1 are calculated from the face detection position [a1, b1] normalized by a captured image size [xmax, ymax] and the angle of view [φ0, θ0] of the imaging unit 104. As to the distance d1, the distance: d1=d0*(w0/w1) is calculated from the reference face size [w0, h0] at the reference distance d0.
The user attribute calculator 134 receives the results of various pieces of image processing performed by the image processor 114 and pieces of attribute information, such as an age of the user, which are obtained by the image processor 114. The user attribute calculator 134 determines whether the user is a child or an adult using the results of various pieces of image processing performed by the image processor 114 and the pieces of attribute information obtained by the image processor 114.
At this point, the image processor 114 transmits the captured image and face detection information (such as the face detection position [a1, b1], the face size [w1, h1], and other pieces of attribute information such as the age and the sex) on each user who utilizes the image displaying apparatus 100 in the captured image to the user direction/distance calculator 132 and user attribute calculator 134 of the viewing state analyzer 116. In the viewing state analyzer 116, the user direction/distance calculator 132 performs processing of analyzing the viewing position of the human who views the image displaying apparatus 100 using the pieces of information transmitted from the image processor 114. The user attribute calculator 134 performs processing of analyzing whether the human who views the image displaying apparatus 100 is a child or an adult using the pieces of information transmitted from the image processor 114.
When a child is located in the improper viewing position of the image displaying apparatus 100, the system optimization processor 120 calculates system control information for processing (child guide implementing processing) of guiding the child to the proper viewing position of the image displaying apparatus 100 using the results of the pieces of analysis processing that are performed by the user direction/distance calculator 132 and the user attribute calculator 134. Examples of the processing of being capable of guiding the child to the proper viewing position of the image displaying apparatus 100 include processing of displaying a guide message at an optimum viewing position on the display panel 102 while lowering the luminance of the display panel 102 and processing of displaying a graph of an approach distance and a warning message on the display panel 102. Examples of the processing of being capable of guiding the child to the proper viewing position of the image displaying apparatus 100 also include processing of displaying the guide message at the optimum viewing position on the display panel 102 while darkening the display panel 102 and processing of producing the warning sound from the speaker 108. The system control information for the child guide implementing processing, which has been calculated by the system optimization processor 120, is transmitted to the system controller 122 and used in child approach preventing processing.
The structure of the controller 110 included in the image displaying apparatus 100 according to the embodiment of the present invention has been described above with reference to
[1-3. Child Approach Preventing Processing]
Referring to
Then, using the pieces of information transmitted from the image processor 114, the viewing state analyzer 116 of the image displaying apparatus 100 performs the processing of analyzing the viewing position of the human who views the image displaying apparatus 100 and the processing of analyzing whether the human is a child or an adult. Using the result of the analysis processing performed by the viewing state analyzer 116, the system optimization processor 120 of the image displaying apparatus 100 determines whether the child is located in the improper viewing position of the image displaying apparatus 100, namely, whether the child approaches the image displaying apparatus 100. Specifically, the system optimization processor 120 determines whether the child's face is detected, and determines whether the size of the child's face, namely, a face size is equal to or more than a predetermined value (Step S604). It is assumed that a reference face size of the child in the reference position is previously registered in the image displaying apparatus 100. The predetermined value is a face size of the child in the improper viewing position of the image displaying apparatus 100.
As a result of the determination in Step S604, when the child's face is detected and when the face size of the child is equal to or more than the predetermined value (YES in Step S604), the system optimization processor 120 determines whether the number of child approach times detected in a past predetermined time is equal to or more than a predetermined value (Step S606). When the number of child approach times detected in a past predetermined time is lower than the predetermined value (NO in Step S606), the flow returns to the processing in Step S602. The predetermined time can properly be set. The predetermined value can properly be changed according to the set predetermined time.
As a result of the determination in Step S606, when the number of child approach times detected in the past predetermined time is equal to or more than the predetermined value (YES in Step S606), the system optimization processor 120 determines that the child approaches the image displaying apparatus 100 (Step S608).
Then the system optimization processor 120 calculates the system control information for the child guide implementing processing described above, and transmits the calculated system control information to the system controller 122. Based on the system control information received from the system optimization processor 120, the system controller 122 performs the luminance control of the display panel 102, the control of the display contents of the display panel 102, the control of the sound contents output from the speaker 108, the volume control of the sound, and the like (Step S610). As illustrated in
As a result of the determination in Step S604, when the child's face is detected and when the face size of the child is smaller than the predetermined value (NO in Step S604), the system optimization processor 120 determines whether the child having the face size smaller than the predetermined value or any face except a child's face has been detected (Step S612). When the child having the face size smaller than the predetermined value or the face except a child's face has not been detected (NO in Step S612), the flow returns to the processing in Step S602.
As a result of the determination in Step S612, when the child having the face size smaller than the predetermined value or the face except a child's face is detected (YES in Step S612), the system optimization processor 120 determines whether the number of child approach times detected in the past predetermined time is equal to or lower than the predetermined value (Step S614). When the number of child approach times detected in the past predetermined time is more than the predetermined value (NO in Step S614), the flow returns to the processing in Step S602.
As a result of the determination in Step S614, when the number of child approach times detected in the past predetermined time is equal to or lower than the predetermined value (YES in Step S614), the system optimization processor 120 determines that the child does not approach the image displaying apparatus 100 (Step S616).
Subsequently, when the system controller 122 performs the luminance control of the display panel 102, the control of the display contents of the display panel 102, the control of the sound contents output from the speaker, the volume control of the sound, and the like based on the system control information for the child guide implementing processing, the system optimization processor 120 calculates the system control information for returning to the normal display processing, and transmits the calculated system control information to the system controller 122. Based on the system control information received from the system optimization processor 120, the system controller 122 performs processing of returning the luminance and display contents of the display panel 102 and the sound contents output from the speaker 108 to the normal state (Step S618). Then, the flow returns to the processing in Step S602.
According to the child approach preventing processing in
In the embodiment, as illustrated in
In the embodiment, in calculating the viewing position of the user, when the user who utilizes the image displaying apparatus 100, for example, a family of the installation place of the image displaying apparatus 100 is previously registered in the image displaying apparatus 100, the face size of each user may be registered in the data table. Therefore, the reference face size can be changed for each user. A method for registering the user face size of each user can be implemented in a manner such that the image of the user is captured along with the distance information in conjunction with another distance sensor (not illustrated), such that the image of the user is captured after the user is guided to a given distance, or such that the image of the user is captured at the same distance as a scale that serves as a reference.
As illustrated in
In the embodiment, the determination of the child approach is made when the number of child approach times detected in the past predetermined time is equal to or more than the predetermined value. However, as illustrated in
In the embodiment, when the determination of whether the user is a child or an adult is made using the attribute information on the age of the user and the like, a rate of a false determination is reduced in the child determination by a combination of a determination of whether the user is a child and a determination of whether the user is an adult. A method for reducing the rate of the false determination in the child determination will specifically be described below.
As illustrated in
Case A: TruePositive (the determination that the user is a child has been correctly made)
Case B: FalseNegative (the determination that the user is a child has not been made when such determination should be made)
Case C: FalsePositive (the determination that the user is a child has been made when such determination should not be made)
Case D: TrueNegative (the determination that the user is not a child has been correctly made)
In the embodiment, in order that the determination that the child approaches the image displaying apparatus 100 is correctly made, the case C, in which the determination that the user is a child is made when such determination should be made, needs to be eliminated in the determination of whether the user is a child. For example, it is assumed that 85%@FPR10% is the determination precision of the determination of whether the user is a child. It is herein assumed that FPR (False Positive Rate)=Case C/(Case C+Case D), and Precision=Case A/(Case A+Case C).
In the case where the determination of whether a use is a child is made on 100 persons who are actually children and 150 persons who are actually adults at the above determination precision, when it is determined that 100 persons are children, 85 persons fall under the Case A, 15 persons fall under the Case B, 15 persons fall under the Case C, and 135 persons fall under the Case D. Accordingly, 85 persons are correctly determined to be children among the 100 persons who are actually children, and 15 persons are falsely determined to be children among the 150 persons who are actually adults.
In the embodiment, as illustrated in
The above series of pieces of processing can be performed by either hardware or software. In the case where the series of pieces of processing can be performed by the software, a program constituting the software is installed from a program recording medium into a computer incorporated in the dedicated hardware. Alternatively the program is installed from a general-purpose personal computer that can execute various functions by installing various programs.
The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, whilst the present invention is not limited to the above examples, of course. A person skilled in the art may find various alternations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2009-210988 | Sep 2009 | JP | national |
This is a continuation of U.S. application Ser. No. 13/393,844, filed on May 10, 2012, now U.S. Pat. No. 8,913,007 B2, which is a U.S. national phase application of International Application PCT/JP2010/062310, filed on Jul. 22, 2010, and claims the benefit of priority from Japanese Application No. 2009-210988, filed on Sep. 11, 2009.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6076928 | Fateh et al. | Jun 2000 | A |
| 8248512 | Kondo et al. | Aug 2012 | B2 |
| 8614674 | Binder | Dec 2013 | B2 |
| 20010012001 | Rekimoto et al. | Aug 2001 | A1 |
| 20010015792 | Fateh et al. | Aug 2001 | A1 |
| 20050216867 | Marvit et al. | Sep 2005 | A1 |
| 20050264758 | Wakamori | Dec 2005 | A1 |
| 20060050267 | Murakami et al. | Mar 2006 | A1 |
| 20060164682 | Lev | Jul 2006 | A1 |
| 20060242662 | Masaki et al. | Oct 2006 | A1 |
| 20060290804 | Mino et al. | Dec 2006 | A1 |
| 20060290890 | Saito et al. | Dec 2006 | A1 |
| 20070126884 | Xu et al. | Jun 2007 | A1 |
| 20080118152 | Thorn et al. | May 2008 | A1 |
| 20080316372 | Xu et al. | Dec 2008 | A1 |
| 20090058845 | Fukuda | Mar 2009 | A1 |
| 20090097716 | Wang et al. | Apr 2009 | A1 |
| 20090295835 | Husoy | Dec 2009 | A1 |
| 20110254691 | Ooi et al. | Oct 2011 | A1 |
| 20120092466 | Choi | Apr 2012 | A1 |
| 20120229610 | Fukushima et al. | Sep 2012 | A1 |
| 20120293405 | Iida et al. | Nov 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 101409784 | Apr 2009 | CN |
| 0656730 | Jun 1995 | EP |
| 07-303195 | Nov 1995 | JP |
| 08-105851 | Apr 1996 | JP |
| EP 0769881 | Apr 1997 | JP |
| 2000 152109 | May 2000 | JP |
| 3075971 | Dec 2000 | JP |
| 2001 16514 | Jan 2001 | JP |
| 2005 44330 | Feb 2005 | JP |
| 2007 65766 | Mar 2007 | JP |
| 2007 81677 | Mar 2007 | JP |
| 2007 158787 | Jun 2007 | JP |
| 2007236668 | Sep 2007 | JP |
| 2008-197447 | Aug 2008 | JP |
| WO 2009067676 | May 2009 | WO |
| 2010 021373 | Feb 2010 | WO |
| Entry |
|---|
| International Search Report Issued Oct. 12, 2010 in PCT/JP10/62310 Filed Jul. 22, 2010. |
| Extended European Search Report issued Aug. 8, 2013 in Patent Application No. 10815221.6. |
| Office Action issued Sep. 3, 2013 in Japanese Patent Application No. 2009-210988. |
| Combined Office Action and Search Report issued Mar. 5, 2014 in Chinese Patent Application No. 201080047771.6 with English language translation. |
| Japanese Office Action issued on Sep. 9, 2014 in Japanese Application No. 2013-238462, (7 pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20150054738 A1 | Feb 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13393844 | US | |
| Child | 14532770 | US |
