Patent Application
20110234784
The present invention relates to a display device that can present a viewing state of video displayed on the display device in terms of power saving.
In recent years, power consumption has been reduced in various electrical apparatuses in response to growing environmental awareness. For example, in addition to power savings in the components of electrical apparatuses, many electrical apparatuses have the function of entering an operation mode with reduced power consumption, that is, a power saving state or a standby state when a user does not issue any commands for a certain period of time.
Under these circumstances, Japanese Patent No. 3862027 (hereinafter, will be called a patent document) discloses a broadcast signal receiving system (television) for efficiently controlling power savings.
A control unit 18 includes a CPU, ROM, and RAM, which are not shown, and controls the tuner unit 11, the video decoder 12, the audio decoder 13, and the OSD circuit 14.
A key input unit 21 notifies the control unit 18 of the output of operation information about the operations of buttons on a device of the broadcast signal receiving system described in the patent document and a key operation signal received by a receiving unit 22 from a remote control 23. A camera unit 25 is provided on the device of the broadcast signal receiving system described in the patent document, and the camera unit 25 is installed so as to image a viewer. The signal of the image captured by the camera unit 25 is inputted to a personal identification unit 26. The personal identification unit 26 detects a face image from the captured image, performs personal identification for identifying a viewer based on the detected face image, and decides whether the eyes of the viewer are opened or closed. The personal identification unit 26 notifies the control unit 18 of the input of a signal indicating an identification result and an open/closed state of the eyes.
A clock unit 28 notifies the control unit 18 of the current time. A timer circuit unit 29 is operated by a command of the control unit 18 and measures a predetermined time.
A recording/reproducing apparatus 40 records a broadcast signal on a recording medium, e.g., a hard disk drive and a DVD and reproduces the broadcast signal recorded on the recording medium. The recording and reproduction of the recording/reproducing apparatus 40 are controlled by the control unit 18.
In the broadcast signal receiving system configured thus, when the remote control 23 is operated to turn on a television, which is the broadcast signal receiving system, the broadcast signal received by the tuner unit 11 passes through the video decoder 12, the audio decoder 13, and the OSD circuit 14 under the control of the control unit 18 and a television program specified by the remote control 23 is displayed on the television monitor 15. The control unit 18 controls the recording/reproducing apparatus 40 to start time-shifting recording, allowing later viewing of video recorded while a viewer is away from the television monitor 15, which will be described later. Further, the control unit 18 instructs the camera unit 25 to start imaging. An image captured by the camera unit 25 undergoes image recognition in the personal identification unit 26. In the personal identification unit 26, the presence or absence of a face is detected, personal identification is performed to identify the face, and an open/closed state of eyes is obtained. The face detection result, the identification result, and the open/closed state of the eyes from the personal identification unit 26 are notified to the control unit 18.
The control unit 18 monitors the information sent from the personal identification unit 26. When the image captured by the camera unit 25 does not contain any face images, a time is obtained from the clock unit 28 and measurement of an elapsed time is started by the timer circuit unit 29. When no face image is detected for a certain period of time, the control unit 18 decides that no viewer is present in front of the television, that is, no one is watching the television, and the control unit 18 turns off the television monitor 15 to enter a power saving mode for reducing power consumption. After that, when a face image is detected again, the control unit 18 turns on the television monitor 15 to enter a normal operation mode. If the detected face image is identical to a previously detected face image, a program is reproduced by the recording/reproducing apparatus 40 from a time when the face image was not detected.
In the case where the face image is detected again within the certain period of time, the control unit 18 causes the timer circuit unit 29 to stop the measurement of an elapsed time.
As previously mentioned, the broadcast signal receiving system of the patent document determines the presence or absence of a viewer in front of the television monitor 15 by using the camera unit 25. When the absence of a viewer is determined, the broadcast signal receiving system enters the power saving mode after the certain period of time, reducing excessive power consumption.
In the broadcast signal receiving system of the related art, however, power still is consumed by the operations of the control unit 18 and the recording/reproducing apparatus 40 even when the television monitor 15 enters a power saving state. In order to minimize the power consumption, it is preferable that a user frequently turns off the broadcast signal receiving system (television). Thus it is considered that the user may be encouraged frequently to turn on and off the broadcast signal receiving system. However, such a proposal has not been made.
Further, it is assumed that the broadcast signal receiving system of the patent document is a television at home, but such a broadcast signal receiving system may be installed in a different environment. For example, in recent years, display devices having large screens are installed in public places such as a station square and gathering places to display advertising video. However, on such a display device, it is not possible to confirm whether or not an audience views (watches) the display device, whether or not the display device is efficiently viewed, or whether or not effective advertisements are provided.
The present invention has been devised to solve the problem. An object of the present invention is to provide a display device that makes it possible to confirm whether the display device is efficiently used (viewed) or not.
In order to solve the problem, a display device or a display method according to the present invention monitors a user of the display device and records a time period during which the display device or a display system is operated but the display device is not viewed.
The present invention makes it possible to record whether the display device is used efficiently or not. Thus it is possible to provide a display device that makes it possible to confirm whether the display device is used efficiently (viewed) or not.
The following will describe embodiments of the present invention in accordance with the accompanying drawings.
As shown in
The image recognition unit 170 acts as a face detector for detecting a face image from an image of the image data generated by the camera unit 160, which is an imaging device, a personal identification device for identifying a person (identifying a viewer), and a decision device for deciding whether or not a viewer views video displayed on the video signal display unit 110. The control unit 138 measures, for example, a non-viewing time while continuously receiving times from a clock unit 140. The control unit 138 also acts as an extracting device for extracting the attributes (including personal information) of viewers and a charge converter for converting non-viewing power consumption into a charge.
Referring to
In
The audio signal processing unit 120 decodes a signal stream of audio signals separated by the signal separating unit 105, and outputs a digital audio signal. The audio signal output unit 130 is a speaker that converts the digital audio signal from the audio signal processing unit 120 into an analog audio signal and outputs the signal. The electric program guide (EPG) data extracted by the signal separating unit 105 is stored in the EPG data storage unit 135 and is referred from the control unit 138 as needed. The clock unit 140 notifies the control unit 138 of the current time. The camera unit 160 is installed so as to image a region in front of the video signal display unit 110, that is, a viewer of the digital television. An image captured by the camera unit 160 is inputted to the image recognition unit 170. The image recognition unit 170 detects a face image from the captured image and performs personal identification to identify the detected face image. The face detection result and the personal identification result are notified to the control unit 138. The control unit 138 includes a CPU (not shown) and a memory including ROM and RAM.
Various techniques are known as face detection techniques. For example, P. Viola and M. Jones announced a face image learning technique using boosting. Personal identification is also known processing in which feature quantities such as a distance between eyes and a nose position are registered beforehand from a viewer's face image captured by the camera unit 160, the feature quantities are compared with the feature quantities of a face image detected from the current image, and it is decided whether or not the detected face image matches the registered face image.
The power consumption measuring unit 175 is connected to a power supply (not shown) for supplying power to the digital television, and measures the power consumption of the digital television of
The viewing information managing unit 180 collectively stores the image recognition result from the image recognition unit 170, the EPG data information from the EPG data storage unit 135, and power consumption information measured for the digital television by the power consumption measuring unit 175. The viewing information managing unit 180 also includes a nonvolatile memory, so that the stored information is not deleted even if the digital television of the present embodiment is turned off.
The remote control 185 has buttons for operating the digital television of the present embodiment. The input of the remote control 185 is transmitted to the receiving unit 190 using infrared rays or the like. In the receiving unit 190, a received signal is converted to an operation command for the digital television and is sent to the control unit 138.
The following will describe the operations of the digital television configured thus.
First, when the digital television is turned on by the remote control 185, broadcast waves received by the antenna 101 are inputted to the signal separating unit 105 through the tuner unit 102, and the EPG data and the video signal and the audio signal of a program, which corresponds to a channel number inputted from the remote control 185, are separated from the broadcast multiplexed signal. The video signal is decoded through the video signal processing unit 106. After OSD display is superimposed on the video signal by the OSD unit 107, the signal is displayed on the video signal display unit 110; meanwhile, the audio signal separated from the broadcast multiplexed signal by the signal separating unit 105 is decoded in the audio signal processing unit 120 and is outputted as sound from the audio signal output unit 130. The EPG data is stored in the EPG data storage unit 135 from the signal separating unit 105. The series of operations is similar to that of a typical digital television. In the present embodiment, the camera unit 160 captures an image in front of the digital television, based on which it is determined whether or not a viewer is viewing a program displayed on the digital television, and the determination result is stored in the viewing information managing unit 180 with the power consumption information of the power consumption measuring unit 175.
The flowchart of
In step S310, when a face is detected in the face detection performed on the image of the camera unit 160 by the image recognition unit 170, the process advances from step S310 to step S324. In step S324, the measurement of power consumption is completed.
Step S324 for the completion of the measurement of power consumption is composed of multiple steps. Specifically, first in step S325, it is decided whether power consumption is being measured or not. When it is decided that power consumption is not being measured, the process exits. When it is decided that power consumption is being measured, the process advances to step S335. In step S335, the control unit 138 obtains the current time from the clock unit 140. Next, in step S340, the control unit 138 instructs the power consumption measuring unit 175 to complete the measurement of power consumption. The power consumption measuring unit 175 completes the measurement of power consumption of the digital television in response to the instruction of the control unit 138, and notifies the control unit 138 of the measurement result of power consumption. After that, in step S345, the viewing information of the viewing information managing unit 180 is updated. To be specific, another record is created by the control unit 138 based on EPG data obtained in step S300 so as to contain a broadcast date, a broadcast day of the week, a broadcast time, a program title, and a genre, a non-viewing time that is a difference between two pieces of time information obtained in steps S315 and S335, and power consumption obtained in step S340, and the record is notified to the viewing information managing unit 180. The viewing information managing unit 180 searches for records matching the broadcast date, the broadcast day of the week, the broadcast time, the program title, and the genre. If there is a match, the non-viewing time of the record notified from the control unit 138 is added to the non-viewing time of the retrieved record and power consumption to be reduced in the record notified from the control unit 138 is added to power consumption to be reduced in the retrieved record. If there is no match, the record notified from the control unit 138 is added to the viewing information of the viewing information managing unit 180.
In step S320, it is decided whether the running of the current program should be terminated or not. To be specific, when the digital television is turned off, when one channel is tuned to another by the remote control 185, or when the broadcasting of the current program ends and the subsequent program starts, the process advances to step S350 along YES; otherwise the process returns to step S302 and an image is captured by the camera unit 160 again.
In step S350, the measurement of power consumption is completed as in step S324 and the explanation thereof is omitted.
The storage of the viewing information was described according to the flow chart of
The processing of the flowchart of
A monthly display button 450 is a button for switching a display period to a month. In
The control unit 138 calculates numeric values required for the information display of
The control unit 138 creates the values of the total display 430 and the graph display region 440 from the viewing information stored in the viewing information managing unit 180. Further, the control unit 138 issues various screen display commands to the OSD unit 107 and controls the OSD unit 107 so as to provide display on the video signal display unit 110 as shown in
In
In
When the daily display button 455 is selected and pressed (not shown) by the remote control 185 in
In
The control unit 138 calculates numeric values required for the information display of
In
In
In
In
As a modification of the present embodiment, for example, a read-only memory (ROM), which is provided in the control unit 138 but is not shown specifically, may contain a conversion rate for power consumption and a charge amount and displayed power consumption to be reduced may be converted to a charge amount. In the screen displays of
By using the digital television (display device) configured thus and the display method, it is possible to monitor whether or not a user is viewing the display contents of the display device, and present the display time of the display device or the power consumption of the display time when the user is not viewing the display contents of the display device. In other words, it is possible to notify the user of whether the digital television is used efficiently or not, e.g., the ratio of the non-viewing time, genres and viewers with a long non-viewing time, and the ratio of power consumption, thus encouraging the user to frequently turn on and off the display device and raising awareness of power saving. Further, it is possible to turn on the display device of the digital television only when necessary, achieving efficient use of the display device and further reducing excessive energy consumption.
The preferred embodiment of the present invention has been discussed. Needless to say, the present invention is not limited to the present embodiment and various modifications can be made without departing from the scope of the invention. Particularly, various modifications can be made on the display format of the graph displayed on the video signal display unit 110, a user interface including the layout and types of the buttons, and the viewing information stored in the viewing information managing unit 180. Further, information display methods other than graphs can be used without causing any problems. For example, a non-viewing time or power consumption to be reduced may be superimposed on a channel guide, in which EPG data is displayed, to easily recognize a program being viewed while a viewer does other things.
Various known methods are available for face detection and personal identification. However, the present invention does not depend on the type of technique.
In the present embodiment, the camera unit 160 captures an image to decide whether or not a viewer is watching the digital television. Further, an open/closed state of eyes may be detected by a high-resolution camera. Thus even when the face of a viewer is detected, it is determined that the viewer with closed eyes has fallen asleep and has not watched the digital television, so that the detection time may be regarded as a non-viewing time. Moreover, the positions of pupils may be detected in addition to the face to detect the line of viewer's sight, so that it is possible to decide whether or not the viewer is watching the digital television.
In the present embodiment, the single camera unit 160 is provided. Multiple cameras may be used to increase the accuracy of face detection. Further, the camera of the present invention is not limited to a camera capable of imaging visible radiation. A camera capable of imaging infrared radiation may be used to increase the accuracy of face detection at a low light intensity.
In the present embodiment, it is decided that a viewer is present when a face image is detected. The present invention is not limited to the decision method. For example, the presence or absence of a person is first determined from captured images (including an image captured by a camera capable of imaging infrared radiation) or an infrared sensor provided separately, and then information about the presence or absence of a person is obtained. In this state, when the presence of a person is decided, the viewing state of the person is detected based on face detection, an open/closed state of eyes, the line of sight, and so on. The presence of a person may be decided at the detection of a face, opened eyes, or a line of sight.
In the present embodiment, the viewing information is created only for a currently broadcast program. Viewing information may be created for programs recorded by a recorder connected to the digital television or recorded on a hard disk drive (HDD) contained in the main unit of the digital television. When recording is reserved in the recorder and the main unit of the digital television, EPG information on programs to be reserved is stored, so that as in the present embodiment, viewing information can be created for viewing recorded programs.
In the present embodiment, the viewing information is created only for the single digital television provided for multiple family members. The number of display devices is not particularly limited and multiple display devices may be connected on a LAN to transmit and receive information. Viewing information may be collected on the display devices and the viewing information collectively displayed on the display devices does not cause any problems.
In the present embodiment, power measurement is started when no faces are detected by imaging of the camera unit 160. Power measurement may be started after no faces have been detected for a certain period of time. After no faces have been detected for the certain period of time, power measurement may be started when the digital television having the function of entering a power saving mode enters the power saving mode.
In the present embodiment, the display device is a digital television. Needless to say, the present invention is applicable to various other display devices such as a PC monitor.
In the first embodiment, the display device is a digital television provided for multiple family members. The present invention is not limited to this installation environment. In a second embodiment, the installation environment is different from that of the first embodiment. The second embodiment describes an example in which a display device having a large screen (also called a large display device) is installed in public places such as a station square and gathering places. The large display device sequentially displays (outputs), e.g., advertising video with sound.
As shown in
The image recognition unit 170 acts as a face detector for detecting a face image from an image of the image data generated by the imaging device, a personal identification device for identifying a person (identifying a viewer), and a decision device for deciding whether or not a viewer is viewing video displayed on the video signal display unit 110. The control unit 138 also acts as an extracting device for extracting the attributes (sex and age) of a viewer and measures a non-viewing time for each of the attributes while continuously receiving times from a clock unit 140.
Referring to
In
The audio signal processing unit 120 converts the signal outputted from the control unit 138 through the communication unit 220, into a digital audio signal and outputs the signal. The audio signal output unit 130 is a speaker for converting the digital audio signal inputted from the audio signal processing unit 120 into an analog audio signal and outputting the signal. Moreover, information about the inputted advertising video is stored in a storage unit (recording unit) (not shown) provided in the control unit 138 and is referred in the storage unit. The clock unit 140 notifies the control unit 138 of the current time. The camera unit 160 is installed so as to image a region in front of the video signal display unit 110 and images a person in the region in front of the video signal display unit 110. An image captured by the camera unit 160 is inputted to the image recognition unit 170. In the image recognition unit 170, a face image is detected from the captured image and the sex and age of the person (regarded as a viewer when a face image is detected) are extracted as attributes from the detected face image. The control unit 138 includes a CPU and memory such as ROM and RAM, which are not shown. The control unit 138 can send the information obtained through the communication unit 220, to advertising agencies and clients who are contracted with the installed large display device.
The following will describe the operations of the large display device configured thus.
First, the control unit 138 obtains advertising video, audio signal information corresponding to the advertising video, and advertising type information associated with the video and information, through an external network line such as the internet 210 and the communication unit 220. The advertising video is displayed on the video signal display unit 110 and sound for the video is outputted from the audio signal output unit 130. The series of operations is similar to that of a typical digital television. Further, in the present embodiment, the camera unit 160 images a location in front of the large display device, viewers are detected (face image detection), the attributes (sex and age) of the viewers are determined, and it is decided whether or not the viewers are viewing video displayed on the large display device. The decision result is stored in the viewing information managing unit 180.
In other words, when a face image can be detected from an image captured by the camera unit 160, the attributes of a viewer can be recognized from the face image, and the viewer (e.g., “underage male”) can be detected for the attributes, it is decided that the viewer with the attributes is viewing the advertising video. When the person with the attributes (face image) cannot be detected, it is decided that the person is not viewing the advertising video, and the measurement of a non-viewing time is started. When the person with the same attributes (e.g., “underage male”) is detected again, it is decided that the person is viewing the video, and the measurement of the non-viewing time is completed. In this way, the non-viewing time is measured by measuring a time during which a person with certain attributes is not detected.
In the present embodiment, the viewing information is provided in a table as shown in
Referring to the flowchart of
First, in step S501, the camera unit 160 images a state of a space in front of the large display device (video signal display unit 110). Next, in step S502, face detection is performed on an image of the camera unit 160 by the image recognition unit 170. When a face image is detected, the image recognition unit 170 acting as a detector detects attributes (age and sex). In step S503 following step S502, the image recognition unit 170 decides whether specified attributes have been detected or not. As shown in
In step S503, when the attribute of “underage male” (an image with the attribute) is not detected (that is, when an “underage male” viewer is not viewing advertising video), the process advances to step S504 and it is confirmed whether the start time of a non-viewing time has been obtained or not. When the start time of a non-viewing time has not been obtained, the process advances to step S505 to obtain the current time and set the start time of a non-viewing time. In step S504, when the start time of a non-viewing time has been obtained, the process returns to step S501.
In step S503, when the attribute of “underage male” (an image with the attribute) is detected (that is, when an “underage male” viewer is viewing advertising video), the process advances to step S506 and it is confirmed whether the start time of a non-viewing time has been obtained or not.
In step S506, when the start time of a non-viewing time has been obtained, it is confirmed that an “underage male” viewer, who has not viewed advertising video, has started viewing the advertising video. In other words, the non-viewing time has been ended. In this case, the process advances to step S507 and the current time is obtained to set the end time of the non-viewing time. By updating the viewing information in step S508, the start time and the end time of the non-viewing time are set (this state is shown in
In step S506, when the start time of the non-viewing time has not been obtained, it is confirmed that an “underage male” viewer keeps viewing advertising video. Thus the process returns to step S501. Until no “underage male” viewer is detected (until no “underage male” viewer is viewing advertising video), loop processing of step S501→step S502→step S503→step S506→step S501 is continued.
As has been discussed, in the present embodiment, a time when a viewer with specified attributes cannot be detected is extracted as the start time of a non-viewing time and a time when the specified attributes can be detected is extracted as the end time of the non-viewing time, so that the non-viewing time of the person (viewer) with the specified attributes can be measured. Consequently, a client who has requested the display of advertising video can confirm how long a person with the attributes has viewed the advertising video and whether the advertising video has been viewed effectively or not, and confirm the attributes of viewers who have viewed the advertising video, thereby confirming time periods during which the advertising video can be viewed efficiently. Thus it is possible to confirm whether or not the display device of the present embodiment has been used efficiently (viewed), confirm the effectiveness of the advertising video, confirm a difference in the effectiveness of the advertising video between the attributes of viewers (age and sex in the present embodiment), and more efficiently use the advertising video.
In the present embodiment, the advertising video is inputted in communication with external network lines such as the internet 210. The present invention is not limited to an external network line and the advertising video may be inputted from recording media such as a disk on which advertising video has been recorded. In the present embodiment, the attributes of age are “underage”, “adult”, and “aged”. The present invention is not limited to these attributes and the attributes may be “teens”, “20s”, “30s”, . . . .
Further, processing blocks in the block diagrams of
In the present embodiment, it is assumed that viewers watch the digital television that outputs not only images but also sound. The present invention is not limited to such a digital television. In other words, the present invention is similarly applicable to viewers of display devices displaying only video.
The display device or the display method according to the present invention can be used as a display device or a display method that allows a user to view the display contents of the display device.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-066038 | Mar 2010 | JP | national |
| 2010-283908 | Dec 2010 | JP | national |
