Patent Grant
9883210
This application is a U.S. National Stage Application under 35 U.S.C. §371 of PCT Application No. PCT/KR2014/002586, filed Mar. 27, 2014, which claims priority to Korean Patent Application No. 10-2014-0000369, filed Jan. 2, 2014, whose entire disclosures are hereby incorporated by reference.
The present invention relates to a display device and a method for operating the display device. More particularly, the present invention relates to a display device and a method for operating the display device, capable of providing information based on a user's watching status.
Recently, a digital TV service using a wired or wireless communication network is becoming common. The digital TV service may provide a variety of services that could not be provided in the existing analog broadcast service.
For example, in case of IPTV (Internet Protocol Television) service which is a kind of a digital TV service, it provides an interactivity that enables a user to actively select kinds and watching time of watching program. The IPTV service may provide a variety of additional services based on such an interactivity, including Internet search, home shopping, online game, etc.
Conventionally, however, if a user is in a status that he or she is not able to watch TV visually, there is a limitation to provide program information and subtitle information of a broadcast program that is currently played back, and event information generating in a display device.
The objective of the present invention is to provide a display device and a method for operating the display device, capable of providing a user with a variety of information even if the user is in a status that he or she is not able to watch the display device visually.
In accordance with an embodiment of the present invention, there is provided a method for operating a display device, comprising playing back a broadcast program; obtaining a user's watching status while the broadcast program is played back; and providing information of the display device if the obtained user's watching status is obtained as a non-visual watching status in which the user does not watch a display unit of the display device.
In accordance with another embodiment of the present invention, there is provided a display device, comprising a tuner configured to receive a broadcast signal; a display unit configured to display an image of the broadcast program included in the broadcast signal; an audio output unit configured to output a sound; and a control unit configured to provide information of the display device if a user's watching status is obtained as a non-visual watching status that a user does not watch the display unit of the display device.
According to various embodiments of the present invention, a user may be provided with a variety of information even if the user is not able to watch a display device visually.
Hereinafter, embodiments of the present invention are described in detail with reference to accompanying drawings. Suffixes in the components, “modules” and “units”, used in the description below are given or mixed only considering the easiness of preparing the description and do not have separate meanings or roles in themselves.
The display system according to an embodiment of the present invention comprises a display device 100, a remote control device 200 and a mobile terminal 300.
The display device 100 is an intelligent device that has a computer support function additionally to a broadcast receiving function, for example. So, it additionally have an Internet function and the like while faithfully performing a broadcast receiving function, which may have more convenient interfaces such as a handwriting input device, a touch screen or a space remote controller. Also, if connected to an Internet and computer with the support of a wired or wireless remote Internet function, the display unit 100 may perform such functions as E-mail, Web browsing, banking or game. A standardized general-purpose OS may be used for such a variety of functions.
Accordingly, since the display device 100 described in the present invention may freely add and remove a variety of applications on a general-purpose OS canal, for example, a variety of user friendly functions may be performed. In more detail, the display device 100 may be a network TV, a HBBTV, a smart TV, an LED TV and an OLDED TV, for example, which may be applied to a smart phone according to need.
The remote control device 200 may transmit a control command to control an operation of the display device 100, thereby controlling the operation of the display device 100. The explanation for that is described in detail below.
The mobile terminal 300 may transmit and receive information to and from the display device 100 through an NFC (Near Field Communication) interface.
Referring to
The external device interface unit 135 may receive an application or an application list existing in an adjacent external device, and transfer it to the control unit 170 or the storage unit 140.
The network interface unit 133 may provide an interface to connect the display device 100 to a wired/wireless network including an Internet network. The network interface unit 133 may transmit or receive data to or from other users or other electronic equipment through a connected network or another network linked to the connected network.
Also, some content data stored in the display device 100 may be transmitted to a user or electronic equipment selected among other users or other electronic equipment that are registered in the display device 100 in advance.
The network interface unit 133 may be connected to a predetermined web page through a connected network or another network linked to the connected network. That is, by connecting to the predetermined web page through a network, it may be possible to transmit or receive data to or from a relevant server.
Further, the network interface unit 133 may receive contents or data provided by a content provider or a network operator. That is, the network interface unit 133 may receive contents such as movies, advertisements, games, VODs, broadcast signals and information related therewith that are provided from the content provider or network operator through a network.
Also, the network interface unit 133 may receive update information and update files of a firmware provided by the network operator, and transmit data to the Internet or content provider or the network operator.
The network interface unit 133 may select and receive a desired application among applications opened to the public through a network.
The storage unit 140 may store programs that are used to process and control each signal in the control unit 170, and signal processed images, sounds or data signals.
Also, the storage 140 may perform a function to temporarily store images, sounds or data signals inputted from the external device interface unit 135 or the network interface unit 133, and store desired image information through a channel memory function.
The storage unit 140 may store an application or an application list inputted from the external device interface unit 135 or the network interface unit 133.
The display device 100 may play back content files (moving image file, still image file, music file, document file, application file, etc.) stored in the storage unit 140 to provide them to a user.
The user input interface unit 150 may transfer a signal inputted by the user to the control unit 170, or transfer a signal from the control unit 170 to the user. For example, the user input interface unit 150 may receive control signals such as power on/off, channel selection and screen setting from the remote control device 200 to process, according to a variety of communication schemes such as Bluetooth scheme, WB (Ultra Wideband) scheme, ZigBee scheme, RF (Radio Frequency) communication scheme or IR communication scheme. Further, the user input interface unit 150 may serve to transmit a control signal from the control unit 170 to the remote control device 200.
Also, the user input interface unit 150 may transfer control signals inputted from a local key (not shown) such as power key, channel key, volume key, setting key, etc. to the control unit 170.
An image signal processed in the control unit 170 is inputted to the display unit 180 so that it may be displayed as an image corresponding to the image signal. Also, the image signal processed in the control unit 170 may be inputted to an external output device through the external device interface unit 135.
The sound signal processed in the control unit 170 may be audibly outputted through the audio output unit 185. Also, the sound signal processed in the control unit 170 may be inputted to an external output device through the external device interface unit 135.
Besides, the control unit 170 may control general operations of the display device 100.
Also, the control unit 170 may control the display device 100 using user commands or internal programs inputted through the user input interface unit 150, and may be connected to a network so that a user's desired application or an application list is to be downloaded into the display device 100.
The control unit 170 enables channel information and the like selected by the user together with a processed image or sound signal to be outputted through the display unit 180 or the audio output unit 185.
Also, the control unit 170 enables image signals or sound signals from an external device such as a camera or a camcorder, which are inputted through the external device interface unit 135, to be outputted through the display unit 180 or the audio output unit 184, according to an external device image play command received through the user input interface unit 150.
Meanwhile, the control unit 170 may control the display unit 180 to display an image. For example, the control unit 170 may control the display unit 180 to display a broadcast image inputted through a tuner 131, an external input image inputted through the external device interface unit 135, an image inputted through the network interface unit, or an image stored in the storage unit 140, on the display unit 180. In this case, the image displayed on the display unit 180 may be a still image or a moving image, and a 2D image or a 3D image.
Also, the control unit 170 may control to play back contents stored in the display device 100, received broadcast contents, or external input contents inputted from the outside, and the contents may be a variety of forms such as a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.
The display unit 180 may convert into R, G and B signals an image signal, a data signal and an OSD signal that are processed in the control unit 170, or an image signal and a data signal received in the external device interface unit 135 to generate a driving signal.
Meanwhile, since the display device 100 illustrated in
That is, according to need, two or more elements may be integrated in one element or one element may be subdivided into two or more elements. Also, functions performed in each block are used to describe embodiments of the present invention, and their detailed operations or devices do not limit the right scope of the present invention.
According to another embodiment of the present invention, the display device 100 may receive images through the network interface unit 133 or the external device interface unit 135 to play back without comprising the tuner 131 and a demodulation unit 132, differently from
For example, the display device 100 may be separately embodied in two devices, that is, an image processing device such as a set top box to receive a broadcast signal or contents depending on a variety of network services and a content play device to play back contents inputted from the image processing device.
In this case, a method for operating a display device to be described below according to embodiments of the present invention may be performed by any one of the image processing device such as the separated set top box and the content play device comprising the display unit 180 and the audio output unit 185, as well as the display device 100 described with reference to
Next, a remote control device according to an embodiment of the present invention will be described with reference to
At first, referring to
Referring to
The remote control device 200 may comprise an RF module 221 capable of transmitting and receiving signals to and from the display device 100 according to an RF communication standard, and an IR module 223 capable of transmitting and receiving signals to and from the display device 100 according to an IR communication standard.
Also, the remote control device 200 transmits signals including information on a motion of the remote control device 200 to the display device 100 through the RF module 221.
Meanwhile, the remote control device 200 may receive the signal transmitted from the display device 100 through the RF module 221, and transmit commands on power on/off, channel change and volume change to the display device 100 through the IR module 223 according to need.
The user input unit 235 may be configured of a key pad, a button, a touch pad, or a touch screen. The user may input commands related to the display device 100 to the remote control device 200 by operating the user input unit 235. If the user input unit 235 comprises a hard key button, the user may input commands related to the display device 100 to the remote control device 200 through a push operation of the hard key button.
If the user input unit 235 comprises a touch screen, the user may input commands related to the display device 100 to the remote control device 200 by touching a soft key of the touch screen. Also, the user input unit 235 may comprise various kinds of input means such as a scroll key or a jog key that can be operated by user, and the present embodiment does not limit the right scope of the present invention.
The sensor unit 240 may comprise a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 may sense information on a motion of the remote control device 200.
For example, the gyro sensor 241 may sense information on an operation of the remote control device 200 based on x, y and z axes, and the acceleration sensor 243 may sense information on a moving speed of the remote control device 200 and the like. Meanwhile, the remote control device 200 may further comprise a distance measuring sensor, so that it may sense the distance between the display device 100 and the display unit 180.
The output unit 250 may output image or sound signals corresponding to an operation of the user input unit 235 or the signals transmitted from the display device 100. The user may recognize whether the user input unit 235 is operated or the display device 100 is controlled, through the output unit 250.
For example, the output unit 250 may comprise an LED module 251 to be turned on, a vibration module 253 to generate a vibration, a sound output module 255 to output a sound and a display module 257 to output an image, if the user input unit 235 is operated or the output unit transmits and receives signals to or from the display device 100 through the wireless communication unit 225.
Also, the power supply unit 260 may supply the remote control device 200 with power, and stop supplying power if the remote control device 200 does not move for a predetermined time, reducing power waste. The power supply unit 260 may begin supplying power again if prescribed keys included in the remote control device 200 are operated.
The storage unit 270 may store various kinds of programs and application data that are needed to control or operate the remote control device 200. If the remote control device 200 transmits and receives signals to and from the display device 100 by wireless through the RF module 221, the remote control device 200 and the display device 100 may transmit and receive signals each other through a predetermined frequency band.
The control unit 280 of the remote control device 200 may store in the storage unit 270 information on the frequency band on which signals are transmitted and received to and from the display device 100 paired to the remote control device 200 and the like by wireless, and refer to the information.
The control unit 280 controls various matters related to the control of the remote control device 200. The control unit 280 may transmit a signal corresponding to an operation of the prescribed keys of the user input unit 235 or a signal corresponding to a motion of remote control device 200 sensed in the sensor unit 240 to the display device 100 through the wireless communication unit 225.
Next, a method for operating the display device 100 according to an embodiment of the present invention will be described with reference to
The control unit 170 of the display device 100 plays back a broadcast program (S101). In an embodiment, the control unit 170 may play back an image of the broadcast program received through a tuner 131. The control unit 170 may output an image of the broadcast program and a sound of the relevant broadcast program so as to play back the broadcast program, and the user may watch the relevant broadcast program through the image and the sound of the broadcast program outputted.
The control unit 170 identifies whether the user is in a visual watching status in which the user is visually watching the broadcast program through the display unit 180 of the display device 100 (S103).
In an embodiment, a user's watching status may include a visual watching status, a non-visual watching status and an audiovisual watching status.
The visual watching status may mean a status in which the user is visually watching the broadcast program through the display unit 180. In case that the user is watching the broadcast program through the sound of the broadcast program only without watching the broadcast program through the display unit 180, it may be obtained that the user is in a non-visual watching status.
The non-visual watching status may mean that the user is not watch the broadcast program through the display unit 180 but is listening to the sound of the broadcast program.
The audiovisual watching status may include the visual watching status and the non-visual watching status, which may mean a status that the user is listening to the sound of the broadcast program while the user is watching the image of the broadcast program.
The non-visual watching status means that the user is doing other actions without watching the display unit 180 that is playing back the broadcast program, which may include status that the user closes his or her eyes, is looking at places other than the display unit 180, or moves other places where there is not the display device 100.
The control unit 170 may identify whether the user is in the visual watching status or the non-visual watching status, using at least one of a face recognition technology and an eye tracking technology.
The face recognition technology may include any one of a knowledge-based method, a feature-based method, a template-matching method and an appearance-based method. Also, the control unit 170 may extract a face area of a user using any one of the above methods and identify whether the user is in the visual watching status through an extracted face area of the user.
The knowledge-based method is a method of detecting a face area based on relationship of fixed distances and positions of eyes, nose and mouth of the face.
The feature-based method is a method of detecting a face area using information on size, shape and interrelation of facial elements, information on color and texture of the face, or information on the combination of such elements.
The template-matching method is a method of producing a standard template for all faces and then comparing it with an input image to detect a face area.
The appearance-based method is a method of detecting a face area using a model learned by a learning image set. The appearance-based method may include a skin color detecting method. The skin color detecting method is a method of simply extracting candidate areas for a face and hands from a user's image, where it may be possible to detect the candidate area easily and quickly although the candidate area may not be correct due to the large effect caused by a lighting and a camera. In detail, the skin color method is a method of obtaining an RGB image of the user, converting it into a YUV image, detecting a skin color using U and V components, and extracting a face area and a hands area of the user through the detected skin color.
The eye tracking means a method of tracking user's eyes for a predetermined time, where a motion of user's pupils is observed and an operation corresponding to the motion is performed.
The display device 100 may further comprise a camera for an eye tracking.
Also, the control unit 170 may identify a motion of the user with respect to a mobile terminal 300 to identify whether the user is in the visual watching status. For example, in case that the user moves to a place other than where the display device 100 is located so that the user is not in the visual watching status, the control unit 170 may receive from the mobile terminal 300 watching status information indicating that the user is not in the visual watching status.
For this, the control unit 170 may comprise a wireless communication unit (not illustrated) to perform a near-field communication NFC with the mobile terminal 300 of the user, and the mobile terminal 300 may also transmit and receive information to and from the wireless communication unit of the display device 100 through the near-field communication.
Also, the mobile terminal 300 may comprise one or more sensors in order to identify a user's watching status. For example, the sensor may comprise at least one of a geomagnetic field sensor, an acceleration sensor, and a hall sensor.
The geomagnetic field sensor may sense the intensity of magnetic field with three (3) axes to identify a moving status of the user. The geomagnetic field sensor may sense the moving status of the mobile terminal 300 to transmit the sensed moving status of the mobile terminal 300 to the display device 100. In case that the display device 100 recognizes that the mobile terminal 300 is away from the display device 100 by a predetermined distance using the received moving status of the mobile terminal 300, the display device 100 may determine that the user is in the non-visual watching status.
The acceleration sensor may sense a moving status of the mobile terminal 300 with three (3) axes and the hall sensor may sense opening and closing status of a cover of the mobile terminal 300 utilizing a principle that the voltage changes depending on the intensity of magnetic field.
The acceleration sensor may sense the moving status of the mobile terminal 300 to transmit the sensed moving status of the mobile terminal 300 to the display device 100, and the hall sensor may sense opening and closing status of a cover of the mobile terminal 300 to transmit the sensed opening and closing status to the display device 100. In case that the mobile terminal 300 is away from the display device 100 by a predetermined distance or more and the cover of the mobile terminal 300 is opened, the display device 100 may determine that the user is in the non-visual watching status using the received moving status of the mobile terminal 300 and the opening and closing status of the cover.
The mobile terminal 300 may be any one of a smart phone, a smart pad, a head set, and an ear phone, and communicate with the display device 100 through the near-field communication.
The near-field communication may be any one of WiFi scheme, Bluetooth scheme, and Zigbee scheme.
A process to identify a user's watching status is described with reference to
At first,
Referring to
Next, referring to
Next,
Referring to
A description for
If it is obtained that the user is in the visual watching status, the control unit 170 returns the process to step S101 to maintain the play back of the broadcast program.
On the contrary, if it is obtained that the user is not in the visual watching status, the control unit 170 identifies whether the sound of the broadcast program is being transmitted to the mobile terminal 300 connected to the display device 100 (S105), and the control unit 170 transmits information of the display device 100 to the mobile terminal 300 (S107) if it is obtained that the sound of the broadcast program is being transmitted to the mobile terminal 300 connected to the display device 100.
If the control unit 170 is transmitting the sound of the broadcast program being played back in the display unit 180 to the mobile terminal 300 of the user, the control unit 170 may transmit information of the display device 100 to the mobile terminal 300.
If the control unit 170 is transmitting the sound of the broadcast program being played back in the display unit 180 to the mobile terminal 300 of the user, the control unit 170 may transmit information of the display device 100 to the mobile terminal 300 while outputting the sound of the broadcast program.
The information of the display device 100 may include watching information and subtitle information of the program being broadcasted currently and event information generating in the display device 100. The subtitle information and event information generating in the display device 100 will be described below.
In an embodiment, the watching information of the broadcast program may include at least one of a remaining time of the broadcast program that is currently played back, an end time of the broadcast program that is currently played back, a title of the broadcast program that is currently played back, a rating of the broadcast program that is currently played back, and a commentary of the broadcast program that is currently played back.
In an embodiment, the control unit 170 may additionally transmit watching information of the broadcast program while transmitting the sound of the broadcast program to the mobile terminal 300. That is, if the user is not in the visual watching status but in non-visual watching status, the user may audibly listen to the watching information of the broadcast program while listening to the sound of the broadcast program that is currently played back in the display device 100 through the mobile terminal 300. As such, the user may audibly identify the watching information of the broadcast program even while he or she is not watching the broadcast program that is currently played, thereby enhancing understanding of the broadcast program.
In another embodiment, if the sound of the broadcast program is being transmitted to the mobile terminal 300 connected to the display device 100 and the user is in the non-visual watching status, the control unit 170 may turn off the power of the display unit 180 and transmit the image of the broadcast program that is currently played back to the mobile terminal 300. The user may audibly watch the sound of the broadcast program through the display device 100 while visually watching the image of the broadcast program through the mobile terminal 300. That is, the user may listen to the sound of the broadcast program through the display device 100 while watching the broadcast program through the mobile terminal 300. More description will be given with reference to
Referring to
A description will be given again with reference to
Meanwhile, if the sound of the broadcast program is not transmitted to the mobile terminal 300 connected to the display device 100, the control unit 170 outputs the information of the display device as an audio (S109).
If the control unit 170 is not transmitting the sound of the broadcast program that is currently played back in the display unit 180 to the mobile terminal 300 of the user, it may output the information of the display device 100 through the audio output unit 185.
The control unit 170 may output the information of the display device 100 through the audio output unit 185 while outputting the sound of the broadcast program that is currently played back.
If the sound of the broadcast program is not being transmitted to the mobile terminal 300 connected to the display device 100 and the user is in the non-visual watching status, the control unit 170 may output the information of the display device 100 through the audio output unit 185 while not playing back the image of the broadcast program. That is, if the sound of the broadcast program is being transmitted to the mobile terminal 300 connected to the display device 100 and the user is in the non-visual watching status, the power of the display unit 180 is turned off so that the information of the display device 100 only may be audibly outputted. Those will be described with reference to
Referring to
In this case, the user may audibly listen to the information of the display device 100 so that it may be possible to utilize information easily and the display device 100 may turn off the power of the display unit 180, thereby preventing the power from being wasted.
Further, if the user is in the non-visual watching status, the display device 100 may control the volume of the sound outputted depending on the distance between the user and the display device 100. For example, if the user is in the non-visual watching status and if it is determined that the distance between the user and the display device 100 becomes great, the display device 100 may increase the volume of the output sound. Also, if it is determined that the distance between the user and the display device 100 becomes short, the volume of the output sound may be reduced. The display device 100 may further comprise a distance sensor (not illustrated) in order to measure the distance between the user and the display device 100. The display device 100 may measure the distance between the user and the display device 100 using any one of triangulation method and TOF (Time Of Flight) method that employ a distance sensor.
A description will be given again with reference to
The information of the display device 100 may include watching information and subtitle information of the broadcast program that is currently played back and event information generating in the display device 100.
The watching information of the broadcast program may include at least one of a remaining time of the broadcast program that is currently played back, an end time of the broadcast program that is currently played back, a title of the broadcast program that is currently played, a rating of the broadcast program that is currently played back, and a commentary of the broadcast program that is currently played back.
Especially, if the user is basically in the non-visual watching status, like the blind, the display device 100 may identify that to audibly output the commentary of the broadcast program among the watching information of the broadcast program. For this, the tuner 131 of the display device 100 may receive the broadcast signal including the commentary of the broadcast program. That will be described with reference to
Each descriptor may be included in the broadcast signal to be transmitted through the tuner 131.
Referring to
If the user is in the non-visual watching status, the display device 100 may audibly output a commentary of the broadcast program based on the descriptor included in the received broadcast signal. Accordingly, there is an advantage that the blind user may understand broadcast program contents easily even though he or she is not able to visually watch the broadcast program.
A description will be given again with reference to
The subtitle information may be subtitles related to the broadcast program that is currently played back. That is, if a foreign broadcast program is currently played back, the subtitle information may be a Korean-translated subtitle. Also, if the user is in the non-visual watching status, the display device 100 may convert the subtitle of the related broadcast program into a sound to output.
The subtitle information may be a subtitle that is not related to the broadcast program that is currently played back. That is, the subtitle information is a subtitle for advertisement or breaking news, which is not related to the broadcast program that is currently played back. Even in this case, if the user is in the non-visual watching status, the display device 100 may convert into a sound the subtitle that is not related to the relevant broadcast program to output.
The display device 100 may convert a subtitle into a sound to output, using Text-to-Speech conversion technology.
Event information generating in the display device 100 may include scheduled recording start notification information and channel switching notification information according to a scheduled watching. If the user is in the non-watching status, the display device 100 may convert the event information into a sound to output.
The more detailed description will be given with reference to
If the user is in the non-visual watching status, the display device 100 may display a notification <1 minute up to the scheduled watching time of the broadcast program 1. Will you switch to the relevant channel?> while playing back an image of the current broadcast program, and convert the notification into a sound to output.
According to an embodiment of the present invention, the above described method may be embodied as codes readable by a processor on a medium that has program recorded thereon. Examples of the medium readable by the processor may include ROM, RAM, CD-ROM, magnetic tape, floppy disc, optical data storage device, etc. and the medium may be embodied in a carrier wave (for example, a transmission on Internet).
The display device described above may not be restrictively applied to the examples and methods of the embodiments described above, and all or parts of the embodiments may be selectively combined so that the embodiments may be variously modified.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2014-0000369 | Jan 2014 | KR | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/KR2014/002586 | 3/27/2014 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2015/102158 | 7/9/2015 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20060061684 | Easley | Mar 2006 | A1 |
| 20110206353 | Yeo | Aug 2011 | A1 |
| 20130183021 | Osman | Jul 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 5302876 | Oct 2013 | JP |
| 10-2013-0107093 | Oct 2013 | KR |
| 10-2013-0107555 | Oct 2013 | KR |
| 10-2013-0127777 | Nov 2013 | KR |
| WO 2012162060 | Nov 2012 | WO |
| Entry |
|---|
| International Search Report dated Oct. 21, 2014 issued in Application No. PCT/KR2014/002586 (with English Translation). |
| Number | Date | Country | |
|---|---|---|---|
| 20160316232 A1 | Oct 2016 | US |
