Patent Application
20140380345
The present application relates generally to user interfaces (UI) for audio video display devices (AVDD) such as televisions (TVs).
Current televisions are controlled mainly by a viewer manipulating a remote control (RC). As understood herein, remote controls can be temporarily misplaced and are not ideal for text entry, and furthermore allow only a single user to control the TV. As also understood herein, a viewer's gestures can be used as an additional command input mode. However, when multiple viewers are watching a TV, multiple people in one room can possibly control a TV with gestures. It is preferred for the viewing experience that only one person be in control at a time to avoid the input in rapid succession of conflicting commands.
As understood herein, a nifty resolution of this conundrum is to accept gestures from only a single person but allow that person to specifically pass control to another viewer in the room.
Accordingly, an AVDD may include a processor and a video display that may present demanded images under control of the processor. In non-limiting embodiments, the AVDD may also include a computer readable storage medium bearing instructions executable by the processor to receive image signals from a camera that images viewers of the AVDD. Responsive to a determination that a first viewer has initiated a gesture command input mode to control the AVDD, the processor responds to gestures only from the first viewer to control the AVDD. Responsive to a determination that the first viewer has looked toward a second viewer, the processor determines whether the first viewer has made a voice commend or body gesture other than looking toward the second person to confirm that the first viewer wishes to pass control of the AVDD to the second person. Thus, responsive to a determination that the first viewer has made a voice command or body gesture other than looking toward the second person confirming a wish to transfer control, the processor transfers gesture control to the second person such that the processor responds to gesture commands only from the second viewer. The body gesture other than looking toward the second viewer may be a pointing gesture of the first viewer toward the second viewer
In some embodiments the processor determines that the first viewer has looked toward the second viewer by determining that the head of the first viewer has turned toward the second viewer. In other embodiments the processor determines that the first viewer has looked toward the second viewer by determining that the pupils of the first viewer have moved toward the second viewer. Responsive to a determination that the first viewer has looked toward the second viewer, the processor can overlay a message on the display prompting the first viewer to confirm that the first viewer wishes to pass control of the AVDD to the second viewer.
In another aspect, a television (TV) includes a display and a processor controlling the display and receiving signals representing human gestures. The processor is programmed to respond to gestures from a first viewer to control the display. Also, the processor is programmed to respond to gestures from a second viewer to control the display only responsive to a determination that the first viewer has both looked toward the second viewer, and that the first viewer has confirmed, as a separate act from looking toward the second viewer, a desire to transfer control of the TV to the second viewer.
In another aspect, a method includes imaging first and second viewers of an audio video display device (AVDD). The method further includes controlling the AVDD based on gesture commands of the first viewer. The gesture commands are gestures in free space of the first viewer. Further, the method includes controlling the AVDD based on gesture commands of the second viewer only responsive to a predetermined first gesture of the first viewer and a confirmation command from the first viewer.
The details of the present invention, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which:
Referring initially to the non-limiting example embodiment shown in
TV programming from one or more terrestrial TV broadcast sources 32 as received by a terrestrial broadcast antenna 34 which communicates with the AVDD 12 may be presented on the display 28 and speakers 22. The terrestrial broadcast programming may conform to digital ATSC standards and may carry within it a terrestrial broadcast EPG, although the terrestrial broadcast EPG may be received from alternate sources, e.g., the Internet via Ethernet, or cable communication link, or satellite communication link.
TV programming from a cable TV head end 36 may also be received at the TV for presentation of TV signals on the display 28 and speakers 22. When basic cable only is desired, the cable from the wall typically carries TV signals in QAM or NTSC format and is plugged directly into the “F-type connector” 38 on the TV chassis in the U.S., although the connector used for this purpose in other countries may vary. In contrast, when the user has an extended cable subscription for instance, the signals from the head end 36 are typically sent through a STB 40 which may be separate from or integrated within the TV chassis but in any case which sends HDMI baseband signals to the TV. Other types of connections may be used, e.g., MOCA, USB, 1394 protocols, DLNA.
Similarly, HDMI baseband signals transmitted from a satellite source 42 of TV broadcast signals received by an integrated receiver/decoder (IRD) 44 associated with a home satellite dish may be input to the AVDD 12 for presentation on the display 28 and speakers 22. Also, streaming video may be received from the Internet 46 for presentation on the display 28 and speakers 22. The streaming video may be received at the computer modem 24 or it may be received at an in-home modem 48 that is external to the AVDD 12 and conveyed to the AVDD 12 over a wired or wireless Ethernet link and received at an RJ45 or 802.11x antenna on the TV chassis.
Also, in some embodiments a video camera 50, which may be integrated in the chassis if desired or mounted separately and electrically connected thereto, may be connected to the processor 16 to provide to the processor 16 video images of viewers looking at the display 28. In some non-limiting embodiments, the processor 16 executes facial recognition using the video images received from the camera 50 to identify particular viewers, to identify the direction in which a viewer is looking, and to capture viewer gestures for purposes of command input to the processor 16. The processor 16 also stores images of viewers on the medium 20 in non-limiting embodiments for future determination of whether a viewer is recognized when compared to stored images, or whether a recently received image relates to a new, or non-recognized, viewer. If desired, the processor 16 also creates and/or associates a profile with a user and/or stored images of the user and store the profile on the medium 20. In embodiments which envision, in addition to gesture command input mode, voice command input mode, a microphone 52 may also be provided on the chassis or separate therefrom and can be electrically connected to the processor 16 to provide viewer-generated voice commands to the processor 16.
Also, a back key 64 may be provided to cause the display 28 to go back a screen shot, i.e., to present the screen shot immediately prior to the one being displayed when the back signal is generated, so as to enable a user to navigate through the various GUIs shown herein. A home key 66 may be provided to cause the below-described “home” GUI to be presented on the display 28, and an options key 68 may be provided to allow a user to cause a list of GUI options to be presented in the display 28. Volume output by the speakers 22 of the AVDD 12 can be controlled using a volume up/down rocker key 70. Manipulation of a key or the pad 62 on the RC 30 causes a respective signal to be sent to an RC processor 72 in the RC 30, which transmits the corresponding command to the AVDD 12 using a wireless transceiver 74 such as but not limited to an infrared transceiver. In addition, if desired four arrow keys may be arranged on the RC 30 to move a cursor up, down, left, and right on the display 28. Also, channel up/down keys may be provided as well as a microphone for voice input. A full QWERTY keyboard or keypad may be provided if desired. The touch pad 62 may be provided with a buzzer or other device to generate tactile feedback to the user.
For illustration purposes,
As shown at 78, a numeral may be provided next to each panel, so that the panels may be numbered onscreen from one to nine as shown. This facilitates a user selecting to play video content from a panel by saying the numeral associated with the panel. Also, as shown at 80 a genre indicator may be presented next to each panel. The genre indicator may be an arrow in the case of the first panel 76 indicating that the first panel 76 is presenting the currently selected video. The genre indicator may be a heart (as shown in
Additionally, if desired the GUI shown in
Further, a menu of selections may be presented as shown along the bottom of the GUI, including a “just for you” selector 84 to cause the GUI to contain panels with content personalized to a recognized user. A search selector 86 can be selected to cause a search for a user-input term to be executed. Also, a bookmark selector 88 can be selected to bookmark the currently playing video in panel 76 or to bookmark content in a panel over which the screen cursor is positioned.
Still referring to
In any case, as mentioned above in the example shown the currently selected video content is always presented in the upper left panel 76 of the GUI. Should the user select another panel by using the RC 30 or by speaking the number of the panel or by the appropriate gesture captured by the camera S0, video content from the source associated with the selected panel automatically replaces the video content which was presented in the upper left panel 76 prior to receiving the new select signal. Indeed, should the user scroll the panels left or right to view additional content the upper left panel 76 remains unmoving as the other panels move on and off the display 28 as they are replaced by previously hidden content panels, with the upper left panel 76 always showing the currently selected video program.
The panel layout shown in
Transition between the two views of
The GUI shown in
Non-limiting examples of corresponding RC, voice, and gesture commands that accomplish the same tasks are:
Now in reference to
Thus, beginning at block 106, the logic enables gesture commands to be detected and/or received by the AVDD 12. Then at block 108 the logic receives a gesture command. Moving to decision diamond 110, the logic then determines whether a conflict of commands exists between the gesture command received at block 10 and a RC command also received within X seconds (before or after) of the gesture command.
If the logic determines that there is a conflict of commands at diamond 110, the logic then moves to block 112 where the logic implements the RC command and not the gesture command. However, if the logic determines that there is no conflict of commands between the gesture command and a RC command at diamond 110, the logic instead moves to decision diamond 114. At diamond 114, the logic determines whether there is a conflict between the gesture command received at block 108 and a voice command also received within X seconds of the gesture command.
If the logic determines that there is a conflict of commands at diamond 114, the logic then moves to block 116 where the logic implements the voice command and not the gesture command. However, if the logic determines that there is no conflict of commands between the gesture command and the voice command at diamond 114, the logic instead moves to decision diamond 118. At diamond 118, the logic determines whether there is a conflict between the gesture command received at block 108 and another gesture command received within X seconds of the first gesture command.
If the logic determines that there is no conflict of gesture commands at diamond 118, the logic then moves to block 120 where the logic implements the non-conflicting gesture command. However, if the logic determines that there was a conflict of gesture commands at diamond 118, the logic instead moves to decision diamond 122. At diamond 122, the logic, determines whether one or more users providing the gesture commands are recognized in accordance with present principles.
If the logic determines that one or more users are not recognized at diamond 122, the logic then moves to block 124 where the logic implements the latest gesture command received by the AVDD 12. However, if the logic determines that at least one user is recognized at diamond 122, the logic instead moves to block 126. At block 126, the logic implements the latest gesture command from a recognized user. The logic may thus conclude at either block 124 or block 126, depending on the determination made at diamond 122.
Now in reference to
As contemplated herein, control may be passed from the active viewer to another viewer who then assumes the role of the active viewer by sensing a predetermined gesture from the active viewer accompanied by a confirmation. In the embodiment shown in
In any case, a positive determination at decision diamond 134 may, in some embodiments, cause the logic to flow to decision diamond 136 to determine whether multiple “second” viewers are located near each enough to each other as to make it infeasible to positively identify which of the “second” viewers the active viewer is looking toward. If this turns out to be the case, the logic moves to block 138 to select one and preferably only one of the second viewers as the potential new active viewer pending confirmation as discussed below. This selection may be according to one or more selection rules. In one implementation, the second viewer closer to the active viewer than other second viewers is selected. In another implementation, the second viewer closer to the AVDD than other second viewers is selected. In yet another implementation, the second viewer who looks back toward the active viewer after the active viewer has directed his gaze at the second viewer is selected.
From block 138 or from decision diamond 136 if the processor can identify which person in the room constitutes the second viewer toward whom the active viewer has directed his gaze, the logic moves to block 140 if desired to prompt the active viewer to confirm that the active viewer wishes to relinquish gesture control or the VADD to the second viewer. This prompt may be an alpha-numeric message overlaid onto video on the display 28 (or played on the speaker) for a short period of time instructing the active viewer to confirm. For example, the message might be “do you want to relinquish control?” If the second viewer is recognized the message may be more specific: ““do you want to relinquish control to Sam?” Or, the message may specifically instruct the viewer that to confirm a desire to relinquish control, speak a particular command such as “Done” or to make a particular gesture such as pointing to the second person or simply nodding.
If confirmation is received by the processor at decision diamond 142, the logic moves to block 144 to make the active viewer for gesture control purposes the second viewer, looping back to block 132. If confirmation is not received, preferably within a timeout period, the logic loops back to block 132 with the active viewer unchanged.
Present principles may be used as a game where a virtual object gets passed around a room.
While the particular PASSING CONTROL OF GESTURE-CONTROLLED APPARATUS FROM PERSON TO PERSON is herein shown and described in detail, it is to be understood that the subject matter which is encompassed by the present invention is limited only by the claims.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13367606 | Feb 2012 | US |
| Child | 14480210 | US |
