This disclosure relates to the field of remote control of home entertainment equipment, and in particular to a remote controller including motion sensing and or a touch sensing display screen.
A typical set top box provides a user interface to allow control of the set top box menu and or an electronic program guide. The menu and electronic program guide are respectively resident, or received, formatted and stored, within in the set top box and can be viewed as on screen displays (OSD) or graphic overlays. Typically the guide or menu features may be activated by physical button pushes or by remote control commands via an exemplary modulated IR link. The guide and menu are intended for user interaction via an on screen display on a TV or video display monitor. Normal display viewing distances dictate that the screen is physically distant from the user and certainly beyond arms reach. A remote audio video display, television, video monitor or viewing tablet may be coupled to view the set top box output signals and clearly substantially the same set top box control as provided to a local viewer must be available to the remote display viewer.
In an inventive arrangement a touch sensitive viewing device has bidirectional communication with a set top box. The touch sensitive viewing device facilitates sound and image display and provides information to control the set top box and ancillary equipment. In a further inventive arrangement transitory motion and or spatial reorientation of the touch sensitive viewing device can control selections within the set top box and ancillary equipment.
a and 3b are exemplary sequences used in the arrangements of
a, 6b, 6c, 6d depict the viewing tablet of
Set top box (STB) 100 also has a separate information and control channel connection (104) which couples a bidirectional data bus signal 110 between set top box 100 and audio video (A/V) interface converter 200. Advantageously the output audio video signal, possibly with graphic overlay, which is coupled to display 10 is also coupled to the audio video interface converter 200 which forms a digital A/V stream using for example, a compression algorithm such as MPEG 4. This compressed digital A/V stream is processed or modulated for transmission by antenna 225 to a remote viewing device 300 using an exemplary Wi-Fi or digital wireless link 226 operating in accordance with an exemplary standard such as IEEE 802.11. An antenna 325 at or within viewing device 300 receives the digital A/V stream 226 and forms therefrom an image with sound, substantially as provided to display 10.
However, as discussed previously, the program guide and set top box menu information are resident and or generated within the set top box and may be activated by physical button pushes or remote control commands. The guide and the menu, are intended for user interaction via an OSD or graphical image on a video display. Furthermore, optimal viewing conditions suggest a certain minimum viewing distance which places the screen of display 10 distant from the user and certainly beyond his reach. Thus, the user set top box interaction is usually performed remotely from the display screen which tends to preclude the concept of screen touch control of the STB user interface, not to mention the undesirability of finger-prints on the screen.
A user viewing display screen 301 may select, by means of a finger or exemplary stylus 310, a particular screen portion, for example a section of an OSD 305 forming part of image 303. As is known, the touch sensitive interface of viewing tablet 300 includes circuitry which generates coordinates 315 that locate the screen touch within the touch sensitive area 302. These touch coordinates 315 are transmitted from viewing tablet 300 via antenna 325 to STB 110 socket 104, via interface converter 200. A bidirectional path 201, 226 provides A/V content 101 for viewing as image 303 on screen 301, and also provides a reverse channel to couple the touch coordinates or control instructions to STB100. The touch coordinate or coordinates are input to STB 100 for processing and mapping to the current screen display, typically the OSD or guide grid. Image mapping for viewing tablet 300 may be performed based on certain parameters such as the aspect ratio of tablet 300 and possibly the aspect ratio of the source image, for example to view wide screen movies. Tablet 300 of
The remote viewing space 301 of display tablet 300 is mapped to the STB user interface space in order that touch coordinates from the remote viewing device are interpreted to initiate the user desired set top box command. The interpreted command may result for example, in a change to the OSD information or change in program selection, and this result is then communicated to both display 10 and remote display 300. Furthermore, manipulation and orientation of display tablet 300 also forms not only an essential part of the mapping to the STB user interface space but in addition may provide control capabilities. As mentioned previously, twisting or rotating tablet 300 through approximately 90 degrees must be accommodated, not only in terms of the image but also to maintain user operability by altering the functional mapping of specific touch screen areas. For example, a user function associated with sense area 302a of
Display tablet 300 advantageously includes a device for detecting the orientation and or movement of the tablet as depicted in
As mentioned previously, display tablet 300 may include an accelerometer such as typel LIS302DL manufactured by ST Microelectronics, or similar device for detecting motion and orientation of the tablet. Tablet orientation or motion information is supplied to interface 200 and set top box 100 to advantageously provide control commands, and for certain tablet orientations to remap only video signal 101 for display on the rotated display tablet 300. In addition the touch coordinates may be remapped in accordance with the display orientation and chosen display layout. Tablet 300 orientation with consequential image size and layout changes must be implemented independently from the video representative signal 101 supplied to display 10. To facilitate the image size and layout changes resulting from reorienting tablet 300 audio video interface converter 200 includes a digital image processing arrangement which includes image processing control, image scaling, and memory. In addition interface converter 200 may receive from set top box 100 data representing the electronic program guide and the set top box menu. These data may be provided separately from and not combined with video representative signal 101. In this way interface converter 200 can format an image signal specifically for display by tablet 300 without effecting the image on display 10.
a shows an exemplary sequence of touch screen control operations 350-365 occurring between remote viewer 300 and set top box 100 and applies to the case where the program guide is generated entirely within the set top box and is to be viewed with the remote viewing device. In block 350 the touch screen senses that a touch has occurred. The coordinates 315 of the screen touch are determined, modulated and transmitted by exemplary digital wireless link 226 to converter 200 for demodulation and coupling as signal 110 to set top box 100. The coordinates are received in the set top box where a state machine containing the local state of the menu system and the current location computes the desired location and the appropriate commands to reach that position. In addition, the state machine may also determine that a particular action, for example a change of channels, may be required. This may be communicated as described previously by a second touch or double tap on the screen with similar coordinates to indicate, in addition to possible cursor movement, a particular action is requested. The user interface then receives the appropriate commands and if necessary sends a new cursor location and menu status to the set top box state machine.
b shows an exemplary sequence relating to movement of viewing tablet 300 as has been described and illustrated in
The operational sequence depicted in
It is important to note that the linked list data structure depicted in
The exemplary sequence of coordinate mapping described with respect to
a shows an image 600 on display tablet 300 with an orientation which produces an aspect ratio of, for example, 16:9. Image 600 is shown, for ease of illustration, with only a peripheral array of broken line rectangles, for example 302a and 302aa. These broken line rectangles are representative of individual sensing areas forming touch sensor 302 and these sensing areas may substantially cover display 301. As mentioned previously, by sensing motion of display 300 it is possible to cause or generate control commands without any requirement for a screen touch. Furthermore, certain control functions can be generated by a momentarily to and fro inclination of the display to left or right or forward and back to reduce or obviate the need for screen touches.
b shows possible user movements or displacements of display tablet 300. Displacement of tablet 300 may be resolved by a device such as an accelerometer which resolves motion in three planes X, Y, Z as depicted. Twisting the tablet in the plane of the display is depicted by arrows 601, 602 is resolved by accelerometer values occurring in at least the X and Y sensing planes. Back and forward motion, depicted by arrows 603, 604 is represented by accelerometer values generated in the Z plane or axis.
Arrow 601 shows a possible clockwise directional rotation in the plane of the display. Clearly rotation is also possible in a counter clockwise direction and, in addition, such clockwise and or counter clockwise rotation may be recognized and used to provide differing control commands for selection different functions in the set top box and or interface 200.
However, such spatial reorientation may also be combined with a screen touch at a particular location, for example sensing area 302a, to minimize the number of touches required for a specific control function. For example, a user may grip the display in the left hand with a thumb covering sensing area 302a. Upon rotating display 300 clockwise, information from both the tilt sensor and the touch coordinates are sent to interface 200 and set top box 100. Interface 200 may currently be processing image 600 from the set top box and in addition acquires data representative of the electronic program guide (EPG). Image 600 and the EPG data are scaled and combined to form an exemplary 9:16 image formatted as a picture out of picture by scalier circuitry resident within interface 200. In this way the remote viewer with display 300 may observe a second or alternative image source without interrupting signal 101 coupled to display 10. However, if a selection is made within the exemplary program guide 620b then signal 101 coupled to display 10 will change accordingly.
In
d shows a further image mapping selection where the height of the image 600 is maintained as in the 16:9 tablet orientation of
It will be appreciated that the touch and motion interface of viewing tablet 300 can offer the remote viewer control capabilities which are different from, or absent in the user interface repertoire of a standard set top box. Furthermore these advantageous control capabilities can be translated to utilize appropriate standard set top box commands by software resident within viewing tablet 300 or by supplemental interpretive algorithms added to a standard set top box. These various advantageous arrangements allow the set top box user interface to remain visually and operationally substantially similar regardless of how the set top box guide or menu is controlled i.e. by push buttons switches, remote control commands, touch or motion commands.
This application claims the benefit, under 35 U.S.C. §119 of U.S. Provisional Patent Application Ser. No. 60/994,186, filed Sep. 18, 2007.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US2008/001458 | 2/4/2008 | WO | 00 | 3/16/2010 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2009/038596 | 3/26/2009 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
4825209 | Sasaki et al. | Apr 1989 | A |
5003396 | Kang | Mar 1991 | A |
5545857 | Lee et al. | Aug 1996 | A |
5612719 | Beernink et al. | Mar 1997 | A |
5764179 | Tsurumoto | Jun 1998 | A |
6018336 | Akiyama et al. | Jan 2000 | A |
6396523 | Segal et al. | May 2002 | B1 |
6496122 | Sampsell | Dec 2002 | B2 |
6765557 | Segal et al. | Jul 2004 | B1 |
6861946 | Verplaetse et al. | Mar 2005 | B2 |
7511632 | Nam et al. | Mar 2009 | B2 |
20010054647 | Keronen et al. | Dec 2001 | A1 |
20020006237 | Nakamura et al. | Jan 2002 | A1 |
20020057399 | Ishitaka | May 2002 | A1 |
20020167699 | Verplaetse et al. | Nov 2002 | A1 |
20030023554 | Yap et al. | Jan 2003 | A1 |
20030034957 | Dubil et al. | Feb 2003 | A1 |
20030035075 | Butler et al. | Feb 2003 | A1 |
20040252120 | Hunleth et al. | Dec 2004 | A1 |
20050212760 | Marvit et al. | Sep 2005 | A1 |
20060026521 | Hotelling et al. | Feb 2006 | A1 |
20060026535 | Hotelling et al. | Feb 2006 | A1 |
20060161871 | Hotelling et al. | Jul 2006 | A1 |
20060213267 | Tronconi et al. | Sep 2006 | A1 |
20080088602 | Hotelling | Apr 2008 | A1 |
20080259094 | Kim et al. | Oct 2008 | A1 |
Number | Date | Country |
---|---|---|
1 503 584 | Feb 2005 | EP |
1 801 690 | Jun 2007 | EP |
6030485 | Feb 1994 | JP |
8018813 | Jan 1996 | JP |
2002502138 | Jan 2002 | JP |
2003333361 | Nov 2003 | JP |
2005-522152 | Jul 2005 | JP |
2006287581 | Oct 2006 | JP |
2002-0006237 | Jan 2002 | KR |
WO 9934599 | Jul 1999 | WO |
WO9934599 | Jul 1999 | WO |
WO 03085965 | Feb 2003 | WO |
WO2007086386 | Aug 2007 | WO |
Entry |
---|
Michael Kwan, “The Self-Orienting EV-KD370 Slider Phone From KTF”, Mobile Magazine, Jan. 29, 2007, pp. 1-3, via akihabaranews.com. |
“Harmony 1000, User Manual”, Logitech, version 1.0, pp. 1-15. |
“Harmony 1000 Advanced Universal Remote”, Logitech, Copyright 2008, pp. 1-3. |
“Royal Philips Electronics RC9800i—The Product”, Philips, Copyright 2004, pp. 1-4. |
“Philips Multimedia Control Panel RC9800i—Simple, yet revolutionary control”, Philips, Nov. 6, 2007, version 4.0, pp. 1-2. |
“RC9800i Touch Screen Remote Control User Manual”, Royal Philips Electronics, Copyright 2004, pp. 1-62. |
Search Report Dated Jun. 12, 2008. |
Number | Date | Country | |
---|---|---|---|
20100238109 A1 | Sep 2010 | US |
Number | Date | Country | |
---|---|---|---|
60994186 | Sep 2007 | US |