Patent Application
20150363003
This application relates to a method, a computer-readable medium and a device for providing improved input, and in particular to a method, a computer-readable medium and a device for an improved input for data input in or for controlling a touchless user interface.
Touchless user interfaces have been known since the late 1990s and many solutions have been proposed for how to track an object.
A disadvantage is that the object to be tracked is usually comparatively large. A hand or finger is of considerable size compared to a common display size and especially compared to objects that are displayed on a display. It can therefore be difficult for a user to achieve precise control such as when inputting detailed or complex graphical data or when manipulating objects that are positioned closely to one another.
Especially with disabled user the input of complicated patterns becomes an issue as the disabled may have limited motor skills and is unable to perform input at a required detailed level, especially for devices with small displays.
There is thus a need for a computing device that is capable of providing accurate input even for comparatively large input means.
It is an object of the teachings of this application to overcome the problems listed above by providing a computing device comprising a display and a controller, wherein said controller is configured to detect and track an object via a video stream provided by a camera, detect a movement of the object, translate said movement of the object to a resulting movement of a marker based on a scale, detect a change in distance to the object, and adapt said scale accordingly.
Such a computing device provides for a more accurate input.
In one embodiment the controller is further configured to display an enlarged portion of an object adjacent to the marker or of a general area adjacent or surrounding the marker.
It is also an object of the teachings of this application to overcome the problems listed above by providing a method for use in a computing device comprising a display, said method comprising detecting and tracking an object via a video stream provided by a camera, detecting a movement of the object, translating said movement of the object to a resulting movement of a marker based on a scale, detecting a change in distance to the object, and adapting said scale accordingly.
It is a further object of the teachings of this application to overcome the problems listed above by providing a computer readable medium comprising instructions that when loaded into and executed by a controller, such as a processor, in a computing device cause the execution of a method according to herein.
The inventors of the present invention have realized, after inventive and insightful reasoning that by adapting a scaling according to a distance change a user is able to simply and intuitively provide (control) input at a higher accuracy in a non-linear manner thereby providing the higher accuracy without requiring the user to move the object to be tracked large distances, which may be clumsy and cumbersome or simply impossible.
It should be noted that the scaling is a translation of movement scaling, i.e. a scale according to which a detected movement is translated into a displayed movement and not a zoom scaling or a scaling of an object.
The teachings herein find use in control systems for devices having user interfaces such as mobile phones, smart phones, tablet computers, computers (portable and stationary), gaming consoles and media and other infotainment devices.
Other features and advantages of the disclosed embodiments will appear from the following detailed disclosure, from the attached dependent claims as well as from the drawings. Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein.
All references to “a/an/the [element, device, component, means, step, etc]” are to be interpreted openly as referring to at least one instance of the element, device, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.
The invention will be described in further detail under reference to the accompanying drawings in which:
The disclosed embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout.
Referring to
Referring to
The laptop computer 100 further comprises at least one input unit such as a keyboard 130. Other examples of input units are computer mouse, touch pads, touch screens or joysticks to name a few.
The laptop computer 100 is further equipped with a camera 160. The camera 160 is a digital camera that is arranged to take video or still photographs by recording images on an electronic image sensor (not shown). In one embodiment the camera 160 is an external camera. In one embodiment the camera is alternatively replaced by a source providing an image stream.
Referring to
The TV 100 may further comprise an input unit such as at least one key 130 or a remote control 130b for operating the TV 100.
The TV 100 is further equipped with a camera 160. The camera 160 is a digital camera that is arranged to take video or still photographs by recording images on an electronic image sensor (not shown). In one embodiment the camera 160 is an external camera. In one embodiment the camera is alternatively replaced by a source providing an image stream.
The computing device 200 further comprises a user interface 220, which in the computing device of
The computing device 200 may further comprises a radio frequency interface 230, which is adapted to allow the computing device to communicate with other devices through a radio frequency band through the use of different radio frequency technologies. Examples of such technologies are IEEE 802.11, IEEE 802.15, ZigBee, WirelessHART, WIFI, Bluetooth®, W-CDMA/HSPA, GSM, UTRAN and LTE to name a few.
The computing device 200 is further equipped with a camera 260. The camera 260 is a digital camera that is arranged to take video or still photographs by recording images on an electronic image sensor (not shown).
The camera 260 is operably connected to the controller 210 to provide the controller with a video stream 265, i.e. the series of images captured, for further processing possibly for use in and/or according to one or several of the applications 250.
In one embodiment the camera 260 is an external camera or source of an image stream.
References to ‘computer-readable storage medium’, ‘computer program product’, ‘tangibly embodied computer program’ etc. or a ‘controller’, ‘computer’, ‘processor’ etc. should be understood to encompass not only computers having different architectures such as single/multi-processor architectures and sequential (Von Neumann)/parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other devices. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device whether instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device etc.
The instructions 31 may also be downloaded to a computer data reading device 34, such as a laptop computer or other device capable of reading computer coded data on a computer-readable medium, by comprising the instructions 31 in a computer-readable signal 33 which is transmitted via a wireless (or wired) interface (for example via the Internet) to the computer data reading device 34 for loading the instructions 31 into a controller. In such an embodiment the computer-readable signal 33 is one type of a computer-readable medium 30.
The instructions may be stored in a memory (not shown explicitly in
References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device whether instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device etc.
An improved manner of receiving input through a scaling of the input will be disclosed below with reference to the accompanying figures. The example will be illustrated focusing on the tracked gestures and the resulting movement displayed on a display, but it should be clear that the processing is performed in part or fully in a computing device comprising a controller as disclosed above with reference to
The laptop computer 100 also has a display 120 on which objects 135 are displayed as well as a marker 136. It should be noted that the description herein will be focused on controlling a marker 136, but it should be noted that the teachings herein may also be utilized for controlling a drawing tool, a text input tool or other tool suitable for use in a graphic user interface.
In one embodiment the laptop computer 100 is configured to detect a movement of the tracked hand H and translate the detected movement to a resulting movement for the marker 136. The laptop computer 100 is configured to scale the detected movement to a scale suitable for the resulting movement. In prior art systems the resulting movement matches the detected movement and the scale of such systems can be said to be 1:1.
Especially in touchless user interfaces the object to be tracked is usually comparatively large. A hand or finger is of considerable size compared to a common display size and especially compared to objects that are displayed on a display. It can therefore be difficult for a user to achieve precise control such as when input detailed or complex graphical data or when manipulating objects that are positioned closely to one another. To allow for a more controlled input the laptop computer 100 is configured to scale the input according to a scale based on the distance or change of distance between an object and the display 120 (or camera 160). In this application there will not be made any difference between the distance between the display 120 and the object H and the distance between the camera 160 and the object H.
In
In
The laptop computer 100 may be configured to detect a distance change through determining that an object H is increased/reduced in size. Alternatively the laptop computer 100 may be configured to detect a distance change through detecting a movement of the tracked hand H in a direction perpendicular to plane of camera 160 or display 120. Details on how such a movement may be detected are disclosed in the Swedish patent application SE 1250910-5 and will not be discussed in further detail in the present application. For further details on this, please see the mentioned Swedish patent application. It should be noted, however, that the teachings of the present application may be implemented through the use of other distance changing detection manners than disclosed in Swedish patent application SE 1250910-5.
As will be discussed with relation to
The laptop computer 100 will thus divide any detected movement by a factor two when translating the detected movement to the resulting movement. This requires a user to move his hand (or other object to be tracked) H twice the distance to achieve the same resulting movement of the marker 136. This compensates for the comparatively large size of the object to be tracked H and allows for a more precise input in that it is easier to perform complicated gestures at a larger scale.
It should be noted that the description is focused on angular distances, but the general teaching herein is equally applicable to a tracking system arranged to detect an absolute distance.
Returning to
By providing an enlarged version of the underlying content the user is enabled to provide a more accurate control input.
In one embodiment the provision of an enlarged view 137 is combined with the scaling of the detected movement and is thus comprised in the scaling.
In one embodiment the provision of an enlarged view 137 constitutes the scaling and the zoom factor of the enlarged view corresponds to the scaling factor.
The provision of the enlarged view is not simply a zoom operation in that it also changes the scale of the translation from detected movement to resulting movement.
It should be noted that even though the scaling has been illustrated as being stepwise and dependent on a distance change it should be noted that the teachings herein should not be construed as being limited to detecting a threshold distance, but may be used in conjunction with detecting any distance change.
Also the scaling may be achieved so that any distance change results in a predetermined increase in the scaling.
In one embodiment the distance change is differentiated from distance changes resulting from normal user movements (most users will involuntarily vary the distance also towards the display 120 when performing a gesture) by requiring that the distance change is significant for example with regards to change of size of tracked object, time for movement in direction away from display 120 to name a few possibilities.
In one embodiment the scaling is further dependent on the user. This allows for one user to have a certain scale, perhaps requiring very precise movements, whereas another user may have another scale, perhaps allowing for un-precise, but large, movements. This enables the system to be customized after the experience and abilities of a user.
In one embodiment the scaling is dependent on the tool or marker 136 used or which application is currently being operated. For example, a pen tool in a drawing program may have one scaling setting, whereas a spray can tool may not be enabled for a more precise and accurate input as taught herein. This implements the real world difference between the two emulated tools in that a pen is more accurate than the spray can.
In one embodiment the controller is further configured to detect and track a second object. The second object may also be a hand. In one embodiment the controller is configured for receiving input from the first hand and base an input scale on the input from the first hand, and to receive input from the second hand as control input, possibly limited to a plane parallel to the display plane. This allows a user to, for example, control a cursor with one hand (X,Y) and to control the scale (Z) with the second hand.
The laptop computer 100 is thus configured to, in addition to detecting and tracking the first object H via a video stream 265 provided by a camera 160, 260, also detect and track a second object, detect a movement G1, G2 of the object H and detect a movement of the second object, translate said movement of second the object to a resulting movement M1, M2 of a marker 136 based on a scale, detect a change in distance to the first object H based on the detected movement G1, G2 of the first hand H, and adapt said scale accordingly.
The teachings herein provide the benefit that a more accurate input is achieved.
Another benefit lies in that a user is provided with an intuitive manner of adjusting the precision of his input based on his movements and the computing device's sensibility.
Yet another benefit lies in that a user is enabled to use a user space that is larger than the space in front of the display, thereby increasing the usability of the devices.
The invention has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1350066-5 | Jan 2013 | SE | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/SE2014/050069 | 1/22/2014 | WO | 00 |
