Patent Application
20110161892
The present invention relates generally to a device and a method for providing feedback via a display to the user of an electronic device, and more particularly, to providing visual feedback of receipt of a gesture, in instances where a function associated with the gesture has reached a limit which would preclude execution of the function.
Touch sensitive surfaces are increasingly becoming popular as interfaces for portable electronic devices, where the general trend for many devices is for an overall reduction in size. One of the challenges with reducing the overall size is the fact that there is a desire for the overall size reductions to take place without compromising or reducing the size of some of the user interface elements, such as screen size. In such instances, overall reductions in size without a corresponding reduction in some of the interface elements can only occur if some of the surface space of a device can be shared or can support multiple functions. For example, in some instances previously distinct portions of the surface of the device that separately supported user input or user output, have been merged such that the same surface that receives input from the user can also convey information or output to a user.
The use of touch sensitive surfaces or alternative forms of user input through gesturing has allowed more of the surface space to be used for supporting a larger display. In the case of touch sensitive surfaces, the interface surface can be configured to share the same space as the display. Such a sharing allows the user to interact with elements forming parts of a currently displayed image, which can be readily changed to accommodate different types of functions to be performed by the device. Such flexibility readily enables customized interfaces through a different displayed image, which enables the interface to better map to the current function intended to be performed by the device.
With the greater emergence of touch sensitive surfaces, gestures have emerged as an increasingly used interaction for purposes of interfacing with the displayed items and/or touch sensitive zones. In at least some cases, the different possible gestures can be received through the same shared surface space, where the particular movement associated with a particular gesture is distinguished from other different movements or different sequence of movements associated with a different particular gesture, which is applied to the same surface. In at least some instances the detected gesture can be relatively intuitive, where the interaction of the user through a pointer relative to the screen mimics the desired effect. For example, a sliding gesture along the surface could be used to indicate a desire to pan an image, or scroll through a list of items, where the image extends beyond the boundaries of the screen, or where the currently displayed elements from the list represents only a portion of the elements contained in the list. In such an instance, the touch sensitive surface tracks the movement of the pointer, such as a stylus or the user's finger at different points in time in order to determine the overall movement and/or traced pattern. The overall movement is then mapped to one of potentially multiple different predefined gestures, which upon detection can be used to invoke a corresponding function.
However there are times when it may be difficult to discern whether the intended gesture is being properly detected. With at least some forms of user input, the user will receive feedback as part of the user interaction. Many button types will provide a vibration, or will have a built in mechanical deformation that can provide the user a form of tactile feedback, which can be perceived by the user upon a successful actuation of the element. For example, the compression of a popple, which might be included in the overall structure of an actuatable button, can often be felt and/or heard by the user. In other instances, the device will actuate an audio and/or vibrational device upon the detection of a successful actuation of a user interface element. In still further instances, the feedback can be in the form of the execution of the intended function.
Because a gesture can be comprised of a sequence of multiple interactions, which might trace a discernable pattern, and because portions of a gesture can be reused as a part of other gestures, any feedback in the form of a sound or vibration which may be triggered as a part of a user interaction with the device may be the result of the detection of a portion of a gesture and not the complete intended gesture. As such, it may be unclear as to whether detected interaction represents the detection of a complete gesture or just a portion of the overall intended gesture. In other words, even though an interaction might trigger a form of feedback to the user, it may not always be clear whether the feedback represents the detection of the complete gesture corresponding to an intended function, or something less than the complete intended gesture, which might be mapped to an alternative unintended function. In other instances, there may not be any purposeful feedback relative to the successful detection of a gesture other than the performance of the associated function.
However in some cases the device may not be able to perform the requested function, even if the associated gesture was successfully detected by the device. Furthermore, it may not be readily apparent to the user that function can not be performed. Such an instance may occur where a limit has been reached relative to a requested function. For example, where the edge of a displayed image already coincides with the edge of the display, such as where there is no more image in a particular direction. Any further attempts to scroll the displayed image in that direction may not be possible. In such an instance, it may not be clear whether the gesture corresponding to the intended function has been properly detected. Alternatively, the user might assume that the lack of an immediate response may be due to a delay in the execution of the function due to a slow user interface and/or a slow processor or heavy processor load.
Correspondingly, the present inventors have recognized that it would be beneficial to provide an indication that the gesture associated with the desired function has been properly detected and that the device is unable to perform the function as requested, such as due to the interface being at a boundary condition which precludes the function being performed in the requested direction and/or as expected by the user.
The present invention provides a method for presenting to a user of an electronic device via a display screen of the electronic device visual feedback of a user interaction, when the device is at a limit of a requested function. The method includes detecting a gesture including a user interaction with the electronic device. The gesture is then associated with a function. A determination is then made as to whether the associated function has reached a limit which would preclude execution of the associated function. If the associated function has not reached the limit which would preclude execution of the associated function, then executing the function associated with the detected gesture. If the associated function has reached the limit which would preclude execution of the associated function, then producing an image distortion proximate the user interaction.
In at least one embodiment, the user interaction with the device includes a movement relative to a touch sensitive surface.
The present invention further provides a display interface for presenting visual feedback of a user interaction with an image being presented via an electronic device. The display interface includes a display adapted for visually presenting to the user at least a portion of an image. The display interface further includes a user input adapted for receiving a gesture including a user interaction with the electronic device. The display interface still further includes a controller. The controller is adapted for associating the gesture with a function, determining whether the associated function has reached a limit which would preclude execution of the associated function, and executing the function associated with the detected gesture. If the associated function has not reached the limit which would preclude execution of the associated function, then controller is adapted for executing the function associated with the detected gesture. If the associated function has reached the limit which would preclude execution of the associated function, then the controller is adapted for producing an image distortion proximate the user interaction.
These and other objects, features, and advantages of this invention are evident from the following description of one or more preferred embodiments of this invention, with reference to the accompanying drawings.
While the present invention is susceptible of embodiment in various forms, there is shown in the drawings and will hereinafter be described presently preferred embodiments with the understanding that the present disclosure is to be considered an exemplification of the invention and is not intended to limit the invention to the specific embodiments illustrated. Furthermore, while the various figures are intended to illustrate the various claimed aspects of the present invention, in doing so, the elements are not necessarily intended to be drawn to scale. In other word, the size, shape and dimensions of some layers, features, components and/or regions for purposes of clarity or for purposes of better describing or illustrating the concepts intended to be conveyed may be exaggerated and/or emphasized relative to other illustrated elements.
In the illustrated embodiment, the electronic device is a hand-held electronic device, which includes a touch sensitive display 102 upon which a pointer, such as a user's finger 104, can trace a pattern 105 and/or pattern 106, corresponding to a gesture or a portion of a gesture, which can be detected by a user input 108, such as a touch or proximity sensor array and can be interpreted as commands or a requested function. In the illustrated embodiment, the sensor array is formed as part of the display assembly, and/or overlays the display screen in order that an interaction with the display surface can be detected by the device.
Generally, the touch or proximity sensor array can employ various types of touch or proximity sensing technologies including capacitive arrays as well as resistive arrays, the touch sensitive arrays can even employ force sensing resistor arrays, for detecting the amount of force being applied at the selected location. In this way, a force threshold determination can be taken into account in determining the intended interaction including the making of a gesture. However while a touch or proximity sensor array is illustrated, one skilled in the art will readily appreciate that other types of user input could alternatively be used to detect the performance by the user of a gesture that can be used to produce an actionable user selection or input. For example, accelerometers and/or tilt sensors could be used to detect the movement of the device in one of one or more predesignated patterns, which might be recognizable as a user input command, and/or might be associated with a function to be executed by the device. Alternatively, a directional pad, mouse, joystick and/or still other forms of inputs could similarly be used to convey a gesture that can be detected as a valid user input.
In some instances a particular controllable interface, such as the user input 108 may be responsive to more than one type of gesture that might produce a related but different effect. For example, a gesture including the repeated writing of a line having a direction and a length might cause a panning or scrolling effect relative to an image being displayed on a display screen. A direction of the line could be used to identify the direction of any associated panning or scrolling. Furthermore, an amount corresponding to the length of the detected line 105 and/or the speed at which the line 105 is traced could be used to adjust a speed and/or the magnitude of the scrolling. While a downward movement 105 could be used to see more of the upper portions of the image, which might currently correspond to a portion of the image not being presently shown on the display screen 102, a movement in the opposite direction 111 (i.e. upward direction) can cause the image to pan or scroll in the opposite direction. Alternatively a movement of a pointer between left and right 106 or 110, can cause the image to be panned or scrolled in a horizontal direction. The tracing of line in a diagonal direction may also be possible for indicating a panning or scrolling in a diagonal direction. Alternatively, the panning or scrolling in a diagonal direction might be facilitated through a combination of both a vertical and a horizontal gesture.
However, it is possible that in some instances there may not be any more of the image to see in the detected direction of the corresponding user gesture. In such an instance the image could be said to have reached its limit relative to the requested function associated with the detected gesture. In such an instance, the display interface in accordance with the present invention will produce an image distortion relative to the direction of the detected gesture and the particular edge of the display at which the image has reached its limit. The resulting visual distortion provides visual feedback to the user that not only has the gesture been detected, but that the execution of the associated function is precluded due to other circumstances, one such circumstance including the image having reached its limit relative to the requested function.
Such a distortion is an alternative to some display interfaces, which in some instances can simply fail to pan or scroll any further in the requested direction. In some instances it may be unclear whether the lack of further panning or scrolling may be the result of the image having reached its limit, or whether the lack of further panning or scrolling may be the result of having failed to properly detect the corresponding gesture. In other instances, the lack of an immediate response may be perceived as a user interface delay, where such delays may be consistent with communication delays associated with retrieving the additional image data via a network connection, such as the Internet, or where other concurrent processor dependent tasks may be loading the processor and correspondingly delaying the update of the image being displayed on the display screen. A further example of a function that has reached its limit includes the display of a list of elements, where the currently displayed portion of the list includes the elements at one of the ends of the list, where there is no further data in the form of elements in the list, which are not already being displayed in the direction that the scrolling or panning has been requested. A still further example of a function that has reached its limit includes a zoom function where the current interface has reached a limit as to whether further zooming in the desired direction is possible or allowed.
The storage element 216 can include one or more forms of volatile and/or non-volatile memory, including conventional ROM, EPROM, RAM, or EEPROM. The storage element 216 may still further incorporate one or more forms of auxiliary storage, which is either fixed or removable, such as a harddrive or a floppydrive. One skilled in the art will still further appreciate, that still other further forms of memory could be used without departing from the teachings of the present invention. In the same or other instances, the controller 206 may additionally or alternatively incorporate state machines and/or logic circuitry, which can be used to implement at least partially, some of modules and their corresponding functionality.
In the illustrated embodiment, the gesture detection module 210 of the controller is adapted to compare a received gesture with one of a plurality of predefined gestures including a plurality of gestures, which are intended to signal a desire to scroll or pan through the image, or group of items, which are arranged in a sequence, such as a menu, and which can be selected by a user. Further gestures may be associated with a desire to zoom in or out relative to the image or items being currently displayed. Upon detection of the particular gesture and the associated function, the limit detection module 211 compares the current display status of the image or the list, and determines whether the function associated with the detected gesture can be performed, or whether the associated function has reached a limit that would preclude its execution.
Where the associated function has not reached the limit which would preclude execution of the associated function, the image display/distortion module 212 then executes the function associated with the detected gesture. Where the associated function has reached the limit which would preclude execution of the associated function, the image display/distortion module 212 causes to be displayed a display image that includes a distortion proximate the user interaction. One such example of an image distortion 300 is illustrated in
As illustrated in
Such a distortion can be understood by the user as being indicative that the function associated with a particular gesture has been properly detected, but that because the associated function is already at its limit relative to the displayed image, that further execution of the associated function is precluded. The illustrated distortion is intended to go away or snap back, when the point of user interaction 402 is released, through a disengagement of the pointer relative to the user input, such as the touch sensitive surface 108, see
Such a distortion is similarly possible in instances where a gesture might have multiple points of user interaction. Both
Alternatively,
In each of the above noted examples, the image distortion serves to confirm to the user that both the gesture was detected, and that associated function has reached one or more limits which precludes the execution of the function as expected. In this way, the user is not left wondering the reason that the device has not yet or might have failed to execute the function as expected. The visual feedback even in instances where execution of the function is precluded conveys more detailed information to the user in a relatively unintrusive manner.
While the preferred embodiments of the invention have been illustrated and described, it is to be understood that the invention is not so limited. Numerous modifications, changes, variations, substitutions and equivalents will occur to those skilled in the art without departing from the spirit and scope of the present invention as defined by the appended claims.
| Number | Date | Country | |
|---|---|---|---|
| 61290766 | Dec 2009 | US |
