Patent Application
20140267130
Touch screens have had enormous growth in recent years. Touch screens are now common in places such as kiosks at airports, automatic teller machines (ATMs), vending machines, computers, mobile phones, etc.
The touch screens typically provide a user with a plurality of options through icons, and the user can select those icons to launch an application or obtain additional information associated with the icon. If the result of that selection did not provide the user with the desired result, then he/she must select a “back” button or “home” button or otherwise back out of the application or information. Such unnecessary reviewing of information costs the user time. Additionally, for mobile phone users, battery life is unnecessarily wasted.
Additionally, the library of touch gestures is limited. Well-known gestures include a flick, pan, pinch, etc., but new gestures have not been developed, which limits the functionality of a mobile device.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Various embodiments herein provide for a method of receiving user input on a touch screen. A hover gesture can be detected and an action performed in response to the detection. The hover gesture can occur without a user physically touching a touch screen. Instead, the user's finger or fingers can be positioned at a spaced distance above the touch screen. The touch screen can detect that the user's fingers are proximate to the touch screen, such as through capacitive sensing. Additionally, finger movement can be detected while the fingers are hovering to expand the existing options for gesture input.
The foregoing and other objects, features, and advantages of the invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.
Embodiments described herein focus on a mobile device, such as a mobile phone. However, the described embodiments can be applied to any device with a touch screen, including laptop computers, tablets, desktop computers, televisions, etc.
Hover Touch is built into the touch framework to detect a finger above-screen as well as to track finger movement. A gesture engine can be used for the recognition of hover touch gestures, including: (1) finger hover pan—float a finger above the screen and pan the finger in any direction; (2) finger hover tickle/flick—float a finger above the screen and quickly flick the finger as like a tickling motion with the finger; (3) finger hover circle—float a finger or thumb above the screen and draw a circle or counter-circle in the air; (4) finger hover hold—float a finger above the screen and keep the finger stationary; (5) palm swipe—float the edge of the hand or the palm of the hand and swipe across the screen; (6) air pinch/lift/drop—use the thumb and pointing finger to do a pinch gesture above the screen, drag, then a release motion; (7) hand wave gesture—float hand above the screen and move the hand back and forth in a hand-waving motion.
The hover gesture relates to a user-input command wherein the user's hand (e.g., one or more fingers, palm, etc.) is a spaced distance from the touch screen meaning that the user is not in contact with the touch screen. Moreover, the user's hand should be within a close range to the touch screen, such as between 0.1 to 0.25 inches, or between 0.25 inches and 0.5 inches, or between 0.5 inches and 0.75 inches or between 0.75 inches and 1 inch, or between 1 inch and 1.5 inches, etc. Any desired distance can be used, but generally such a distance can be less than 2 inches.
A variety of ranges can be used. The sensing of a user's hand can be based on capacitive sensing, but other techniques can be used, such as an ultrasonic distance sensor or camera-based sensing (images taken of user's hand to obtain distance and movement).
Once a hover touch gesture is recognized, certain actions can result, as further described below. Allowing for hover recognition significantly expands the library of available gestures to implement on a touch screen device.
The illustrated mobile device 100 can include a controller or processor 110 (e.g., signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, input/output processing, power control, and/or other functions. An operating system 112 can control the allocation and usage of the components 102 and support for one or more application programs 114. The application programs can include common mobile computing applications (e.g., email applications, calendars, contact managers, web browsers, messaging applications), or any other computing application.
The illustrated mobile device 100 can include memory 120. Memory 120 can include non-removable memory 122 and/or removable memory 124. The non-removable memory 122 can include RAM, ROM, flash memory, a hard disk, or other well-known memory storage technologies. The removable memory 124 can include flash memory or a Subscriber Identity Module (SIM) card, which is well known in GSM communication systems, or other well-known memory storage technologies, such as “smart cards.” The memory 120 can be used for storing data and/or code for running the operating system 112 and the applications 114. Example data can include web pages, text, images, sound files, video data, or other data sets to be sent to and/or received from one or more network servers or other devices via one or more wired or wireless networks. The memory 120 can be used to store a subscriber identifier, such as an International Mobile Subscriber Identity (IMSI), and an equipment identifier, such as an International Mobile Equipment Identifier (IMEI). Such identifiers can be transmitted to a network server to identify users and equipment.
The mobile device 100 can support one or more input devices 130, such as a touchscreen 132, microphone 134, camera 136, physical keyboard 138 and/or trackball 140 and one or more output devices 150, such as a speaker 152 and a display 154. Touchscreens, such as touchscreen 132, can detect input in different ways. For example, capacitive touchscreens detect touch input when an object (e.g., a fingertip) distorts or interrupts an electrical current running across the surface. As another example, touchscreens can use optical sensors to detect touch input when beams from the optical sensors are interrupted. Physical contact with the surface of the screen is not necessary for input to be detected by some touchscreens. For example, the touchscreen 132 can support a finger hover detection using capacitive sensing, as is well understood in the art. Other detection techniques can be used, as already described above, including camera-based detection and ultrasonic-based detection. To implement a finger hover, a user's finger is typically within a predetermined spaced distance above the touch screen, such as between 0.1 to 0.25 inches, or between .0.25 inches and 0.05 inches, or between .0.5 inches and 0.75 inches or between 0.75 inches and 1 inch, or between 1 inch and 1.5 inches, etc.
Other possible output devices (not shown) can include piezoelectric or other haptic output devices. Some devices can serve more than one input/output function. For example, touchscreen 132 and display 154 can be combined in a single input/output device. The input devices 130 can include a Natural User Interface (NUI). An NUI is any interface technology that enables a user to interact with a device in a “natural” manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls, and the like. Examples of NUI methods include those relying on speech recognition, touch and stylus recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence. Other examples of a NUI include motion gesture detection using accelerometers/gyroscopes, facial recognition, 3D displays, head, eye, and gaze tracking, immersive augmented reality and virtual reality systems, all of which provide a more natural interface, as well as technologies for sensing brain activity using electric field sensing electrodes (EEG and related methods). Thus, in one specific example, the operating system 112 or applications 114 can comprise speech-recognition software as part of a voice user interface that allows a user to operate the device 100 via voice commands. Further, the device 100 can comprise input devices and software that allows for user interaction via a user's spatial gestures, such as detecting and interpreting gestures to provide input to a gaming application.
A wireless modem 160 can be coupled to an antenna (not shown) and can support two-way communications between the processor 110 and external devices, as is well understood in the art. The modem 160 is shown generically and can include a cellular modem for communicating with the mobile communication network 104 and/or other radio-based modems (e.g., Bluetooth 164 or Wi-Fi 162). The wireless modem 160 is typically configured for communication with one or more cellular networks, such as a GSM network for data and voice communications within a single cellular network, between cellular networks, or between the mobile device and a public switched telephone network (PSTN).
The mobile device can further include at least one input/output port 180, a power supply 182, a satellite navigation system receiver 184, such as a Global Positioning System (GPS) receiver, an accelerometer 186, and/or a physical connector 190, which can be a USB port, IEEE 1394 (FireWire) port, and/or RS-232 port. The illustrated components 102 are not required or all-inclusive, as any components can be deleted and other components can be added.
Once the gesture engine interprets the gesture, the gesture engine 212 can alert an operating system 214 of the received gesture. In response, the operating system 214 can perform some action and display the results using a rendering engine 216.
Other example applications of the hover gesture can include having UI elements appear in response to the hover gesture, similar to a mouse-over user input. Thus, menu options can appear, related contextual data surfaced, etc. In another example, in a multi-tab application, a user can navigate between tabs using a hover gesture, such as swiping his or her hand. Other examples include focusing on an object using a camera in response to a hover gesture, or bringing camera options onto the UI (e.g., flash, video mode, lenses, etc.) The hover command can also be applied above capacitive buttons to perform different functions, such as switching tasks. For example, if a user hovers over a back capacitive button, the operating system can switch to a task switching view. The hover gesture can also be used to move between active phone conversations or bring up controls (fast forward, rewind, etc.) when playing a movie or music. In still other examples, a user can air swipe using an open palm hover gesture to navigate between open tabs, such as in a browser application. In still other examples, a user can hover over an entity (name, place, day, number, etc.) to surface the appropriate content inline, such as displaying addition information inline within an email. Still further, in a list view of multiple emails, a hover gesture can be used to display additional information about a particular email in the list. Further, in email list mode, a user can perform a gesture to delete the email or display different action buttons (forward, reply, delete). Still further, a hover gesture can be used to display further information in a text message, such as emoji in a text message. In messaging, hover gestures, such as air swipes can be used to navigate between active conversations, or preview more lines of a thread. In videos or music, hover gestures can be used to drag sliders to skip to a desired point, pause, play, navigate, etc. In terms of phone calls, hover gestures can be used to display a dialog box to text a sender, or hover over an “ignore” button to send a reminder to call back. Additionally, a hover command can be used to place a call on silent. Still further, a user can perform a hover gesture to navigate through photos in a photo gallery. Hover commands can also be used to modify a keyboard, such as changing a mobile device between left-handed and right-handed keyboards. As previously described, hover gestures can also be used to see additional information in relation to an icon.
With reference to
A computing system may have additional features. For example, the computing environment 1900 includes storage 1940, one or more input devices 1950, one or more output devices 1960, and one or more communication connections 1970. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing environment 1900. Typically, operating system software (not shown) provides an operating environment for other software executing in the computing environment 1900, and coordinates activities of the components of the computing environment 1900.
The tangible storage 1940 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other medium which can be used to store information which can be accessed within the computing environment 1900. The storage 1940 stores instructions for the software 1980 implementing one or more innovations described herein.
The input device(s) 1950 may be a touch input device such as a touchscreen, keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing environment 1900. For video encoding, the input device(s) 1950 may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or a CD-ROM or CD-RW that reads video samples into the computing environment 1900. The output device(s) 1960 may be a display, printer, speaker, CD-writer, or another device that provides output from the computing environment 1900.
The communication connection(s) 1970 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can use an electrical, optical, RF, or other carrier.
Although the operations of some of the disclosed methods are described in a particular, sequential order for convenient presentation, it should be understood that this manner of description encompasses rearrangement, unless a particular ordering is required by specific language set forth below. For example, operations described sequentially may in some cases be rearranged or performed concurrently. Moreover, for the sake of simplicity, the attached figures may not show the various ways in which the disclosed methods can be used in conjunction with other methods.
Any of the disclosed methods can be implemented as computer-executable instructions stored on one or more computer-readable storage media (e.g., non-transitory computer-readable media, such as one or more optical media discs, volatile memory components (such as DRAM or SRAM), or nonvolatile memory components (such as flash memory or hard drives)) and executed on a computer (e.g., any commercially available computer, including smart phones or other mobile devices that include computing hardware). As should be readily understood, the term computer-readable storage media does not include communication connections, such as modulated data signals. Any of the computer-executable instructions for implementing the disclosed techniques as well as any data created and used during implementation of the disclosed embodiments can be stored on one or more computer-readable media (e.g., non-transitory computer-readable media, which excludes propagated signals). The computer-executable instructions can be part of, for example, a dedicated software application or a software application that is accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software can be executed, for example, on a single local computer (e.g., any suitable commercially available computer) or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a client-server network (such as a cloud computing network), or other such network) using one or more network computers.
For clarity, only certain selected aspects of the software-based implementations are described. Other details that are well known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any specific computer language or program. For instance, the disclosed technology can be implemented by software written in C++, Java, Perl, JavaScript, Adobe Flash, or any other suitable programming language. Likewise, the disclosed technology is not limited to any particular computer or type of hardware. Certain details of suitable computers and hardware are well known and need not be set forth in detail in this disclosure.
It should also be well understood that any functionality described herein can be performed, at least in part, by one or more hardware logic components, instead of software. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
Furthermore, any of the software-based embodiments (comprising, for example, computer-executable instructions for causing a computer to perform any of the disclosed methods) can be uploaded, downloaded, or remotely accessed through a suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.
The disclosed methods, apparatus, and systems should not be construed as limiting in any way. Instead, the present disclosure is directed toward all novel and nonobvious features and aspects of the various disclosed embodiments, alone and in various combinations and subcombinations with one another. The disclosed methods, apparatus, and systems are not limited to any specific aspect or feature or combination thereof, nor do the disclosed embodiments require that any one or more specific advantages be present or problems be solved.
In view of the many possible embodiments to which the principles of the disclosed invention may be applied, it should be recognized that the illustrated embodiments are only preferred examples of the invention and should not be taken as limiting the scope of the invention. Rather, the scope of the invention is defined by the following claims. We therefore claim as our invention all that comes within the scope of these claims.
