1. Field of the Invention
The present invention relates to systems and methods for multitouch device user interfaces. More particularly, the present invention relates to a scrollbar user interface for multitouch devices.
2. Related Art
Multitouch input technologies provide opportunities for user interface design that provide sensing capabilities for locations of user contact with a display, such as with a finger or stylus. By sensing user contact locations, user interface modules may detect sliding or dragging operations initiated by the user based upon changes to the contact locations. As such, user interface applications have been designed that favor user interactions that mimic physically-oriented activities. For example, user interfaces have been designed that allow a user to flip through a series of album covers by repeatedly sliding a single finger across a display surface and that allow a user to drag a web page up and down by sliding a single finger across the display surface.
The subject matter described herein provides a scrollbar user interface for multitouch devices. A graphical user interface (GUI) for a multitouch input display device allows a user to manipulate and mark content by performing coincident sliding actions on a display surface that resemble physical manipulations (e.g., pinching, stretching, flicking, twisting, etc.) of a scroll element. The scroll element may be a scrollbar element or other content control element. Two sliding user interactions, such as with a finger and thumb, are detected along the display surface in approximately opposite directions (e.g., pinching or stretching) near a display location of a scroll element and content is automatically zoomed within the GUI. Content is scrolled in response to detecting two user interactions in approximately similar directions near the display location of the scroll element. The scroll element may be adjusted in proportion to the automated zoom or scroll operations. Content may be marked in response to detecting flicking and twisting motions on the display surface. Accordingly, the present subject matter provides an intuitive interface for automated resizing and scrolling of content for multitouch devices.
A method includes detecting two sliding user interactions along a display surface of a multitouch input display device in approximately opposite directions proximate to a display location of a scroll element displayed on a graphical user interface (GUI) of the device; and automatically zooming available content displayed within the GUI based upon the detected two sliding user interactions along the display surface.
A system includes a multitouch input display device adapted to display visual information to a user and provide user input via user contact on a display surface; and a processor programmed to: detect two sliding user interactions along the display surface of the multitouch input display device in approximately opposite directions proximate to a display location of a scroll element displayed on a graphical user interface (GUI) of the device; and automatically zoom available content displayed within the GUI based upon the detected two sliding user interactions along the display surface.
An alternative system includes a multitouch input display device adapted to display visual information to a user and provide user input via user contact on a display surface; and a processor programmed to: detect that the user has contacted the display surface of the multitouch input display device at first and second user contact locations that each approximate a display location for one of first and second edges of a displayed scroll element respectively, displayed within a graphical user interface (GUI); detect a first and second sliding movement in approximately opposite directions, associated with the first and second user contact locations respectively, along the display surface to third and fourth user contact locations; calculate a first spacing distance between the first and second user contact locations; calculate a second spacing distance between the third and fourth user contact locations; calculate a proportional difference between the first spacing distance and the second spacing distance relative to a quantity of the available content; automatically adjust a displayed portion of available content displayed within the GUI based upon the calculated proportional difference; and automatically adjust a size of the displayed scroll element in proportion to the automated adjustment of the displayed portion of the available content displayed within the GUI.
A computer program product includes a computer useable storage medium including a computer readable program. The computer readable program when executed on a computer causes the computer to detect two sliding user interactions along a display surface of a multitouch input display device in approximately opposite directions proximate to a display location of a scroll element displayed on a graphical user interface (GUI) of the device; and automatically zoom available content displayed within the GUI based upon the detected two sliding user interactions along the display surface.
Those skilled in the art will appreciate the scope of the present invention and realize additional aspects thereof after reading the following detailed description of the preferred embodiments in association with the accompanying drawing figures.
The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the invention, and together with the description serve to explain the principles of the invention.
The examples set forth below represent the necessary information to enable those skilled in the art to practice the invention and illustrate the best mode of practicing the invention. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the invention and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure and the accompanying claims.
The subject matter described herein provides a scrollbar user interface for multitouch devices. A graphical user interface (GUI) for a multitouch input display device allows a user to manipulate and mark content by performing coincident sliding actions on a display surface that resemble physical manipulations (e.g., pinching, stretching, flicking, twisting, etc.) of a scroll element. The scroll element may be a scrollbar element or other content control element. Two sliding user interactions, such as with a finger and thumb, are detected along the display surface in approximately opposite directions (e.g., pinching or stretching) near a display location of a scroll element and content is automatically zoomed within the GUI. Content is scrolled in response to detecting two user interactions in approximately similar directions near the display location of the scroll element. The scroll element may be adjusted in proportion to the automated zoom or scroll operations. Content may be marked in response to detecting flicking and twisting motions on the display surface. Accordingly, the present subject matter provides an intuitive interface for automated resizing and scrolling of content for multitouch devices.
The scrollbar user interface for multitouch devices described herein may be performed in real time to allow prompt response to content manipulation operations. For purposes of the present description, real time shall include any time frame of sufficiently short duration as to provide reasonable response time for information processing acceptable to a user of the subject matter described. Additionally, the term “real time” shall include what is commonly termed “near real time”—generally meaning any time frame of sufficiently short duration as to provide reasonable response time for on-demand information processing acceptable to a user of the subject matter described (e.g., within a portion of a second or within a few seconds). These terms, while difficult to precisely define are well understood by those skilled in the art.
As will be described in more detail below, the multitouch input display device 102 allows a user of the consumer electronics device 100 to manipulate content displayed within a graphical user interface (GUI) 110 by sliding the user's index finger 112 and thumb 114 across a display surface of the multitouch input display device 102. It should be noted that while the present examples illustrate use of the user's index finger 112 and thumb 114 for interacting with the multitouch input display device 102, this should not be considered limiting. For example, the user may use an index finger from each hand, two stylus pens, or any other contact mechanisms without departure from the scope of present subject matter.
Within the GUI 110, a scrollbar 116 is illustrated. Within the scrollbar 116 a scroll element 118 is depicted. As described above and in more detail below, the user interacts with the GUI 110 by sliding the index finger 112 and the thumb 114 in various directions relative to one another across the surface of the multitouch input display device 102. For purposes of the present subject matter, an x-axis 120 and a y-axis 122 are illustrated, with the y-axis 122 axially-oriented relative to a length of the scrollbar 116. The x-axis 120 is illustrated perpendicular to y-axis 122. Movements of the user's index finger 112 and thumb 114 are defined herein to be axially-oriented if they move in a generally axial direction relative to the y-axis 122. Likewise, movements of the user's index finger 112 and thumb 114 will be considered laterally-oriented if they move in a generally axial direction relative to the x-axis 120.
Using the index finger 112 as an example, a region defined by the arced arrow A and the arced arrow B illustrate regions relative to the y-axis 122 that are considered to be axial regions for movement of the index finger 112. As such, if the user moves the index finger 112 vertically upward relative to the illustration within
A similar coordinate system is oriented relative to the user's thumb 114. However, for ease of illustration purposes, the coordinate system associated with the thumb 114 is not illustrated within
As will be described in more detail below, when a user contacts the display surface of the multitouch input display device 102, the consumer electronics device 100 detects this contact. If the contact is in a location approximating an axial end of the scroll element 118, the consumer electronics device 100 detects this contact location and orients a coordinate system as described above at the location of the contact on the display surface. When a second contact is detected that is in a location approximating a display location for an opposite axial end of the scroll element 118, the consumer electronics device 100 detects this contact and orients a coordinate system as described above at the second contact location.
As the user slides the index finger 112 in any direction from the initial contact location of the index finger 112, as defined by the arced arrows A through D, the consumer electronics device 100 detects this movement and performs an action upon content displayed within the GUI 110 based upon this detected movement. Likewise, as the user slides the thumb 114 in any direction from the initial contact location for the thumb 114, as defined by the arced arrows A through D, the consumer electronics device 100 detects this movement and performs an action upon content displayed within the GUI 110 based upon this detected movement.
Furthermore, the user controls manipulation of content displayed within the GUI 110 by sliding the index finger 112 and the thumb 114 across the display surface of the multitouch input display device 102 relative to current display locations of the axial ends of the scroll element 118. For example, by performing a pinching action across the display surface such that the index finger 112 is slid downward (e.g., into the region defined by the arced arrow B) and the thumb 114 is slid upward (e.g., into the region defined by its relative arced arrow A) within their respective coordinate systems, the user indicates a desire to zoom in on the content displayed within the GUI 110 to show less of the content at a higher resolution. Similarly, by performing a stretching operation on the scroll element 118 by sliding the index finger 112 upward (e.g., into the region defined by the arced arrow A) and the thumb 114 downward (e.g., into the region defined by its relative arced arrow B) within their respective coordinate systems, the user indicates a desire to zoom the content displayed within the GUI 110 outward to show more of the available content for display at a lower resolution. For purposes of the present description, the above-described pinching and stretching motions are recognized to be axially-oriented motions in opposite directions.
As another example, if a user wishes to mark a row or item of content displayed within the GUI 110, the user may perform a lateral sliding motion with either the index finger 112 or the thumb 114 as defined by the regions referenced by the arced arrow C and the arced arrow D to cause the consumer electronics device 100 to mark a portion of the content. After making an initial mark associated with the content displayed within the GUI 110, the user may again initiate movements that cause the consumer electronics device 100 to zoom the content as described above or may initiate movements that cause the consumer electronics device 100 to scroll the content by sliding the index finger 112 and the thumb 114 in similar axial directions as defined by the regions illustrated with the arced arrow A and the arced arrow B. The user may then perform another lateral sliding motion with either the index finger 112 or the thumb 114 to cause the consumer electronics device 100 to mark a second location within the content displayed within the GUI 110.
Additionally, a region of the content displayed within the GUI 110 may be marked and defined by a region of the content between the first and second marking operations. The user may also initiate movements that cause the consumer electronics device 100 to perform a zoom operation as described above prior to or coincident with the second marking operation. As such, the user is provided with a fluid way to manipulate content displayed within the GUI 110 and an intuitive user interface mechanism. It should be noted that as the user performs the pinching and stretching movements described above, the consumer electronics device 100 detects these movements and adjusts a displayed length of the scroll element 118 within the scrollbar 116 based upon relative distances of the sliding movements of the index finger 112 and the thumb 114 and the available content for display. Accordingly, in addition to zooming the content displayed within the GUI 110, the consumer electronics device 100 adjusts the length of the scroll element 118 to provide the user with feedback representing the proportion of the content that is available for display relative to that which is presently displayed within the GUI 110.
As will be described in more detail below in association with
The display 202 may include any display device, such as a cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED), projection, touchscreen, or other display element or panel. The multitouch input processing module 204 may include any form of multitouch input processing module capable of sensing user contacts and interactions with a surface of the display 202, such as by capacitive sensing or other suitable technology, by which the user may interact with and respond to information on the display 202.
A communication module 206 provides interconnection capabilities that allow the consumer electronics device 100 to communicate with other modules (not shown). For example, the consumer electronics device 100 may communicate with content servers (e.g., web servers) and communication networks (e.g., wireless and wired networks). The communication module 206 may include any electrical, protocol, and protocol conversion capabilities useable to provide the interconnection capabilities. Though the communication module 206 is illustrated as a component-level module for ease of illustration and description purposes, it should be noted that the communication module 206 includes any hardware, programmed processor(s), and memory used to carry out the functions of the communication module 206 as described above and in more detail below. For example, the communication module 206 may include additional controller circuitry in the form of application specific integrated circuits (ASICs), processors, antennas, and/or discrete integrated circuits and components for performing communication and electrical control activities associated with the communication module 206. Additionally, the communication module 206 also includes interrupt-level, stack-level, and application-level modules as appropriate. Furthermore, the communication module 206 includes any memory components used for storage, execution, and data processing for performing processing activities associated with the communication module 206. The communication module 206 may also form a portion of other circuitry described without departure from the scope of the present subject matter.
A memory 208 provides storage capabilities for the consumer electronics device 100. It is understood that the memory 208 may include any combination of volatile and non-volatile memory suitable for the intended purpose, distributed or localized as appropriate, and may include other memory segments not illustrated within the present example for ease of illustration purposes. For example, the memory 208 may include a code storage area, a code execution area, and a data area without departure from the scope of the present subject matter.
The microphone 106 and the speaker 108 are also depicted to illustrate additional input and output interconnections for the consumer electronics device 100, respectively. The CPU 200, the display 202 and the multitouch input processing module 204 of the multitouch input display device 102, the communication module 206, the memory 208, the microphone 106, and the speaker 108 are interconnected via an interconnection 210. The interconnection 210 may include a system bus, a network, or any other interconnection capable of providing the respective components with suitable interconnection for the respective purpose.
While the consumer electronics device 100 is illustrated with and has certain components described, other modules and components may be associated with the consumer electronics device 100 without departure from the scope of the present subject matter. Additionally, it should be noted that, while the consumer electronics device 100 is described as a single device for ease of illustration purposes, the components within the consumer electronics device 100 may be co-located or distributed and interconnected via a network without departure from the scope of the present subject matter. For a distributed arrangement, the display 202 and the multitouch input processing module 204 of the multitouch input display device 102 may be located at a point of sale device, kiosk, or other location, while the CPU 200 and memory 208 may be located at a local or remote server. Many other possible arrangements for components of the consumer electronics device 100 are possible and all are considered within the scope of the present subject matter. Accordingly, the consumer electronics device 100 may take many forms and may be associated with many platforms.
As can be seen from
Returning to the description of
For purposes of the present example, as described above, the sequence of images within the row 308 that are cross-hatched are not the graduation images. Accordingly, it assumed that the user performs a stretching operation with the index finger 112 and the thumb 114 as represented by the arrows 402 and 404, respectively, and that these sliding movements in approximately opposite directions are detected by the consumer electronics device 100 and the content displayed within the GUI 110 is processed in response to detecting these sliding movements. Additionally, though not depicted within
For purposes of the present example, it is assumed that the current location at which the content is marked within the GUI 110 is the first row 700 illustrated within
Accordingly, as described above in association with
When a determination is made at decision point 1204 that the two contact locations are not proximate to a display location of the scroll element 118 displayed on the GUI 110, the process 1200 returns to decision point 1202 to await detection of two subsequent user contact locations. When a determination is made that the two contact locations are proximate to a display location of the scroll element 118 displayed on the GUI 110, the process 1200 calculates a first spacing distance between the two detected contact locations at block 1206.
At decision point 1208, the process 1200 makes a determination as to whether two sliding movements (e.g., user interactions) have been detected along the display surface of the multitouch input display device 102. As discussed above, for purposes of the present example, timeout procedures and other error control procedures are not illustrated for ease of illustration purposes. As such, it is assumed that the process 1200 provides sufficient time at each stage of processing for the user to interact with the display surface of the multitouch input display device 102. Accordingly, when a determination is made at decision point 1208 that two sliding movements have been detected along the display surface of the multitouch input display device 102, the process 1200 detects resulting contact locations associated with each sliding movement and calculates a second spacing distance for each of the two resulting contact locations at block 1210.
At decision point 1212, the process 1200 makes a determination as to whether the two sliding movements were in opposite directions based upon the calculated second spacing distance relative to the first spacing distance.
When a determination is made that the two sliding movements were in opposite directions, the process 1200 calculates a proportional difference between the first spacing distance and the second spacing distance relative to a quantity of the available content at block 1214. At block 1216, the process 1200 automatically adjusts a displayed portion of the available content to zoom the content based upon the calculated proportional difference. For example, if the two resulting contact locations are closer together, the content may be adjusted to effect a zoom action inward to enlarge a portion of the content. In contrast, if the two resulting contact locations are farther apart, the content may be adjusted to effect a zoom action outward to display more content at a lesser resolution. At block 1218, the process 1200 automatically adjusts a size of the scroll element 118 in proportion to the automated zooming of the available content displayed within the GUI 110 and returns to decision point 1202 to await detection of two new user contacts on the display surface of the multitouch input display device 102.
Returning to the description of decision point 1210, when a determination is made that the two sliding movements were not in opposite directions (e.g., were in similar directions), the process 1200 calculates relative positions between the initial contact locations and the resulting contact locations at block 1220. At block 1222, the process 1200 scrolls content displayed within the GUI 110 in proportion to the sliding movements and the calculated relative positions. The process 1200 continues to block 1214 and continues processing as described above to determine whether a zoom operation is to be performed coincident with the scroll operation based upon proportional differences between the initial and resulting user contact locations.
Returning to the description of decision point 1208, when a determination is made that two sliding movements were not detected, a determination is made at decision point 1224 as to whether a single lateral movement was detected. As described above, a detected lateral movement may be used to mark content displayed within the GUI 110. When a determination is made that a lateral movement has not been detected, the process 1200 returns to decision point 1202 to continue processing as described above. When a determination is made that a lateral movement has been detected, the process 1200 sets a first marker within the available content at a first focus location associated with the available content displayed within the GUI 110 at block 1226. At decision point 1228, the process 1200 waits for a second lateral movement to be detected. When a second lateral movement is detected, the process 1200 sets a second marker within the available content at a second focus location associated with the available content displayed within the GUI 110 at block 1230. At block 1232, the process 1200 highlights a portion of the available content between the first marker and the second marker for display and returns to decision point 1202 to continue processing as described above.
Accordingly, the process 1200 provides automated zooming, scrolling, and marking of content displayed within the GUI 110 in response to detecting sliding movements along a display surface of the multitouch input display device 102 in one of opposite directions, similar directions, and lateral directions, respectively. The process may be modified to provide additional behavior associated with the described activities and additional processes may be derived from the process 1200 without departure from the scope of the present subject matter.
As described above in association with
Those skilled in the art will recognize, upon consideration of the above teachings, that certain of the above examples are based upon use of a programmed processor such as CPU 200. However, the invention is not limited to such exemplary embodiments, since other embodiments could be implemented using hardware component equivalents such as special purpose hardware and/or dedicated processors. Similarly, general purpose computers, microprocessor based computers, micro-controllers, optical computers, analog computers, dedicated processors, application specific circuits and/or dedicated hard wired logic may be used to construct alternative equivalent embodiments.
As will be appreciated by one skilled in the art, the present invention may be embodied as a system, method or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer-usable program code embodied in the medium.
Any combination of one or more computer usable or computer readable storage medium(s) may be utilized. The computer-usable or computer-readable storage medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a transmission media such as those supporting the Internet or an intranet, or a magnetic storage device. Note that the computer-usable or computer-readable storage medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In the context of this document, a computer-usable or computer-readable storage medium may be any medium that can contain, store, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to example embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible example implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers.
Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems and Ethernet cards are just a few of the currently available types of network adapters.
Those skilled in the art will recognize improvements and modifications to the preferred embodiments of the present invention. All such improvements and modifications are considered within the scope of the concepts disclosed herein and the claims that follow.
Number | Name | Date | Kind |
---|---|---|---|
5553225 | Perry | Sep 1996 | A |
5880722 | Brewer et al. | Mar 1999 | A |
7380216 | Feig et al. | May 2008 | B2 |
20060026535 | Hotelling et al. | Feb 2006 | A1 |
20080034316 | Thoresson | Feb 2008 | A1 |
20080165141 | Christie | Jul 2008 | A1 |
20100070913 | Murrett et al. | Mar 2010 | A1 |
Number | Date | Country | |
---|---|---|---|
20100251167 A1 | Sep 2010 | US |