Patent Application
20090225042
1. Field of the Invention
This invention relates to systems and methods for enhancing mobile devices and/or devices with touch-sensitive display screens.
2. Description of Related Art
Phones such as the Apple iPhone (Apple Inc. of Cupertino, Calif.) use a multi-touch system for user interface functionality. Multi-touch refers to the device having a touch screen that can sense two or more simultaneous contact inputs. Gestures such as dragging and pinching two fingers together, or widening two fingers apart when dragging on the screen are used to control program functionality. In most devices, however, the touch screen can only sense one contact input at a time. This is inconvenient for programs desirous of the ability to utilize a multi-touch input without a touch screen capable of receiving multiple touch inputs. Multi-touch screens can also be inconvenient for users with only one hand to use on the device (i.e., needing to hold the device and to drag multiple fingers on the face of the device concurrently with a single hand), for example while driving or for disabled users with a single hand or missing digits.
Therefore, a need exists for emulation software for allowing multi-touch behavior using a touch screen capable of supporting only a single contact input.
Certain operating systems, such as those operating on mobile devices, for example WindowsCE (Microsoft, Inc. or Redmond, Wash.), place a limit on the amount of heap memory available to each process. However, there is also a shared memory space for memory that is unassigned, but that a process could use in addition to its own private memory if assigned to the process. For example, on a 64MB device, some operating systems limit the process-designated memory to 32MB, yet there is still additional memory available to the process in the shared memory area.
Therefore, a need exists for software that can allocate shared memory to the private heap memory, regardless of the per process memory limitation by the operating system.
Usually high interactivity is desired on a client device display for functions such as zooming and panning when viewing a web page or other rendered mark-up code. In the past, there was not enough computational power on the client device to allow for smooth and full browser rendering on the client device. As shown in
Alternatively, as shown in
Therefore, there exists a need for a rendering process that can be performed entirely on the client device, yet also preserves partial rendering of the page.
A system and method for enabling the use of multi-touch applications on a device with a touch screen configured to only receive a single touch input at a time is disclosed. For example, the device can be a computer such as a cellular phone or PDA. The device can perform manipulation of at least the first cursor and a second cursor. For example, this can be performed by a multi-touch software application (i.e., computer code), process, or function.
The device can receive a first touch input on the touch screen. The device can then control the first cursor location for the first cursor based on the first touch input (e.g., if the first touch input is dragged across the screen, the first cursor can similarly move across the screen). The device can then toggle off the first cursor being the active cursor and make the second cursor the active cursor. The device can then receive a second touch input on the touch screen. The device can then control the second cursor location for the second cursor with the second touch input.
The device can then toggle off the second cursor being the active cursor and either toggle back to the first cursor being the active cursor or signal completion of the controlling of the first cursor and the second cursor. If completion of the cursor controls is signaled, the device can deliver the initial, and/or final, and/or change in locations of the first cursor and the second cursor to a multi-touch function (e.g., the operating system, a web browser, a picture or video viewer, an audio player).
The device can have a dual heap software application (i.e., computer code), process or function that can allocate shared and private memory to an executing software application. The dual heap process can allocate memory in a variation selected by the user or by a default (e.g., manufacturer-based) setting, or optimized base on the executing software application, or a combination thereof. The dual heap process can use the same memory allocation scheme for a single executing software application and/or a variety of memory allocations for a single executing software application.
The dual heap process can allocate memory so that the private memory is completely utilized before the shared memory is allocated. The dual heap process can allocate memory so that the memory allocation is alternated between the private and shared memory in about equal portions or for every memory request by the software application. The dual heap process can allocate memory so that small memory needs for the executing software application are allocated from one of the memory heaps (e.g., private or shared) and large memory needs are allocated from the other memory head (e.g., shared or private, respectively).
The device can have a meta-layer software application (i.e., computer code), process or function. The meta-layer process can render mark-up code. For example, the meta-layer process, or an application (e.g., web browser) within which the meta-layer process is functioning can receive a complete mark-up code file. The device (e.g., via the application and/or meta-layer process) can partially render the mark-up code, save the partially-rendered meta-layer mark-up code, and then complete the rendering and output (e.g., via the display and/or speakers) the completely rendered mark-up code. The saved meta-layer mark-up code can be finally rendered later, if needed, for example to alter the scale of the output of the fully rendered mark-up code.
a illustrates a variation of a client device having a touch screen. The client device can be a mobile phone or personal data assistant, for example.
b illustrates the device of
c illustrates the device of
d illustrates the device of
e and 1f illustrate the device of
g illustrates the device of
a and 3b illustrate a variation of a method for using a dual-heap system.
a, 4b and 4c illustrate a variation of a method for using a dual-heap system.
a through 5d illustrate a variation of a method for using a dual-heap system.
The client device 10 can have a touch screen 12. The touch screen 12 can be configured to receive inputs on the touch screen 12 due to pressure, for example via contact from a stylus or one or more of the user's digits. The touch screen 12 can be configured to receive a single pressure source (e.g., stylus, finger) at a given time.
The client device 10 can have a multi-touch button or key 14. The multi-touch key 14 can be the only non-screen based input device on the client device 10 or the multi-touch key 14 can be one of two or more non-screen based input devices, such as being part of or separate from a keyboard 16. The multi-touch key 14 can be on the front, either side, back, top or bottom of the client device 10. The client device 10 can have more than one multi-touch key 14.
The client device 10 can have a processor and memory and be configured to execute computer software. The client device 10 can be, for example, an HTC 8925 (HTC Corporation of Taoyuan, Taiwan).
The client device 10 can execute a multiple-touch (“multi-touch”) software application that can allow a user to control the location of multiple cursors on the touch screen with a single contact point. For example, the multi-touch software can enable the client device 10 to use a software application requiring multiple contact points (e.g., an application that zooms when two fingers are dragged toward or away from each other). The multi-touch software application can be executed concurrently with other programs. For example, the multi-touch software application can be executed concurrently with a graphics-displaying application, such as a video or image file displaying application and/or a web browser.
b illustrates that the device 10 can be placed into a multi-touch emulation mode (“multi-touch mode”). When the multi-touch mode is activated, for example, a first cursor 18 and a second cursor 20 can be displayed on the touch screen 12. Additional (e.g., third, fourth) cursors can also be displayed on the touch screen 12.
The multi-touch mode can be indicative that the multi-touch software is executing at all, or the multi-touch mode can be activated by the multi-touch software even though the multi-touch software can also be executing in a “sleep” mode when the multi-touch mode is off. The multi-touch mode can be activated by a control, such as the multi-touch key 14, a menu item selected via the keyboard, the touch screen, or audio control (e.g., voice activation) via an audio transducer, such as a microphone, or combinations thereof.
The first cursor 18 can initially be at a first cursor first x-location 22a and a first cursor first y-location 22b on the touch screen 12. The second cursor 20 can be at a second cursor first x-location 24a and a second cursor first y-location 24b. The first cursor first x-location 22a can be equal to, greater than or less than the second cursor first x-location 24a. The first cursor first y-location 22b can be equal to, greater than or less than the second cursor first y-location 24b. For example, the first cursor 18 and the second cursor 20 can be vertically aligned. The first cursor 18 and the second cursor 20 can be vertically or horizontally aligned. The first cursor and second cursor first x and y-locations 22a, 22b, 24a and 24b, can be preset to match user-defined preferences. For example, the user can set the first cursor and second cursor first x and y-locations 22a, 22b, 24a and 24b to fit the user's hand size and/or handedness.
The multi-touch software can have predefined (e.g., by the user and/or by the manufacturer) left and right-handedness settings for the first cursor and second cursor first x and y-locations 22a, 22b, 24a and 24b. For example, the right-handedness can have a smaller first cursor first x-location 22a than the second cursor first x-location 24a setting. The left-handedness setting can have a larger first cursor first x-location 22b than the second cursor second x-location 24b setting. The multi-touch software can have predefined hand size settings (e.g., small, medium and large) for the first cursor and second cursor first x and y-locations 22a, 22b, 24a and 24b. For example, the larger the hand size, the farther away the first cursor 18 is located from the second cursor 20 in the first locations 22a, 22b, 24a and 24b.
Other predefined settings can include whether to initially move the first cursor, the second cursor, or both (or all) cursors concurrently.
The user's hand 26 (digits such as finger 28 and thumb 30 are also shown) can be near the client device 10. The hand 26 is shown in
The multi-touch software can define the first cursor 18 as the initially active cursor. The active cursor can be the cursor to be moved or otherwise controlled by user interaction with the touch screen.
c illustrates that a stylus or digit, such as the thumb 30, can be pressed to the touch screen 12 (indicated by the thumb 30 being in solid lines in
d illustrates that after the first cursor 18 is moved, the first cursor 18 can be at a first cursor second x-location 32a and a first cursor second y-location 32b on the touch screen 12. The second cursor 20 can be at a second cursor second x-location 34a and a second cursor second y-location 34b on the touch screen 12. If the multi-touch software is set to not move the second cursor 20 during movement of the first cursor 18, the second cursor second x and y-locations 34a and 34b can be identical to the second cursor first x and y-locations 24a and 24b.
Once the active first cursor 18 has been moved, for example to a desired final location, the active cursor can be toggled by the multi-touch software so the second cursor 20 is the active cursor. Toggling the active cursor from the first cursor 18 to the second cursor 20 can be performed by triggering a control, such as a button on the keyboard or elsewhere, multi-touch key 14, voice command, or menu command selected from the touch screen with the keyboard joystick, other input device, time-out mechanism, or combinations thereof. For example, the thumb 30 can be lifted away from the touch screen 12 and press the multi-touch key 14 to toggle the active cursor. The multi-touch key can be pressed and released to toggle the active cursor, or pressed and held (e.g., releasing the multi-touch key 14 can signal the completion of all cursor movement) to toggle the active cursor. The active cursor can be toggled back and forth one or more times between the first and second cursors 18 and 20, and other cursors if available.
e illustrates that the finger 28 can be pressed (as indicated by the finger 18 being shows in solid lines) to the touch screen 12, for example when the second cursor 20 is the active cursor. The finger 28 can be dragged, as shown by arrow, across the touch screen 12. The multi-touch software can keep the first cursor 18 motionless and move the second cursor 20 corresponding to the movement of the finger 28.
f illustrates that after the second cursor 20 is moved, the second cursor can be at a second cursor third x-location 36a and a second cursor third y-location 36b on the touch screen 12.
After moving the second cursor 20, the active cursor can be toggled back to the first cursor 18 if desired. The method can also be used to toggle between and move additional active cursors, such as third and fourth cursors.
g illustrates that after the first cursor 18 and the second cursor 20 are moved to final desired positions, the user's hand 26 can be removed from the touch screen 12. The multi-touch function can then be activated, for example, by pressing or releasing the multi-touch key 14, voice activation, or combinations thereof. The multi-touch function that is the process or application that receives the location of the cursors as input. The multi-touch function can be, for example, a function within a program for viewing images, video, playing audio files, web browsers, word processing, spreadsheets, animation effects, scientific data viewing/analysis, or combinations thereof. The multi-touch function can be for zooming (e.g., in/out, center, cursor-based zooming, default zoom settings), panning, scrolling, toggling between applications or options within applications, web browser operations (e.g., page forward, page back, refresh page), rotating, stretching, animation and lighting special effects, or combinations thereof
a illustrates that an executing software application 50 on the client device can interact with a dual heap process 52. The executing software application 50 can be a web browser, photo effects program, any other application listed herein, or other applications.
The client device can have private memory 54 and shared memory 56. Shared memory 56 is memory not yet assigned to an executing application. The private memory 54 can be allocated by the operating system to the executing software application. The dual heap process 52 can execute within the operating system of the client device, as a stand-alone application or within the executing software application. The dual heap process 52 can access the shared memory 56 and the private memory 54. The dual heap process 52 can control the location of the memory used by the executing software application 50.
As shown in
b illustrates that when the private memory is substantially completely used by the executing software application, the dual heap process can begin to allocate the shared memory for the executing application. After the private memory is initially filled, if any private memory is freed from use during the executing of the executing software application, the dual heap process can assign the recently-freed private memory to the executing software application (i.e., to refill the private memory before returning to use additional shared memory).
a illustrates that the dual heap process 52 can initially allocate memory from the private memory 54 to the executing software application 50. The amount of memory initially allocated from the private memory 54 can be a predetermined quantity of memory (e.g., about 200 kBytes).
b illustrates that once the predetermined quantity of memory has been initially allocated from the private memory 54, the dual heap process 52 can allocate memory from the shared memory 56 to the executing software application 50. The amount of memory initially allocated from the shared memory 56 can be a predetermined quantity of memory (e.g., about 2 MB).
c illustrates that once the predetermined quantity of memory has been initially allocated from the shared memory 56, the dual heap process 52 can allocate additional memory from the private memory 54 to the executing software application 50. The amount of memory additionally allocated from the private memory 54 can be a predetermined quantity of memory, for example the same predetermined quantity of memory allocated initially from the private memory 54 and/or shared memory 56 (e.g., about 2 MB).
The dual head process 52 can continue to alternate memory allocation to the executing software application 50 between the private memory 54 and the shared memory 56 in approximately equally sized increments of memory (e.g., about 2 MB) until the private memory 54 or shared memory 56 is filled, at which time the dual heap process 52 can allocate no additional memory to the executing software application 50, or the dual heap process 52 can allocate additional memory to the executing software application 50 from whichever of the private or shared memories 54 or 56 that still have available memory.
The evenly alternating memory allocation method by the dual heap process 52 shown in
a through 5d illustrate that the dual heap process 52 can alternate allocation of memory between the private memory 54 and the shared memory 56 based on the size of the individual allocation of memory needed. For example, the dual heap process can allocate larger individual allocations (e.g., greater than about 200 kBytes) of memory from the shared memory 56 and smaller individual allocations (e.g., smaller than about 200 kBytes) of memory from the private memory 54, or vice versa (i.e., smaller allocations from the shared memory 56 and larger allocations from the private memory 54).
a illustrates that the dual heap process 52 can allocate a first small individual memory allocation from the private memory 54 for the executing software application 50.
b illustrates that the dual heap process 52 can then allocate a second small individual memory allocation from the private memory 54 for the executing software application 50.
c illustrates that the dual heap process 52 can then allocate a first large individual memory allocation from the shared memory 56 for the executing software application 50.
d illustrates that the dual heap process 52 can then allocate a third small individual memory allocation from the private memory 54 for the executing software application 50.
The meta mark-up code 64 can then be finally and completely rendered 70b into the final completely rendered mark-up 66 by the client device 10.
If the final rendering 70b is desired to be re-performed (e.g., to rescale a web page for zooming, panning, or to render for pull down menus or other click-activated actions), the client device 10 can retrieve the saved meta mark-up code 64 from memory on board the client device 10. The client device 10 can then re-perform the final complete rendering on the meta mark-up code 64 with the new criteria (e.g., rescale ratio), for example, without the need to retrieve or receive the unrendered mark-up 62 nor the need to partially render the unrendered mark-up to the meta mark-up 64 configuration. Therefore, the meta-layer client process can reduce the processing resources and time required by the client device 10, and reduce repeated downloading of unrendered of partially rendered mark-up 62 or 64 from the server device 60 to the client device 10.
The first meta mark-up code 64 can be saved on the client device 10 while the client device retrieves and processes a second, third, or more mark-up codes 62. Also, first, second, third, or more meta mark-up codes 64 can also be concurrently saved 72 on the client device 10, and retrieved as desired.
The mark-up code can be HTML, XML, SVG, SMIL, XHTML, or combinations thereof.
The multi-touch software application, the dual heap process and the meta-layer client process can be used concurrently and/or sequentially with each other on a single client device, for example with and/or in a single web browser application.
As used herein, software applications are a set of instructions that can be executed by one or more processors in the client device 10. The software applications can be executed within the operating system of the client device 10 or as a process separate from the operating system.
It is apparent to one skilled in the art that various changes and modifications can be made to this disclosure, and equivalents employed, without departing from the spirit and scope of the invention. Elements shown with any variation are exemplary for the specific variation and can be in used on or in combination with other variations within this disclosure.
