The present invention relates generally to wireless mobile devices and more particularly to applications that offer navigate, click and drag capabilities. Among such applications, one type is a mobile photos application.
Mobile-friendly technologies are advanced to provide a rich multimedia environment and enhance the wireless device users' experience. An outcome of this evolution is the manifest closeness between the wireless universe and the Internet domain, as well as the advent of wireless devices with multimedia capabilities. The newest versions of mobile wireless devices such as digital mobile phones, pagers, personal digital assistants (PDAs), handsets, and any other wireless terminals, have multimedia capabilities including the ability to retrieve e-mail, and push and pull information via the Internet. Evidently, developers of pocket personal computers such as the Pocket PC™ are working to adopt the elegance and simplicity of devices such as the Palm Pilot™, and the developers of the Palm Pilot™ are moving towards more features, multimedia and the higher-end processors of the Pocket PC™.
The various configurations of mobile device tend to fit their main purpose. For example, devices focused on wireless email access tend to use the Research In Motion (RIM)™ or Danger™ operating systems while devices whose core functionality is personal information management (PIM) tend to use the Palm-OS™ operating system. In many of the mobile devices, voice and PIM are key features along with e-mail, calendaring and in many instances image viewing, Web browsing and connectivity such as Bluetooth™ and Wi-Fi™ connectivity.
With the advancements in display technology, including better resolution, bit depth, and graphics, the range of mobile device display capabilities continues to expand and increase the usefulness of image viewing and Web browsing. Image viewing and Web browsing are important features of mobile devices. Such features enable users to access and view information remotely while on the road for, among other things, mobile marketing, remote presentations, process monitoring, on-location image capturing and delivery, and more. However, even the advanced display capabilities of mobile devices lack certain user interface features.
Indeed, even with the limited memory capacity of mobile devices there are ways to improve the range of image viewing and web browsing capabilities with minimal impact on the memory bandwidth consumption. In one instance, user interface with the ability to arrange images on the mobile display adds to the usefulness of mobile devices.
Thus, the invention provides ways for manipulating (moving and rearranging) images on the mobile display. In essence, the navigation key (up-down-left-right arrows) is used to navigate to the desired image. The ‘OK’ button is used to activate the drag (move-rearrange) feature with respect to the desired image. The arrows are additionally used, once the drag feature is activated, to rearranged the images (moving the desired image by swapping its location with that of another image). Finally, the ‘OK’ button is again used, once the desired image has been moved, to fix the location of this image at its current location. Otherwise, until the ‘OK’ key has been pressed, activating the navigation key would keep moving the desired image around the screen. The approach presented here provides for easier navigation between and arrangement of images on the mobile display with little or no additional burden on the limited resources of mobile devices. Specifically, the navigate, click and drag concept is implemented in mobile applications so as to increases their usefulness despite the limited memory capacity of mobile devices. In one example, the Yahoo! Photos application includes the navigate, click and drag feature for manipulating and arranging images on mobile display.
For the purpose of this invention, as embodied and broadly described herein, a method and a system are proposed as possible implementations of the backup and restore concept. These implementations typically involve a server in communication link with a plurality of mobile devices. In the context of the Yahoo! Photos application, the mobile devices are typically wireless devices such as wireless camera phones and the memory items includes photograph data (or simply one or more photos).
In one embodiment, a method for providing the image movement by navigation click and drag includes maintaining place holders and a selected holder in the memory of the mobile device. The method further includes attributing indicia to the image, assigning the indicia of the image to one of the place holders, maintaining a selected holder in the memory, selecting the image and saving in the selected holder the screen location associated with the place holder to which indicia of the selected image is assigned. To move the selected image, the method additionally includes reassigning the indicia of the selected image to a different one of the place holders and replacing in the selected holder the saved screen location associated with the place holder with the screen location associated with the different place holder. Memory bandwidth consumption is lower with the use of the place holders and the selected holder than it would be with only movement of the selected image.
In one implementation, the mobile device is a wireless device such as a wireless camera phone. The mobile device uses the method above as part of a mobile application one example of which is Yahoo! photos. The mobile device includes system components, including memory and processor, to facilitate the image movement as described above. The memory, in addition to the place holders and selected holder, embodies a computer program with code to cause the processor to perform the method as described above.
As can be understood from these examples, by introducing the navigation click and drag capability to the system, the present invention makes the user interface more useful and reacting more responsively. Such advantages will be appreciated by those of ordinary skill in the art from the description and practice of the invention disclosed herein.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention. Wherever convenient, same or similar numbers or designations are used throughout the drawings to refer to the same or like elements.
The present invention contemplates navigate click and drag and the implementation of this concept in mobile applications, as illustrated in
Although it can be implemented in various applications, for clarity and for illustration, the approach contemplated by the present invention is described here in the context of the Yahoo! Photos application. The server side of this application is the “server Yahoo! Photos,” and the client side of this application is the mobile client application, or “client Yahoo! Photos.” A client application is generally considered to be a downloadable application; namely, J2ME™ (Java™ 2 platform, Micro Edition, by Sun Microsystems, Inc.), Yahoo! Photos™, or any other application that is downloadable to the mobile device. In the example here, the client Yahoo! Photos runs on a mobile phone, and more specifically, a mobile camera phone.
The Wireless Communication Environment
Indeed, with this capability, users can capture photo images in their mobile devices, store and manipulate the captured images, and upload data of the captured images to a server (e.g., server 18). Thus, the server 18 operates as a repository for the data of photo images, and users can download from the server to their mobile devices data of previously captured photo images, as well as store and manipulate such images. Photos resident on one mobile device can be shared with another via the server 18 and the communication network(s) 12 and 16.
In this wireless interconnect model, the mobile phones used to download the Yahoo! Photos client side program are WAP-enabled (WAP stands for Wireless Application Protocol). As shown in
To enable web-based access to content, service providers deploy wireless data through the carrier network 12 while controlling the data communications to their subscribers and tracking the billable activity. Typically, the gateway 14 is tasked with tracking subscriber activities, controlling access and, in addition, functioning as the proxy for the mobile device 100, on the one hand, and for the server 18, on the other hand. The gateway 14 is implemented, building on standard web proxy technology, to interconnect the services offered by the wireless service providers to the HTTP protocol so as to permit access to content on the wired Internet.
One model of interaction between a WAP-enabled device, the WAP-enabled proxy/gateway, and the server, is the HTTP 1.1 response/request transaction, where HTTP 1.1 is profiled for the wireless environment. The gateway (13 & 14) translates requests from the WAP protocol to the WWW protocol, and vice versa; translating WML(/HTML) documents to HTML(/WML), resolving domain names in URLs and providing a control point for managing access. From the WAP-enabled gateway with encoders/decoders, the URL requests or WML documents (possibly in encoded form) can be sent encoded/decoded to add security to the user interaction. Note that, unlike the flat structure of HTML documents, WML documents are divided into a set of user interaction units, namely a deck of cards. Each user interaction unit is a card (or page), and the user can navigate between cards in one or more WML documents.
Another model of interaction between a WAP-enabled device, the WAP-enabled proxy/gateway, and the server, is the HTTP response/request transaction (protocol running on top of the Internet's TCP/IP suite of protocols). This model is appropriate for the newer WAP 2.0 (with protocol stack not shown in
Yet another model of interaction via bearer networks, between 3rd-generation (3G)-enabled mobile devices and servers or other devices, is shown in
Various services are supported by these protocols, including web browsing, short message service (SMS), multimedia messaging service (MMS), e-mail, M-commerce, real-time video, and pre-paid. The MMS, for example, is a store and forward messaging service capable of adding multimedia elements to SMS, including images, text, audio clips, and video clips. MMS is synchronized across a common timeline, rather than being discrete like e-mail and SMS; it is akin to a presentation layer over e-mail and looking like a slide show with images. On a compatible phone, the MMS message will appear with a new message alert. The picture message will open on the screen, the text will appear below the image and the sound will begin to play automatically.
Downloadable applications such as Yahoo! Photos and network games are likewise supported in the 3G terminal and interact with services such as MMS. The originator can easily create a multimedia message, either using a built-in or accessory camera, or can use images and sounds stored previously in the phone (and possibly downloaded from a web site). However, for simplicity, the following description assumes that the mobile device is a WAP-enabled camera phone used for downloading photo applications such as the Yahoo! Photos.
As further shown in
It should be mentioned that, although the manufacturer provides the Yahoo!-enabled phone 100 with camera functionality—i.e., functionality for capturing images, and saving, displaying, manipulating, transmitting and receiving data of images—this camera functionality is independent from the Yahoo! Photos program. That is, data of the captured images resides in the mobile phone outside the Yahoo! Photos environment until such time that this data is introduced to the Yahoo! Photos environment by being first uploaded to the Yahoo! server and then downloaded to the local (mobile) Yahoo! Photos album, as will be later explained.
Note also that the example here focuses on the camera phone, but the principles of the present invention are not limited to camera phones. Any phone or other wireless mobile device can embody a variation of the present invention. When the mobile device is a smart handset, downloading application programs and implementing the navigate click and drag scheme are possible even though the communications with the service provider may be different in character.
The mobile device functionality is preferably implemented using a platform, such as the J2ME™ platform, which is tailored for a broad range of embedded devices including mobile phones. The J2ME™ platform includes a set of standard Java APIs (application programming Interface), and provides a user interface, a security model, built-in network protocols (e.g., WAP, as shown in
With the J2ME™ platform, applications are written once for a wide range of device. Applications leveraging each device's native capabilities are then downloaded dynamically. The J2ME™ platform defines configurations, profiles and optional packages as elements for building complete Java run time environments. Configurations are composed of a virtual machine and a minimal set of class libraries and provide the base functionality for a particular range of devices that share similar characteristics. Current configurations include connected limited device configuration (CLDC) for devices with limited memory and processing capabilities (e.g., mobile phones, two-way pagers, and PDAs) and connected device configuration (CDC) for devices with better memory, processing and network bandwidth capabilities (e.g., TV set-top boxes, residential gateways, in-vehicle telematics systems, and hi-end PDAs). However, in order to provide a complete runtime environment targeted at specific device categories, the configurations must be combined with a set of the high-level APIs, or profiles, that further define the application life cycle model, access to device-specific properties, and user interface.
One example of profiles is the mobile information device profile (MIDP) which is designed for mobile phones and entry-level PDAs. MIDP offers a core application functionality required by mobile applications, including user interface, network connectivity, local data storage, and application management. The J2ME™ can be further extended by combining various optional packages and their corresponding profiles to address specific market requirements, e.g., Bluetooth™, web services, wireless messaging, multimedia, and database connectivity.
Navigate Click and Drag in the Context of Mobile Yahoo! Photos
Before describing its relevant features, we first discuss how the Yahoo! Photos program is obtained and run by the mobile device. On mobile devices, various client application programs are offered to the user on a default start-up or main menu screen or on a manufacturer-installed virtual vending machine screen (vending machines for client (J2ME) applications are the digital rights management systems of the carriers designed to manage the ownership or licensing of applications from the carrier on behalf of the application developer/distributor to be purchased by the consumer). Other selection items include, for example, the menu item for setting the sound. These start up and vending screens show a menu with a list (or icons) of applications which the user can obtain by following an install procedure. The menu provides links to various service web sites, including, for example, the Yahoo! Photos site. The links, of course, are URLs (Uniform Resource Locator)—i.e., the world wide web address of a site on the Internet, and on the Yahoo!-enabled phone, at least one such menu item is the link for downloading the Yahoo! Photos application.
If the user accepts the offer to buy the application, the order is confirmed 322 and the application is downloaded into the mobile phone, becoming resident on the mobile phone. Once the Yahoo! Photos program is resident on the mobile phone it can be invoked from the landing page or menu page (using the menu button on the phone to bring up the menu or using the default menu if Yahoo! Photos is presented as one of the default menu options). Invocation of the Yahoo! Photos application allows, among others, user access and manipulation of the user's mobile album as well as online albums in the user account.
Invocation of Yahoo! Photos prompts this program to display the ‘home’ page 2.0 with two main options: mobile album, and online album (as shown in
As mentioned before, photo images can be captured and manipulated by the mobile phone outside the Yahoo! Photos environment. These photo images will not be available at the mobile or online albums until they are uploaded to the server and then (selectively or in batch) downloaded from the server. Accordingly, selecting ‘online album’ allows the user to access and manipulate photo images that have already been uploaded to the server from the user's PC or mobile phone and stored in the online album. The online album is dynamically rendered within the client (mobile device) based on the client-server interactions (i.e., photos are pulled from the server dynamically and placed in the online album). It reflects the exact state of the server at the time of the query and requires a live online connection. To save a photo from the online album to the mobile album the user clicks “Save to Mobile Album.” This process takes time, in some instances approximately 10 seconds, depending on the device and network connectivity. Each time this action takes place it is recorded (as backup) by the server.
By analogy, selecting ‘mobile album’ allows the user to access and manipulate photo images that have been already downloaded from the server and saved in the mobile album. If the user saves, for example, 5 photos to the mobile album, the user is then able to view the 5 photos without again accessing the server via the network (wireless and Internet). Viewing these photos in the mobile album is significantly faster than viewing them in the online album because they are native (present locally) instead of being dynamically pulled from the server over the network.
Then, if the ‘online album’ option is selected from the Yahoo! Photos client program ‘home’ page (2.0), as shown in
The next page is the ‘my online albums’ page (2.1). For the specific user, this online albums page lists the names of photo albums available to the named user which are associated with the user's account. Of course, the album listing includes only albums that are on the server and can be dynamically pulled from it. If the selected album is empty the next page will display an indication to that effect (i.e., “this album is currently empty” at page; 2.1.6). Alternatively, if the album is not empty, selecting that album will bring up the next page, the ‘photo list’ page for that album (2.1.2). In the ‘photo list’ page, a photo can be selected for downloading it from the server onto the mobile phone. Additionally, a selected photo can be opened or other actions can be invoked in relation to it. The other actions are presented in a menu that is shown on the screen as a pull-down menu, pop-up menu, or a menu superimposed on any part of the current page (in this example the menu is shown as a pull-down menu).
Such menu (hereafter “photo options menu”) provides a number of selection items, each of each representing an action, including: ‘save to mobile,’ ‘email photo,’ ‘screen saver,’ ‘thumbnails,’ ‘online albums,’ and ‘home.’ Each selection brings up a page that corresponds to the selected action item. Two of the action items have already been discussed above, ‘home’ and ‘online album.’ Selecting home, will lead the user back to the home page (2.0), and selecting online album, will lead the user to the aforementioned ‘my online albums’ page (2.1).
Selecting ‘thumbnails’ brings up a ‘photo thumbs’ page 2.1.1 that shows a group of thumbnail photo images from the selected album. Note that the number of photo thumb groups downloaded from the server depends on the memory size of the mobile phone (or whatever device is used). With this feature, the user can then thumbnail through the groups of photos in the album. The groups of thumbnail photo images in this album are each loaded from the server. The user can then move between the images back and forth (scroll back and forth) and select any one of the photos in the ‘thumbnails’ page. A selected thumbnail image will be enlarged in the next page, the ‘online photo’ page (2.1.3). More on the navigation, click and drag will be described later in conjunction with
As can be seen in
When ‘email photo’ action is invoked, the ‘share as email’ page comes up (2.1.5). This page shows the photo(s) selected for emailing and prompts the user for the email address. In this implementation, a number of recently-used email addresses are provided. Incidentally, ‘email’ is simply a transport mechanism which is presently used to send photos from camera phones. Other transport mechanisms may be developed and employed for this application. Then, when the photo is emailed from the mobile phone to the selected e-mail address, a message pops up indicating that the email has been sent or, if not, that an error occurred. For example, a transmission will fail if the user is not authorized to upload photos to the selected e-mail. An error message of this kind is a product of the upload security scheme as embodied in the Yahoo! Photos application program.
When the ‘screen saver’ action is invoked, the selected photo will be used to populate the screen when the phone is idle, standing by, or starting up. The ‘screen saver’ option is associated with screen saver page (2.1.4) which shows the selected photo and requires the user to select ‘OK’ or ‘cancel’ to add this photo to the screen saver photo roster. A message pops up to indicate the status of the photo download.
Going back to the mobile album is possible with the photo options menu via the ‘home’ page, using the ‘home’ option as discussed above. Another way for getting to the mobile album or any other previous page is with the “back” action using the ‘back’ button. Also, as mentioned above, when the Yahoo! Photos application is invoked from the landing/menu page, the ‘home’ page (2.0) presents the ‘mobile album’ as one of the selection items. Accordingly, the mobile album can be accessed directly via the ‘home’ page.
The mobile album screen flow, shown in
Except for the photos being local (at the mobile album), the thumbnails feature, associated with the ‘photo thumbs’ page (3.1.2), works as described above with reference to the online album. A photo selected on the mobile ‘photo thumbs’ page can be enlarged as shown in the next page, the ‘mobile photo’ page (3.1.3). The menu for the ‘photo thumbs’ and ‘mobile photo’ pages includes a subset of the aforementioned mobile photo action menu.
When the slide show is invoked from such a menu the ‘mobile slide show’ page comes up (3.3). While this feature is active, the slide show scrolls through the mobile album photos, showing each photo for a certain period. The slide show goes on until the user selects ‘stop’ on the bottom of the page. If the user selects ‘actions’ a slide show menu gives the user the options of ‘pause,’ ‘show,’ ‘normal,’ and ‘fast.’ The ‘pause’ option is selected for pausing the slide show; ‘slow’ will slow down the slide show, ‘speed’ will speed up the slide show, and ‘normal’ will bring it to normal speed. (
As further shown in
As with the previous page (“where are my photos?” page 3.1.4), the ‘restore album’ page (3.1.4.2) allows the user to go to the ‘home’ page (2.0). This time, via ‘OK’, it allows the user to go to the next mobile ‘restore album’ page (3.1.4.2.1) for a historical photo download list (of photos previously downloaded to the mobile phone).
Note that the pages shown in
From the screen flows and page navigation patterns we turn now to discuss the image navigation, clicking and dragging. The principles of image navigation, i.e., scrolling up-down-left-right between images, and clicking and dragging are illustrated in
As shown in
For implementing the foregoing navigate, click and drag feature, one embodiment includes a plurality of place holders, one for each image plus one that is a holder of the location index of a selected image. For the four images in the foregoing example, the program uses five placeholders (4 indices and one selected index holder). As shown in
Initially, when the program starts running and the display is created, the selected index holder is empty, and we assume that:
When a photo thumbs or photo move page is loaded and put on the screen, as shown in
The place holders for the image indices, e.g., index1-index4, are location indices associated with the location of images on the screen. For example, index1 is associated with the location at the top-left corner and index4 is associated with the location at the bottom-right corner of the screen.
In terms of possible variations in the implementation, the place holders and indices can include a value or a pointer to the value. Also, the use of indices is a mere convenience and not necessarily a limitation on the manner in which the images or their location can be referenced. For example, any indicia to properly reference the images can be used without departing from the scope and spirit of the present invention.
As shown in
As shown in
It is important to remember that although the navigate, click and drag functionality is described in the context of the Yahoo! Photos program, it is useful in any mobile device application where this type of user interface capability is desired. Thus, although this feature is implemented for the Yahoo! Photos application, it can be implemented more generically for other applications.
Going back to the general overview, in the context of Yahoo! Photos, every photo from the user's online album that is saved to the mobile album is ‘remembered’ by the server. Indeed, any action (for example, the history required for the back in sequence implementation), not just a photos download history, can be recorded as a backup on the server side. Preferably, since the page traversal path is not predictive the history is recorded accurately and fully.
One navigation feature of the Yahoo! Photos that benefits from the aforementioned backup and restore of history records is the back functionality, particularly the back in sequence functionality. As to navigating through the pages on the mobile phone, the pages can be traversed forward as described above and they can be traversed backwards using the “back button” feature.
As can be seen, the “back a level” mode allows hierarchical backwards sequence traversal one level each time the ‘back’ button is touch activated or clicked (hereafter “clicked”). The “back in sequence” mode allows sequential backwards one page each time the ‘back’ button is pressed. For example, in back a level mode, back a level takes the application from a photo page (e.g., 6) one level up to the list of photos page (3); and from there one more level up to the list of albums page (2) and one more level up to the home page (1). As can be further seen in this example, the back in sequence mode functions to take the application from the current photo page (6) to the former photo page (5), rather than up one level (3), when the back button is touched. Additional activations of the back button will traverse through all the pages in reverse sequence.
It makes no difference if the “back button” feature is used while in the online album or mobile album part of the application. The principles apply equally well to both situations. Either way, the steps (pages traversed) are remembered, and they can be recorded server side, locally, or both on the server side and locally.
In view of the above, the navigate, click and drag functionality allows image selection and arrangement with relative ease. The implementation details of this functionality as described above are applicable but are not exclusive to Yahoo! Photos. Also, other applications may allow for alternative implementation of navigate, click and drag using the same principles.
Implementation Details
Additional implementation details associated with the foregoing description are provided below. These implementation details include an initial list of devices, soft key mapping, labels, global elements and screen flows tables for the online albums and mobile albums. These details are described in the following pages.
Possible Mobile Devices
The visual and interaction design as described herein should accommodate various types of mobile devices, including, for example, those listed in the table below.
Soft Key Mapping
For the purpose of this invention, the following keys are available on the mobile devices: Up; Down; Left; Right; Select/OK; Left Soft key; Right Soft key; and Back. If a device does not have an obvious select key, it is assumed that the MIDP (mobile information device profile) implementation will automatically provide a select option at one of the soft keys or in one of the soft key menus.
Soft key & Menu Labels
In a representative implementation, labels that may appear on a soft key are restricted to 7 characters. Menu-only items are restricted to 14 characters.
Common Labels
Global Elements
Confirmation Popup
One type of global elements, presented as “Confirm Popup” screens, are used for displaying a confirmation to the user. The confirmation popup screens contain simple text such as “Done” or “Saved”, and they disappears automatically after a short time.
In Progress Screen
The “in progress” screen informs the user that the application is waiting for a response from the server or is processing a request. Each device has a default screen with text and a moving graphic, and, alternatively, it is replaced with a Yahoo! Canvas screen.
Screen Flows: Online Albums
As described above, the online album pages are made available to the user in forward and backwards traversal; each page having default selection items associated with it. The forward traversal starts, of course, with the home page (2.0). The following tables outline for each page separately the default selection items available in that page for screen flows.
Screen Flows: Mobile Album
As with the online album, the mobile album pages are made available to the user in forward and backwards traversal; each page having default selection items associated with it. Here again, the forward traversal starts, of course, with the home page (2.0). The following tables outline for each page separately the default selection items available in that page for screen flows.
Although the present invention has been described in accordance with the embodiments shown, variations to the embodiments would be apparent to those skilled in the art and those variations would be within the scope and spirit of the present invention. Accordingly, it is in tended that the specification and embodiments shown be considered exemplary only, with a true scope of the invention being indicated by the following claims and equivalents.
This application claims the benefit of and incorporates by reference U.S. Provisional Application No. 60/518,898 entitled “Back Button in Mobile Application,” U.S. Provisional Application No. 60/518,858, entitled “Navigation Pattern on a directory Tree,” U.S. Provisional Application No. 60/518,857, entitled “Backup and Restore in Mobile Applications,” and U.S. Provisional Application No. 60/518,897, entitled “Upload Security Scheme,” all of which were filed Nov. 10, 2003.
Number | Date | Country | |
---|---|---|---|
60518898 | Nov 2003 | US | |
60518858 | Nov 2003 | US | |
60518857 | Nov 2003 | US | |
60518897 | Nov 2003 | US |