1. Field
The disclosed embodiments generally relate to user interfaces and, more particularly to a user interface for contextual selection of media files or other data items.
2. Brief Description of Related Developments
In the current mobile devices, the constantly increasing storage capacities have resulted in the capability of storing very large numbers of audio, images, video and other multimedia files in the devices themselves. This large number of possible selections combined with rather limited user interface (“UI”) capabilities (e.g. reduced display size) can result in increasingly difficult navigation through the content stored in the device or locating desired content.
The usual approach for organizing the media files generally includes presenting the user with a sorted list according to specific criteria (release years, song/album titles, singer etc.). The user will typically have to browse and search the list to find a desired file, files, or data items.
It would be advantageous to be able to reduce the user effort when locating or browsing multimedia content or other data items.
The disclosed embodiments are directed to a method, apparatus, user interface and computer program product for providing each file and data item identified in a device with at least one multi-dimensional descriptor, selecting at least one component from the at least one multi-dimensional descriptor as an initial search criteria for a selected item, identifying in the device all other items that have a relationship with respect to the initial criteria, and presenting the identified files or data items to the user. An ordering relation induced by the selected search criteria can be used for presenting the closest matches in the proximity of the selected item.
The foregoing aspects and other features of the embodiments are explained in the following description, taken in connection with the accompanying drawings, wherein:
The disclosed embodiments generally allow a user of a device or system, such as the system 100 shown in
For example, in one embodiment, the data or data items comprise contacts in an instant messaging application. Each contact is considered the data, or content, that will establish the criteria for the search.
In one embodiment, the search is not limited to a data type or file type of the anchor application. The search can be conducted across any or all data and file types stored in the device. For example, the criteria for the search is the name “George”, that is selected from a contact list in an instant messaging application. The search can include determining whether the word “George” is present in any other data source or file in the device. These can include, for example, music files, image files, contact files, or instant message presence application.
Although the disclosed embodiments will generally be described with respect to data, content or files stored or located in a device, the data, content and files could also be remotely located from the device. In one embodiment, where the content is remotely located, the device can include metadata that relates the content identified in the device to the actual content that is stored remotely.
The aspects of the disclosed embodiments allow a user of a device 100 to select one or more search criteria related to an active or selected file or content on a device, such as a music file. Using the search criteria, a search is executed in the device 100 for other files or content 100 that have some relationship with the active file, relative to the search criteria. In one embodiment, the relationship may only be the search criteria. The results of the search can be displayed on the display of the device 100. Further criteria can be selected from the search criteria options being presented and a subsequent search executed to find files or content that match, are a near match or have some relationship or similarity to all of the activated search criteria and the selected file or content. The aspects of the disclosed embodiments allow step-by-step navigation in locating desired content, in a list based, contextual navigation system.
In one embodiment, certain factors can be associated with each content/data item or file stored or active in the device 100. These factors can generally represent certain features or descriptive aspects that can be associated with such content. These factors shall be referred to herein as multi-dimensional factors or descriptors. For example, with respect to music files, the dimensions or factors can include descriptors such as tempo, rhythm, loudness or energy, vocal characteristics of the interpreter, singer, group, band, title, release year, instrument, environment, genre and others. With respect to photos or image files, the factors can include time and date taken, location, color balance, intensity variations, exposure value, lens used, aperture, camera model and any other factor or descriptor related to photos, for example. The factors are generally specific to the file content or data type. For example, when the data type is instant messaging, the factors can include contact information. Other factors and search criteria can include for example, instant message status, presence indicators, geographic location, online status or any other suitable factor that can be associated with the underlying application of the selected file. For different file types the search criteria can also include for example, metadata or the lack of metadata, the file name, the file type, the file properties, creation dates/times, word count, play count, language, origin or extension. These lists are merely exemplary and are not intended to include all factors or criteria that might be used to describe, categorize or quantify a file or content type or search for related files. For each content file, a suitable list of features or factors can be developed and stored. Although the embodiments disclosed herein are described with reference to content files, it should be understood that any suitable files, information lists or application lists can be used other than including content files. For example, in one embodiment, the list of search results can include all instant messaging contacts stored in the device. The user can select one or more of the search criteria 204a-204n to narrow the list of instant messaging contacts. Similarly, with an address list, the search criteria 204a-204n can correspond to factors relevant to members of the list. For example, a relevant criterion in this context is membership in a given set of persons, such as messages from family members. A search for certain types of address contacts can be carried out and narrowed as described herein. This can allow a user to move from one information item to another, or the next information item, in a manner as established by the search criteria.
In one embodiment the files, content and data items described herein can be stored in or on the device 100. Although as described herein the content is stored in the device, in one embodiment, the files, content and data items can also be located or stored remotely from the device 100, such as on an external server or hard drive. In this case, the information related to the remotely stored content handled by the user device is the metadata stored within the device, except when content is of a type that is being played or presented by the device.
In one embodiment, each file shall be associated with one or more multi-dimensional factors or descriptors. The factors can be set by the user for each file, file types can be preset with such factors, or such relevant factors can be automatically extracted on the device itself. Thus, in the example of music content, each music file will have associated with it one or more multi-dimensional factors that can be used to identify and categorize various aspects of the particular music file.
As shown in
The search criteria 204a-204n can be directly related with the elements of the multi-dimensional representation or descriptor. By using the multi-dimensional representation, one can quickly navigate the device 100 database starting with the anchor selection and repeated applications of various selection criteria. Thus, for example, a search using the criterion “rhythm” would order the elements that are returned from the search based on the value associated with it. In this example, the “value” can be the number of beats per second (“bps”). If a criteria is value dependent, a second descriptor can appear after choosing the first criteria. For example, if a criterion is value dependent, the value would be 120 beats per second and not just “beats per second.” In this example, the second descriptor that appears after choosing the first allows the user to set the value for the criteria.
In one embodiment, the user can change the criteria. For example, if one criteria is beats per minute greater than 100 (bpm>100), the user can change the number as well as the logical operator. Alternatively, for the same criteria, the user may be able to add or remove the number. As another example, the criteria is bpm>100±X. For this particular example, the user might be able to select the ± operator and the value “X”. The selection can be based on the ease with which the search can be narrowed. For example, it might be easier to narrow a search by raising or lower the number using the extra factor X. The device can also include small key inputs that allow the user to change the number X, up and down in suitable increments, without having to enter the number itself.
One or more selection criterion can be used in conjunction with the active file to develop a list of matches, based on all the activated selection criteria. As shown in
In conjunction with the selection of the anchor song 312, one or more search criteria 314 can be displayed. In one embodiment the search criteria 314 can appear in conjunction with the selection of the anchor song 312. The criteria 314 may be unique to the selected anchor song or may be preset values. The criteria 314 can automatically be generated and then customized by the user. For example, in one embodiment a list might automatically populate the screen. In one embodiment, the list can comprise a graphical representation of the information or content. For example, if the search is of an album, the album cover or a portion thereof can be displayed. In a situation where 3-D graphics are used, the list can also be shown as an xyz map or such other three-dimensional image or graphic. The user can then select/deselect criteria as desired. The user interface can also include controls for other functions of the device, such as play 316 and reset 318.
As shown in
In screen 330, the user has activated a second dimension, Crit 4334, in conjunction with Crit 2, 332. The list 324 will be re-ordered and a new list 336 is generated with the closest matches appearing as “Near match”, taking into account both Crit 2 and Crit 4. The distance in this case is for instance “the sum of distances from these two dimensions.” Generally, all of the search criteria can be viewed as introducing a distance value to the active item. We can have search predicates with 0 and 1 as output (no/yes) or smoother matches, as in the examples from the paragraph. In all cases we can imagine a significance threshold to prune down the list to top matches. When combining several criteria, a possible weighting will aim at preserving the range of the overall distance and evenly balancing the contribution of the components. As an example, if we combine only predicates the weights can be 1/N (N—number of currently active criteria) making the range of possible value [0→1]. Hence, for instance, 0.66 can be used as a significance threshold. If we have smooth components the weights for them can be 1/N*W, where W is chosen as to have its maximal distance for that component limited to 1 or, perhaps more relevant, the standard deviation for these distances to be unity. For instance, if one criterion is “date” then the time interval between the active file and the other files is used to sort the list. If the search criterion is geographic “location”, then an actual distance between locations can be used. For several search criteria which have a corresponding “distance” measure when jointly activated there is a need to produce a single distance value to order the list. In one embodiment, a weighted Euclidean measure can be used, with the weights aiming at equally balancing the contributions of individual measures (e.g. combining time with spatial distance), and/or also allowing for a unique, perhaps user specified, significance threshold. In one embodiment, depending on the distance measure which results from the selected criteria, the active item can be placed at the top (as illustrated in
While the ordering relation with respect to
In one embodiment, the active song 312 can be exchanged with any member in the current result list. For example, in screen 320, the active song 312 can be exchanged with any one of the members or results in the list 324. Similarly, the selection criteria 326 can be switched on or off, or directly and independently activated and disabled at times. In this fashion, a user can navigate across media files based on neighboring relations formed with the enabled selection criteria.
The disclosed embodiments allow a user to see what is currently being played or selected, when working with multimedia files, and see matches, and near matches to the current selection. These matches and near matches can include, for example, possible next songs or contenders. In one embodiment the user can select which criteria are to be present and which criteria should not be present with the contenders.
A search is executed and initial search result set returned 406. The search results can be ordered in any suitable manner. In one embodiment the most relevant results are displayed to the user in a more prominent fashion than non or less relevant search results. If the user is satisfied with the result set, the search ends 410. If additional refinement is desired or additional searching 408, the search criteria can again be set by deleting or adding 404 search criteria.
After each search the user can reposition the active item as one of the current matches in the list 402. He is also free to modify the list of selected search criteria 404 to widen or narrow the scope of his search.
In one embodiment, the list of matching items can be dynamically updated as the user activates/deactivates selected search criteria 404 or 506. There is no need for a specific user input to perform the actual search 406 or 508. The user is free to alternate between 402 and 404 or 502, 504 and 506 at all times. In this way the user can navigate the data items based on similarity/dissimilarity relations as imposed by the active search criteria in the context of the currently active item.
Referring to
The applications module 180 can include any one of a variety of applications that may be installed, configured or accessible by the device 100. In one embodiment the applications module 180 can include media player and multimedia applications.
In one embodiment, the system 100 comprises a mobile communication device. The mobile communication device can be Internet enabled. The input device 104 can also include a camera or such other image capturing system. The applications of the device may include, but are not limited to, data acquisition (e.g. image, video and sound) and multimedia players (e.g. video and music players). In alternate embodiments, the system 100 can include other suitable devices and applications for capturing and storing images and transferring the images to an online service.
While the input device 104 and output device 106 are shown as separate devices, in one embodiment, the input device 104 and output device 106 can be combined and be part of and form, the user interface 102. The user interface 102 can be used to display information pertaining to multi-media content as will be described below.
In one embodiment, the user interface of the disclosed embodiments can be implemented on or in a device that includes a touch screen display or a proximity screen device. In alternate embodiments, the aspects of the user interface disclosed herein could be embodied on any suitable device that will display information and allow the selection and activation of applications or system content. The terms “select” and “touch” are generally described herein with respect to a touch screen-display. However, in alternate embodiments, the terms are intended to encompass the required user action with respect to other input devices. For example, with respect to a proximity screen device, it is not necessary for the user to make direct contact in order to select an object or other information. Thus, the above noted terms are intended to include that a user only needs to be within the proximity of the device to carry out the desired function.
Similarly, the scope of the intended devices is not limited to single touch or contact devices. Multi-touch devices, where contact by one or more fingers or other pointing devices can navigate on and about the screen, are also intended to be encompassed by the disclosed embodiments. Non-touch devices are also intended to be encompassed by the disclosed embodiments. Non-touch devices include, but are not limited to, devices without touch or proximity screens, where navigation on the display and menus of the various applications is performed through, for example, keys 110 of the system or through voice commands via voice recognition features of the system.
Some examples of devices on which aspects of the disclosed embodiments can be practiced are illustrated with respect to
As shown in
In the embodiment where the device 600 comprises a mobile communications device, the device can be adapted for communication in a telecommunication system, such as that shown in
The mobile terminals 700, 706 may be connected to a mobile telecommunications network 77 through radio frequency (RF) links 702, 708 via base stations 704, 709. The mobile telecommunications network 710 may be in compliance with any commercially available mobile telecommunications standard such as for example global system for mobile communications (GSM), universal mobile telecommunication system (UMTS), digital advanced mobile phone service (D-AMPS), code division multiple access 2000 (CDMA2000), wideband code division multiple access (WCDMA), wireless local area network (WLAN), freedom of mobile multimedia access (FOMA) and time division-synchronous code division multiple access (TD-SCDMA).
The mobile telecommunications network 710 may be operatively connected to a wide area network 720, which may be the Internet or a part thereof. An Internet server 722 has data storage 724 and is connected to the wide area network 720, as is an Internet client computer 726. The server 722 may host a worldwide web/wireless application protocol server capable of serving worldwide web/wireless application protocol content to the mobile terminal 700.
A public switched telephone network (PSTN) 730 may be connected to the mobile telecommunications network 710 in a familiar manner. Various telephone terminals, including the stationary telephone 732, may be connected to the public switched telephone network 730.
The mobile terminal 700 is also capable of communicating locally via a local link 701 or 751 to one or more local devices 703 or 750. The local links 701 or 751 may be any suitable type of link with a limited range, such as for example Bluetooth, a Universal Serial Bus (USB) link, a wireless Universal Serial Bus (WUSB) link, an IEEE 802.11 wireless local area network (WLAN) link, an RS-232 serial link, etc. The local devices 703 can, for example, be various sensors that can communicate measurement values or other signals to the mobile terminal 700 over the local link 701. The above examples are not intended to be limiting, and any suitable type of link may be utilized. The local devices 703 may be antennas and supporting equipment forming a wireless local area network implementing Worldwide Interoperability for Microwave Access (WiMAX, IEEE 802.16), WiFi (IEEE 802.11x) or other communication protocols. The wireless local area network may be connected to the Internet. The mobile terminal 700 may thus have multi-radio capability for connecting wirelessly using mobile communications network 710, wireless local area network or both. Communication with the mobile telecommunications network 710 may also be implemented using WiFi, Worldwide Interoperability for Microwave Access, or any other suitable protocols, and such communication may utilize unlicensed portions of the radio spectrum (e.g. unlicensed mobile access (UMA)). In one embodiment, the navigation module 122 of
Although the above embodiments are described as being implemented on and with a mobile communication device, it will be understood that the disclosed embodiments can be practiced on any suitable device incorporating a display, processor, memory and supporting software or hardware. For example, the disclosed embodiments can be implemented on various types of music players. In one embodiment, the system 100 of
The user interface 102 of
Referring again to
The disclosed embodiments may also include software and computer programs incorporating the process steps and instructions described above. In one embodiment, the programs incorporating the process steps described herein can be executed in one or more computers.
Computer systems 802 and 804 may also include a microprocessor for executing stored programs. Computer 802 may include a data storage device 808 on its program storage device for the storage of information and data. The computer program or software incorporating the processes and method steps incorporating aspects of the invention may be stored in one or more computers 802 and 804 on an otherwise conventional program storage device. In one embodiment, computers 802 and 804 may include a user interface 810, and/or a display interface 812 from which aspects of the invention can be accessed. The user interface 810 and the display interface 812, which in one embodiment can comprise a single interface, can be adapted to allow the input of queries and commands to the system, as well as present the results of the commands and queries.
The aspects of the disclosed embodiments provide a direct access, touch based user interface that makes navigation through files and data items stored on, or remotely to a device, expedient and efficient. The contextual navigation of the list of files and data items can be based on a selection of similarity/dissimilarity. The overall look of the collection is not changed and offers step by step navigation in locating the desired content. User effort is diminished both in the number and the precision of inputs required and the number of steps to take. This provides advantages over visualizations that require expensive graphical capabilities.
It is noted that the embodiments described herein can be used individually or in any combination thereof. It should be understood that the foregoing description is only illustrative of the embodiments. Various alternatives and modifications can be devised by those skilled in the art without departing from the embodiments. Accordingly, the present embodiments are intended to embrace all such alternatives, modifications and variances that fall within the scope of the appended claims.