Field of Invention
The present invention relates to browsing digital media assets on a display device and, more particularly, to browsing digital media assets with assistance of audio cues.
Description of the Related Art
Personal computers supporting multimedia usage typically operate media management programs for managing various digital media assets, such as music files and video files. Media management programs typically utilize a graphical user interface that allows a user to play, sort, modify, store and catalog their media. Conventionally, graphical user interfaces have a text-based browse window that allow a user to browse through and select a particular media track for playback. More recently, graphical user interfaces have been able to provide an image-based browse environment in which a user can browse images (e.g., cover art) that pertain to corresponding digital media assets. The media management programs may also be linked with an online media store so that new media can be purchased and transferred to the personal computer and subsequently managed by the media management program. One example of a media management program is iTunes™ manufactured by Apple Inc. of Cupertino, Calif., and one example of an online media store is iTunes Store™.
Today, media playback devices, such as media players, have storage capacity to store large collections of media, such as songs (music). With such large media collections, in order for a user to select a particular song to be played by the media player, the user typically navigates through a long list of songs. As an example, the user could interact with the media player to display a list of artists that have songs that are stored within the media collection on the media player. The user then scrolls through the list and selects an appropriate song to be played. Unfortunately, however, as the storage capacity of a media player increases, the number of songs stored in a media collection on the media player also increases. Consequently, the list through which users must navigate in order to select a song can be extensive.
Therefore, there remains a need for improved techniques for browsing digital media assets.
The invention pertains to methods, systems and apparatus for image-based browsing through a plurality of digital media assets assisted by media cues. The media cues can be provided by playback of short segments, referred to as snippets, of the digital media assets being browsed.
In one embodiment, as a user browses through the digital media assets using visual representations of collections of digital media assets that are displayed for the user, media cues can be provided. For example, the user can cause a pointing action with respect to the visual representation, and the proximity of the pointing action can be used to determine which one or more media cues are to be provided. The media cues can be provided by playback of short segments the one of more of the digital media assets of a collection. Characteristics of the pointing action and/or position of the pointing action with respect to a given visual representation can influence the media cues provided and also how it is played back.
The digital media assets (i.e., digital media items) can pertain to video items (e.g., video files or movies), audio items (e.g., audio files or audio tracks, such as for songs (music) or audiobooks), or image items (e.g., photos).
The invention can be implemented in numerous ways, including as a method, system, device, apparatus (including computer readable medium and graphical user interface). Several embodiments of the invention are discussed below.
As a method for browsing digital media assets, one embodiment can, for example, include at least: displaying a plurality of images having associated active regions on a display screen of a client computer, each of the images being associated with and representing one or more of the digital media assets; detecting a user pointing input at one of the active regions; identifying one or more digital media assets associated with the one of the active regions where the user pointing device is at; and initiating playback of a media snippet of the one or more identified digital media assets if the detecting detects that the user pointing device is at the one of the active regions.
As a computer-implemented method for review media items, one embodiment can, for example, include at least: displaying a representation of at least one collection of media items; receiving a user input movement over the representation of the at least one collection of media items; and playing a short segment of media content pertaining to at least one of the media items in the collection of media items in response to the user input movement over the representation of the at least one collection of media items.
As an apparatus for browsing and playing digital media assets, one embodiment can, for example, include at least: a storage device for storing a plurality of digital media assets; a display device for displaying visual representations pertaining to the digital media assets; and a controller operatively connected to the storage device and the display device. The controller can be configured to at least (i) display a plurality of visual representations, each of the visual representing being associated with and representing one or more of the digital media assets stored in the storage device; (ii) detect a user pointing input at a particular one of the visual representation; (iii) identify one or more digital media assets associated with the particular one of the visual representations; and (iv) initiate playback of a media snippet of the one or more identified digital media assets if the user pointing device is detected at the particular one of the visual representations.
As a computer readable storage medium including at least executable computer program code stored thereon for browsing digital media assets, one embodiment can, for example, include at least: computer program code for displaying a plurality of images having associated active regions on a display screen of a client computer, each of the images being associated with and representing one or more of the digital media assets; computer program code for detecting a user pointing input at one of the active regions; computer program code for identifying one or more digital media assets associated with the one of the active regions where the user pointing device is at; and computer program code for initiating playback of a media snippet of the one or more identified digital media assets if the user pointing device is detected to be at the one of the active regions.
As a computer-implemented method for review media items, one embodiment can, for example, include at least: computer program code for displaying a representation of at least one collection of media items; computer program code for receiving a user input movement over the representation of the at least one collection of media items; and computer program code for playing a short segment of media content pertaining to at least one of the media items in the collection of media items in response to the user input movement over the representation of the at least one collection of media items.
Other aspects and advantages of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings which illustrate, by way of example, the principles of the invention.
The invention will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like elements, and in which:
The invention pertains to methods, systems and apparatus for image-based browsing through a plurality of digital media assets assisted by media cues. The media cues can be provided by playback of short segments, referred to as snippets, of the digital media assets being browsed.
In one embodiment, as a user browses through the digital media assets using visual representations of collections of digital media assets that are displayed for the user, media cues can be provided. For example, the user can cause a pointing action with respect to the visual representation, and the proximity of the pointing action can be used to determine which one or more media cues are to be provided. The media cues can be provided by playback of short segments the one of more of the digital media assets of a collection. Characteristics of the pointing action and/or position of the pointing action with respect to a given visual representation can influence the media cues provided and also how it is played back.
The image-based browsing of digital media assets assisted by media cues can, according to one embodiment, be performed on an electronic device, such as a computing device, that that supports a display device as well as an audio output device. The computing device can, for example, be a personal computer (e.g., desktop computer, notebook computer or netbook computer), or a handheld portable electronic device.
The digital media assets (i.e., digital media items) can pertain to video items (e.g., video files or movies), audio items (e.g., audio files or audio tracks, such as for songs (music), podcasts or audiobooks), image items (e.g., photos), or any combination thereof.
Embodiments of various aspects of the invention are discussed below with reference to
The controller 102 can include one or more specialized modules that can be implemented in hardware or software. As illustrated in
In one embodiment, the browsing apparatus 100 can pertain to a computing device. The computing device can, for example, be a personal computer, such as a desktop computer, notebook computer or netbook computer. The computing device can also be a handheld portable electronic device that supports a display device as well as an audio output device. The audio output device 116 for the browsing apparatus 100 can, for example, be a speaker or an audio output connector. One or more headsets, earbuds or earphones can be connected to the audio output connector.
The user input device 114 can provide a pointing action by the user. The pointing action can be provided by a cursor controlled by a mouse, trackball, touch pad, etc. The pointing action can also be provided by a user touch input with respect to a touch-sensitive surface, which may or may not be integrated with the display device 112.
The browse process 200 can begin with a decision 202 that determines whether a user input has been received with respect to an active region. Here, it is assumed that representations for a plurality of different media items (or collections of media items) are presented on a display device. Each of the representations has one or more active regions associated therewith. The decision 202 determines whether a user input, such as a user pointing input, has been received with respect to an active region. When the decision 202 determines that a user input with respect to an active region has not been received, the browse process 200 awaits such a user input. Alternatively, when the decision 202 determines that a user input with respect to an active region as being received, one or more media items associated with the active region can be identified 204. Next, playback of a snippet (i.e., short segment) of the one or more media items can be initiated 206.
A decision 208 can then determine whether media playback of the one or more media items that have been initiated 206 should continue. When the decision 208 determines that the media playback of the one or more media items are not to be continued, the browse process 200 can return to repeat the decision 202 and subsequent blocks so that additional user inputs can be processed to activate playback of other snippets. Alternatively, when the decision 208 determines that the media playback of the one or more media items is to be continued, the media items for which the snippet is being play can be identified 210. Extended playback of the identified media items can then be initiated 212. Following the block 212, the browse process 200 can return to repeat the decision 202 and subsequent blocks so that additional user inputs can be processed to activate playback of other snippets.
The browse process 400 can initially display 402 a plurality of images. The plurality of images are displayed 402 on a display device. In one embodiment, each of the images can pertain to a collection of the items. After the plurality of images are displayed 402, a decision 404 can determine whether a user pointing device is over any particular one of the images. When the decision 440 determines that the user pointing input is not over a particular image, the browse process 400 can await such a user pointing input. However, in some cases different types of user inputs can be processed by the electronic device while the browse process 400 is awaiting the user pointing input over a particular image.
Once the decision 404 determines that a user pointing input is over a particular image, a zone of an active region containing the user pointing input can be determined 406. The active region can be generally commensurate with the image, and the active region can be divided into one or more zones. After the zone of the active region containing the user pointing input has been determined 406, a media item associated with the zone of the active region can be identified 480. As an example, each of the images can pertain to a collection of media items, and each of the zones can correspond to one of the media items of the collection of media items. After the media item associated with the zone of the active region has been identified 408, a snippet of the media item can be acquired 410. The snippet is a short segment of the media content associated with the media item. The particular segment of the media content to which the segment corresponds may be predetermined, randomly determined or user determined. After the snippet of the media item has been acquired 410 playback of the snippet can be initiated 412.
Next, a decision 414 can determine whether a playback request has been received. A playback request being received while in the snippet is being played can be a request from the user to continue to play the media item to which the snippet corresponds. When the decision 414 determines that a playback request has been received, extended playback of the media item can be initiated 416. The extended playback of the media item can be implemented by continued playing of the media item from the end of the snippet, or can be implemented by playback of the media item from the beginning following the playback of the snippet. Alternatively, when the decision 414 determines that a playback request has not been received, the block 416 can be bypassed.
Following the block 416 or the bypassing of the block 416, a decision 418 can determine whether a list display request has been received. A list display request is a request from the user to display a list of media items associated with a particular image being displayed 402. For example, since the images being displayed 402 correspond to a collection of media items, the list can provide in the listing of those of the media items within the collection. When the decision 418 determines that a list display request has been received, a list of media items associated with the particular image can be determined 420. The list of media items can then be displayed 422. Alternatively, when the decision 418 determines that a list display request has not been received, blocks 420 and 422 can be bypassed.
Following the block 422 or the bypassing of the block 422, a decision 424 can determine whether a list undisplay request has been received. The list undisplay request operates to remove the display of the list of media items, if such a list was previously display, such as by block 422. When the decision 424 determines that a list undisplay request has been received, the list of media items can be undisplayed 426. In other words, in response to the list undisplay request, the prior display the list of media items at block 422 can be removed such that the list of media items is no longer displayed. Alternatively, when the decision 424 determines that the list undisplay request has not been received, the block 426 can be bypassed. Following the block 426, or its being bypassed, the browse process 400 can return to repeat the block 402 and subsequent blocks so that the browse process 400 can continue with respect to the same plurality of images or a different plurality of images.
The browse processing according to one or more embodiments of the invention can display a plurality of images on a display device. The images are used to facilitate the browse process. As noted above (e.g.,
Regardless of the particular arrangement of images, each of the images 502, 526, 528, 546 and 548 serves as a visual representation for one or more digital media assets. When the visual representation pertains to more than one digital media asset, the visual representation can be said to pertain to a collection (of digital media assets). For example, a collection of songs can pertain to an album, and the image 502, 526, 528, 546, 548 displayed for a given album can be the cover art for the album. As another example, a collection of songs can pertain to a user-formed mixture of digital media content, and the image 502, 526, 528, 546, 548 displayed for the collection of songs can be a mosaic formed from two or more images pertaining to songs within the collection.
In
In
In
The ordering of the images associated with media assets within the groupings (e.g., clusters, stacks, grids, timelines, etc.) discussed above can be based on various attributes of metadata or other data. As few examples, ordering can be alphabetical, by date (e.g., release date), popularity (aggregated popularity data), or play count.
The various aspects, features, embodiments or implementations of the invention described above can be used alone or in various combinations.
Embodiments of the invention can, for example, be implemented by software, hardware, or a combination of hardware and software. Embodiments of the invention can also be embodied as computer readable code on a computer readable medium. The computer readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of the computer readable medium generally include read-only memory and random-access memory. More specific examples of computer readable medium are tangible and include Flash memory, EEPROM memory, memory card, CD-ROM, DVD, hard drive, magnetic tape, and optical data storage device. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
Numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will become obvious to those skilled in the art that the invention may be practiced without these specific details. The description and representation herein are the common meanings used by those experienced or skilled in the art to most effectively convey the substance of their work to others skilled in the art. In other instances, well-known methods, procedures, components, and circuitry have not been described in detail to avoid unnecessarily obscuring aspects of the present invention.
In the foregoing description, reference to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the order of blocks in process flowcharts or diagrams representing one or more embodiments of the invention do not inherently indicate any particular order nor imply any limitations in the invention.
The many features and advantages of the present invention are apparent from the written description. Further, since numerous modifications and changes will readily occur to those skilled in the art, the invention should not be limited to the exact construction and operation as illustrated and described. Hence, all suitable modifications and equivalents may be resorted to as falling within the scope of the invention.
This application is a continuation of U.S. application Ser. No. 12/554,946, filed Sep. 7, 2009, which is hereby incorporated by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
5303388 | Kreitman et al. | Apr 1994 | A |
5345543 | Capps et al. | Sep 1994 | A |
5396590 | Kreegar | Mar 1995 | A |
5452414 | Rosendahl et al. | Sep 1995 | A |
5463725 | Henckel et al. | Oct 1995 | A |
5838326 | Card et al. | Nov 1998 | A |
5864868 | Contois | Jan 1999 | A |
5880733 | Horvitz et al. | Mar 1999 | A |
5995101 | Clark et al. | Nov 1999 | A |
6006227 | Freeman et al. | Dec 1999 | A |
6023275 | Horvitz et al. | Feb 2000 | A |
6346951 | Mastronardi | Feb 2002 | B1 |
6466237 | Miyao et al. | Oct 2002 | B1 |
6486895 | Robertson et al. | Nov 2002 | B1 |
6489951 | Wong et al. | Dec 2002 | B1 |
6577330 | Tsuda et al. | Jun 2003 | B1 |
6638313 | Freeman et al. | Oct 2003 | B1 |
6667751 | Wynn et al. | Dec 2003 | B1 |
6725427 | Freeman et al. | Apr 2004 | B2 |
6768999 | Prager et al. | Jul 2004 | B2 |
6975306 | Hinckley et al. | Dec 2005 | B2 |
7085590 | Kennedy et al. | Aug 2006 | B2 |
7479949 | Jobs et al. | Jan 2009 | B2 |
7492350 | Fabre et al. | Feb 2009 | B2 |
7581186 | Dowdy et al. | Aug 2009 | B2 |
7656393 | King et al. | Feb 2010 | B2 |
7710393 | Tsuk et al. | May 2010 | B2 |
7818691 | Irvine | Oct 2010 | B2 |
7877687 | Blinnikka et al. | Jan 2011 | B2 |
7966638 | Gossweiler, III | Jun 2011 | B2 |
7996792 | Anzures et al. | Aug 2011 | B2 |
8296656 | Dowdy et al. | Oct 2012 | B2 |
8365235 | Hunt | Jan 2013 | B2 |
8600529 | Shafer et al. | Dec 2013 | B2 |
20010015719 | Van Ee et al. | Aug 2001 | A1 |
20010030660 | Zainoulline | Oct 2001 | A1 |
20010050687 | Iida et al. | Dec 2001 | A1 |
20020030699 | Van Ee | Mar 2002 | A1 |
20020113824 | Myers, Jr. | Aug 2002 | A1 |
20020198909 | Huynh et al. | Dec 2002 | A1 |
20030090504 | Brook et al. | May 2003 | A1 |
20030122787 | Zimmerman et al. | Jul 2003 | A1 |
20040013416 | Mok | Jan 2004 | A1 |
20040095371 | Haynes et al. | May 2004 | A1 |
20040100479 | Nakano et al. | May 2004 | A1 |
20040119757 | Corley et al. | Jun 2004 | A1 |
20040122683 | Grossman et al. | Jun 2004 | A1 |
20040123320 | Daily et al. | Jun 2004 | A1 |
20040125088 | Zimmerman et al. | Jul 2004 | A1 |
20040189690 | Poslinski et al. | Sep 2004 | A1 |
20040199582 | Kucharewski et al. | Oct 2004 | A1 |
20040235520 | Cadiz et al. | Nov 2004 | A1 |
20050012723 | Pallakoff | Jan 2005 | A1 |
20050024341 | Gillespie et al. | Feb 2005 | A1 |
20050091596 | Anthony et al. | Apr 2005 | A1 |
20050110768 | Marriott et al. | May 2005 | A1 |
20050114778 | Branson et al. | May 2005 | A1 |
20050114800 | Rao | May 2005 | A1 |
20060026535 | Hotelling et al. | Feb 2006 | A1 |
20060195790 | Beaupre et al. | Aug 2006 | A1 |
20060247980 | Mirrashidi | Nov 2006 | A1 |
20070033295 | Marriott | Feb 2007 | A1 |
20070044036 | Ishimura et al. | Feb 2007 | A1 |
20070053268 | Crandall et al. | Mar 2007 | A1 |
20070070066 | Bakhash | Mar 2007 | A1 |
20070085841 | Tsuk et al. | Apr 2007 | A1 |
20070124680 | Robbin et al. | May 2007 | A1 |
20070136750 | Abanami et al. | Jun 2007 | A1 |
20070168413 | Barletta et al. | Jul 2007 | A1 |
20070260994 | Sciammarella et al. | Nov 2007 | A1 |
20080036743 | Westerman et al. | Feb 2008 | A1 |
20080062141 | Chaudhri | Mar 2008 | A1 |
20080066016 | Dowdy et al. | Mar 2008 | A1 |
20080066135 | Brodersen et al. | Mar 2008 | A1 |
20080086687 | Sakai et al. | Apr 2008 | A1 |
20080086688 | Chandratillake et al. | Apr 2008 | A1 |
20080120291 | Delgo et al. | May 2008 | A1 |
20080122796 | Jobs et al. | May 2008 | A1 |
20080155458 | Fagans et al. | Jun 2008 | A1 |
20080168403 | Westerman et al. | Jul 2008 | A1 |
20080288867 | Jeong et al. | Nov 2008 | A1 |
20080313570 | Shamma et al. | Dec 2008 | A1 |
20090002335 | Chaudhri | Jan 2009 | A1 |
20090019398 | Hansson et al. | Jan 2009 | A1 |
20090030991 | Vakkalanka | Jan 2009 | A1 |
20090049383 | Haynes et al. | Feb 2009 | A1 |
20090049385 | Blinnikka et al. | Feb 2009 | A1 |
20090049979 | Naik et al. | Feb 2009 | A1 |
20090100093 | Makipaa | Apr 2009 | A1 |
20090150784 | Denney et al. | Jun 2009 | A1 |
20090158201 | Iampietro et al. | Jun 2009 | A1 |
20090158326 | Hunt | Jun 2009 | A1 |
20090172532 | Chaudhri | Jul 2009 | A1 |
20090174677 | Gehani et al. | Jul 2009 | A1 |
20090198359 | Chaudhri | Aug 2009 | A1 |
20090228919 | Zott et al. | Sep 2009 | A1 |
20100070523 | Delgo et al. | Mar 2010 | A1 |
20100325549 | Gibson | Dec 2010 | A1 |
20110282745 | Meoded et al. | Nov 2011 | A1 |
Number | Date | Country |
---|---|---|
1 148 412 | Oct 2001 | EP |
1 677 497 | Jul 2006 | EP |
2000-105772 | Apr 2000 | JP |
2003-330586 | Nov 2003 | JP |
WO 0179980 | Oct 2001 | WO |
WO 2006013485 | Feb 2006 | WO |
WO 2006020305 | Feb 2006 | WO |
WO 2008030779 | Mar 2008 | WO |
WO 2008030976 | Mar 2008 | WO |
Entry |
---|
“CoverFlow,” www.steelskies.com/coverflow/, downloaded Jun. 15, 2006, 2 pages. |
“CoverFlow—Download,” www.steelskies.com/coverlfow/download.php, down loaded Jun. 15, 2006, 4 pages. |
“CoverFlow—FAQ & Tips,” www.steelskies.com/coverlfow/faq.php, downloaded Jun. 15, 2006, 3 pages. |
“CoverFlow—Features,” www.steelskies.com/coverflow/features.php, downloaded Jun. 15, 2006, 1 page. |
Damm et al., “Multimodal Presentation and Browsing of Music,” ICM '08, Oct. 20-22, 2008, Chania, Crete, Greece, 4 pages. |
Enright, “Dissatisfaction Sows Innovation,” The Treehouse + The Cave, Dec. 29, 2004, 4 pages. |
Enright, “Dissatisfaction Sows Innovation—Visual Browsing in iTunes,” Internet Blow of Andrew Coulter Enright, Dec. 29, 2004, 6 pages. |
Enright, “Meet CoverFlow,” The Treehouse + The Cave, Aug. 13, 2005, 2 pages. |
Enright, “Visual Browsing on an iBook DS,” The Treehouse + The Cave, Dec. 29, 2004, 1 page. |
Hinze, “CoverFlow—A Beautiful Way to Browse Your MP3s,” Noise Magazine Blog, Feb. 5, 2006 2 pages. |
“Mechanism for Visual Lists and Selections,” IBM Technical Disclosure Bulletin IBM, vol. 40, No. 5, May 1, 1997, 2 pages. |
Rose, “Music in the Home: Interfaces for Music Applications,” Personal Technologies, v. 4, No. 1, Mar. 2000, 9 pages. |
Sawyer, “Get with the CoverFlow,” The Olive Press, Dec. 8, 2005, 1 page. |
Website prinout, www.steelskies.com, downloaded Jun. 15, 2006, 4 pages. |
Wikipedia, “Image Map,” http://wayback.archive.org/web/20071021151336/http://en.,wikipedia.org/wiki/Image_map, Oct. 21, 2007, 3 pages. |
Office Action, dated Feb. 17, 2012, received in U.S. Appl. No. 12/544,946, 31 pages. |
Final Office Action, dated Jul. 6, 2012, received in U.S. Appl. No. 12/544,946, 39 pages. |
Office Action, dated Aug. 15, 2013, received in U.S. Appl. No. 12/544,946, 36 pages. |
Final Office Action, dated Jan. 2, 2014, received in U.S. Appl. No. 12/544,946, 39 pages. |
Office Action, dated Jul. 11, 2014, received in U.S. Appl. No. 12/544,946, 15 pages. |
Final Office Action, dated Nov. 4, 2014, received in U.S. Appl. No. 12/544,946, 16 pages. |
Notice of Allowance, dated Jul. 1, 2015, received in U.S. Appl. No. 12/544,946, 12 pages. |
Number | Date | Country | |
---|---|---|---|
20160054978 A1 | Feb 2016 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 12554946 | Sep 2009 | US |
Child | 14929065 | US |