As the popularity of the Internet and mobile devices such as cell phones and media players rises, the ability to easily access, locate, and interact with content available through these entertainment/information portals becomes more important. A popular way to consume video content is via playlists of video assets. Playlists may be user-generated, for example a playlist of music videos, or created by a media provider, for example a playlist of the top ten news stories of the clay. The ability to generate, navigate, and watch playlists typically requires a system equipped with a graphical user interface, software, and other systems to enable management of the video assets that make up a playlist. For example, the system can offer an interface with which a user can jump to a desired video asset by clicking on a corresponding link and must be able to locate and retrieve video assets in the playlist. Transferring a playlist to another media player (e.g., from a personal computer to a handheld media player) requires compatible software capable of playlist management as well, which limits the media players and/or interfaces with which a user can access a desired playlist. A need remains for a method of generating playlists of video assets that are more universally compatible.
Methods and systems of generating and delivering composite videos having a plurality of video assets are provided. A composite video may be a single asset formed from the plurality of video assets and having graphics which facilitate navigation of the assets. The composite video may also include advertising that may be related to contents of the video assets. Metadata corresponding the plurality of video assets is used to facilitate delivery of the composite videos. In particular, metadata may be used to enable selection of video assets for inclusion, for generating the video and graphics presented in the composite video, and for selecting or locating the advertising included in the composite video. The graphics included in the composite video may facilitate navigation of the composite video using generic playback devices, namely ones capable of at least fast-forward and rewind functions. Such a system provides great utility as it facilitates the broad distribution of video assets.
According to one aspect of the invention, systems and methods for generating a composite video having a plurality of video assets are provided. Systems may include storage having a plurality of video assets, a metadata generator, and a composite video generator. The metadata generator processes respective ones of the video assets to generate a metadata track representative of information that is descriptive of the content of the video asset. The composite video generator receives a plurality of metadata tracks and, in response, processes the associated video assets and the metadata tracks to generate a composite video asset having video data from the video assets. The composite video asset may have a metadata list panel that presents information representative of the metadata information as video data appearing within the composite video asset. The metadata list panel visually presents the sequence of video assets in the composite video asset.
According to another aspect of the invention, methods and systems for delivering video assets are provided. Methods may include the step of providing a plurality of video assets and for respective ones of the video asset, metadata tracks representative of information about a respective video asset. The video assets are representative of video content of the type capable of being displayed on a video player. A user may select from the plurality of video assets to identify a set of video assets to be joined sequentially into a composite video. The selected video assets may be processed as a function of their respective metadata tracks to generate a composite video. The composite video has a metadata list panel that presents metadata information as video data appearing within a composite video that includes video data from the video asset. The metadata list panel visually presents the sequence of video assets in the composite video. In one embodiment, a plurality of advertisement videos representative of video content for promoting a good or service are provided, where the advertisement videos are capable of display on a video player. At least one advertisement video may be joined within the sequence of video assets in the composite video.
In the detailed description which follows, reference will be made to the attached drawings, in which:
The invention includes methods and systems for generating and delivering composite videos having a plurality of video assets. The following illustrative embodiments describe systems and methods for processing and presenting video content. The inventions disclosed herein may also be used with other types of media content, such as audio or other electronic media.
The content receiving system 102 may receive video content via a variety of methods. For example, video content may be received via satellite 114, imported using some form of portable media storage 116 such as a DVD or CD, or downloaded from or transferred over a network such as the Internet 118, for example by using FTP (file transfer protocol). Other methods and systems for delivery of content to content receiving system 102 may also be used. Video content broadcast via satellite 114 may be received by a satellite dish in communication with a satellite receiver or set-top box. A server may track when and from what source video content arrived and where the video content is located in storage. Portable media storage 116 may be acquired from a content provider and inserted into an appropriate playing device to access and store its video content. A user may enter information about each file such as information about its contents. The content receiving system 102 may receive a signal that indicates that a website monitored by the system 100 has been updated. In response, the content receiving system 102 may acquire the updated information using FTP.
In the embodiment 100 depicted in
Video content may include broadcast content, entertainment, news, weather, sports, music, music videos, television shows, and/or movies. Exemplary media formats include MPEG standards, Flash Video, Real Media, Real Audio, Audio Video Interleave, Windows Media Video, Windows Media Audio, Quicktime formats, and any other digital media format. After being receiving by the content receiving system 102, video content may be stored in storage 120, such as Network-Attached Storage (NAS) or directly transmitted to the tagging station 104 without being locally stored. Stored content may be periodically transmitted to the tagging station 104. For example, news content received by the content receiving system 102 may be stored, and every 24 hours the news content that has been received over the past 24 hours may be transferred from storage 120 to the tagging station 104 for processing.
The tagging station 104 processes video to generate metadata that corresponds to or is associated with the video. The metadata may be information, commentary, hypertext links, audio narratives, additional video, or any other information a user may wish to associate with the video. The metadata may enhance an end user's experience of video content by describing a video, providing markers or pointers for navigating or identifying points or segments within a video, or generating playlists of videos or video segments. In one embodiment, metadata identifies segments of a video file that may aid a user to locate and/or navigate to a particular segment within the video file. Metadata may include the location and description of the contents of a segment within a video file. The location of a segment may be identified by a start point of the segment and a size of the segment, where the start point may be a byte offset of an electronic file or a time offset from the beginning of the video, and the size may be a length of time or the number of bytes within the segment. In addition, the location of the segment may be identified by an end point of the segment. The contents of the segment may be described through a segment name, a description of the segment, tags such as keywords or short phrases associated with the contents. Metadata may also include information that helps a presentation device decode a compressed video file. For example, metadata may include the location of the I-frames or key frames within a video file necessary to decode the frames of a particular segment for playback. Metadata may also designate a frame that may be used as an image that represents the contents of a segment, for example as a thumbnail image. The tagging station 104 may also generate playlists of segments that may be transmitted to users for viewing, where the segments may be excerpts from a single received video file, for example highlights of a sports event, or excerpts from multiple received video files. The segments of a playlist may be selected by an operator interacting with the tagging station 104 (e.g., by selecting segments of a sporting event that the operator considers the highlights) and/or automatically based on a predetermined method (e.g., the most viewed segments of the day). Metadata may be stored as an XML (Extensible Markup Language) file separate from the corresponding video file and/or may be embedded in the video file itself. Metadata may be generated by a user watching or listening to the content being tagged while using a software program on a personal computer to generate the metadata or automatically by a processor configured to recognize particular segments of video.
The publishing station 106 processes and prepares the video files and metadata, including any segment identifiers or descriptions, for transmittal to various platforms. Video files may be converted to other formats that may depend on the platform. For example, video files stored in storage 120 or processed by the tagging station 104 may be formatted according to an MPEG standard, such as MPEG-2, which may be compatible with cable television 112. MPEG video may be converted to flash video for transmittal to the Internet 108 or 3 GP for transmittal to mobile devices 110. Other formats may be selected and the format or formats selected may depend upon the application.
Video files may be converted to multiple video files, each corresponding to a different video segment, or may be merged to form one video file.
In one embodiment, metadata is stored in at least two different formats. One format is a relational database, such as an SQL database, to which metadata may be written when generated. The relational database may be include tables organized by user and include, for each user, information such as user contact information, password, and videos tagged by the user and accompanying metadata. Metadata from the relational database may be exported periodically as an XML file to a flat file database, such as an XML file. The flat file database may be read, searched, or index, e.g. by an information retrieval application programming interface such as Lucene. Multiple copies of databases may each be stored with corresponding metadata servers, similar to the metadata server 312, at different colocation facilities that are synchronized.
Using the tagging station 402, a user may enter the location, e.g. the uniform resource locator (URL), of a video into a URL box 410 and click a load video button 412 to retrieve the video for playback in a display area 414. The video may be an externally hosted Flash Video file or other digital media file, such as those available from YouTube, Metacafe, and Google Video. For example, a user may enter the URL for a video available from a video sharing website, such as http://www.youtube.com/watch?v=kAMIPudalQ, to load the video corresponding to that URL. The user may control playback via buttons such as rewind 416, fast forward 418, and play/pause 420 buttons. The point in the video that is currently playing in the display area 414 may be indicated by a pointer 422 within a progress bar 424 marked at equidistant intervals by tick marks 426. The total playing time 428 of the video and the current elapsed time 430 within the video, which corresponds to the location of the pointer 422 within the progress bar 424, may also be displayed.
To generate metadata that designates a segment within the video, a user may click a start scene button 432 when the display area 414 shows the start point of a desired segment and then an end scene button 434 when the display area 414 shows the end point of the desired segment. The metadata generated may then include a pointer to a point in the video file corresponding to the start point of the desired segment and a size of the portion of the video file corresponding to the desired segment. For example, a user viewing a video containing the comedian Frank Caliendo performing a variety of impressions may want to designate a segment of the video in which Frank Caliendo performs an impression of President George W. Bush. While playing the video, the user would click the start scene button 432 at the beginning of the Bush impression and the end scene button 434 at the end of the Bush impression. The metadata could then include either the start time of the desired segment relative to the beginning of the video, e.g., 03:34:12, or the byte offset within the video file that corresponds to the start of the desired segment and a number representing the number of bytes in the desired segment. The location within the video and length of a designated segment may be shown by a segment bar 436 placed relative to the progress bar 424 such that its endpoints align with the start and end points of the designated segment.
To generate metadata that describes a designated segment of the video, a user may enter into a video information area 438 information about the video segment such as a name 440 of the video segment, a category 442 that the video segment belongs to, a description 444 of the contents of the video segment, and tags 446, or key words or phrases, related to the contents of the video segment. To continue with the example above, the user could name the designated segment “Frank Caliendo as Pres. Bush” in the name box 440, assign it to the category “Comedy” in the category box 442, describe it as “Frank Caliendo impersonates President George W. Bush discussing the Iraq War” in the description box 444, and designate a set of tags 446 such as “Frank Caliendo George W Bush Iraq War impression impersonation.” A search engine may index the video segment according to any text entered in the video information area 438 and which field, e.g. name 440 or category 442, the text is associated with. A frame within the segment may be designated as representative of the contents of the segment by clicking a set thumbnail button 450 when the display area 414 shows the representative frame. A reduced-size version of the representative frame, e.g. a thumbnail image such as a 140×100 pixel JPEG file, may then be saved as part of the metadata.
When finished with entering information, the user may click on a save button 448 to save the metadata generated, without necessarily saving a copy of the video or video segment. Metadata allows a user to save, upload, download, and/or transmit video segments by generating pointers to and information about the video file, and without having to transmit the video file itself. As generally metadata files are much smaller than video files, metadata can be transmitted much faster and use much less storage space than the corresponding video. The newly saved metadata may appear in a segment table 452 that lists information about designated segments, including a thumbnail image 454 of the representative frames designated using the set thumbnail button 450. A user may highlight one of the segments in the segment table 452 with a highlight bar 456 by clicking on it, which may also load the highlighted segment into the tagging station 402. If the user would like to change any of the metadata for the highlighted segment, including its start or end points or any descriptive information, the user may click on an edit button 458. The user may also delete the highlighted segment by clicking on a delete button 460. The user may also add the highlighted segment to a playlist by clicking on an add to mash-up button 462 which adds the thumbnail corresponding to the highlighted segment 464 to the asset bucket 404. To continue with the example above, the user may want to create a playlist of different comedians performing impressions of President George W. Bush. When finished adding segments to a playlist, the user may click on a publish button 466 that will generate a video file containing all the segments of the playlist in the order indicated by the user. In addition, clicking the publish button 466 may open a video editing program that allows the user to add video effects to the video file, such as types of scene changes between segments and opening or closing segments.
Metadata generated and saved by the user may be transmitted to or available to other users over the network and may be indexed by the metadata index of the search engine corresponding to the search button 408. When another user views or receives metadata and indicates a desire to watch the segment corresponding to the viewed metadata, a playback system for the other user may retrieve just that portion of a video file necessary for the display of the segment corresponding to the viewed metadata. For example, the hypertext transfer protocol (http) for the Internet is capable of transmitting a portion of a file as opposed to the entire file. Downloading just a portion of a video file decreases the amount of time a user must wait for the playback to begin. In cases where the video file is compressed, the playback system may locate the key frame (or I-frame or intraframe) necessary for decoding the start point of the segment and download the portion of the video file starting either at that key frame or the earliest frame of the segment, whichever is earlier in the video file.
The user may also during playback of a video or video segment mark a point in the video and send the marked point to a second user so that the second user may view the video beginning at the marked point. Metadata representing a marked point may include the location of the video file and a pointer to the marked point, e.g. a time offset relative to the beginning of the video or a byte offset within the video file. The marked point, or any other metadata, may be received on a device of a different platform than that of the first user. For example, with reference to
In general, a device on a platform 108, 110 or 112 depicted in
The video assets generated using the systems and methods described above, as well as music videos and other video assets may be combined into a composite video using the system of
Based on the user input 610, the composite video generator 602 locates and retrieves the video assets indicated by the user input 610 from the database 608 using metadata from the metadata generator 604, where each video asset may have an associated metadata track including information representative of the contents of the video asset. The metadata may be transmitted directly from the metadata generator 604 or via a storage device storing metadata from the metadata generator 604. The video assets may be transmitted directly from the database 608 or via an index of locations of video assets. The advertisement scheduler 606 transmits data indicating when and where to place the ads within the composite video to the composite video generator 602. The advertisement scheduler 606 may include an ad server which provides ads (e.g., banner ads or video ads selling or promoting a good or service), or data indicating where such ads are located to the composite video generator 602. In some embodiments, the ad server selects ads that are targeted based on criteria associated with the end consumer. To this end, the ad server may select ads to provide to the generator 602 as a function of the metadata of the associated metadata tracks provided by the metadata generator 604.
In some embodiments, the composite video generator 602 includes a controller 612, a video processor 614, a graphics generator 616, and a compositer 618. The controller 612 generates a timeline of assets, graphics, and/or ads based on the user input 610 and using data from the advertisement scheduler 606 and the metadata generator 604. For example, the user input 610 may indicate a playlist of video assets. The controller 612 receives from metadata generator 604 metadata for the video assets including a location of each video asset (e.g., a time stamp of a video file at which the video asset begins), a length of each video asset (e.g., a start time and an end time of the video asset), and data describing the contents of the each video asset (e.g., a title, description, and/or keywords) which may be displayed as graphics within the composite video. From the advertisement scheduler 606, the controller 612 receives ads or data indicating ad locations and criteria guiding ad placement within the composite video. Exemplary criteria includes requiring a video ad to play between each video asset or every other video asset or every 10 minutes or some other predetermined length of time, an order in which to present video ads or banner ads, and where to place ads on a presentation screen (e.g., the corner of the screen or along the length of the screen). Criteria guiding ad placement may alternatively or additionally be received from the user input 610. The controller 612 may request ads from the advertisement scheduler 606 based on the user input 610 and/or on metadata tracks associated with the video assets indicated by the user input 610. From the metadata and ad information received from the metadata generator 604 and the advertisement scheduler 606, the controller 612 generates a sequence of time stamps for the composite video and corresponding events that occur at the times indicated by the time stamps, such as a video asset, ad, or graphics to be displayed in the composite video.
The controller 612 transmits the timeline and metadata to a video processor 614 and a graphics generator 616 for processing. Based on the metadata, the video processor 614 retrieves video assets, which may include video ads, from the database 608 and concatenates the retrieved video assets into a single video according to the timeline. It is well known in the art how to process multiple video files into one video file that presents the contents of the multiple video files consecutively. The graphics generator 616 generates graphics for the composite video based on the metadata and timeline received from the controller 612. In particular, the graphics generator can generate graphics that correspond to the single video generated by video processor 614 for overlaying on top of the single video. Exemplary graphics include metadata list panels, or visual playlists, that highlight a playlist item that corresponds to a video asset, visual indicators indicating the current temporal location within a video asset, and ads. In particular, the graphics generator 616 may include a visual indicator tool for generating visual indicators such as a progress mark, i.e., a visually perceptible mark having a characteristic that changes function as a function of temporal progress of the video asset. Exemplary progress marks may be generated by a progress mark generator and are described further below with respect to
The compositer 618 receives the single video from video processor 614 and graphics from graphics generator 616 and generates a composite video of both. In particular, the composite video can be a single file, such as an MPEG encoded file, in which the video assets play consecutively with corresponding graphics overlaid onto the video assets.
Graphics of the composite video may include visual indicators to indicate the current location within the composite video. For example, during playback, rewind, or fast-forward of the composite video file, the playlist 710, the ad 712, the progress bar 706, and/or the logo 714 may remain substantially unchanged for a period of time, even as the composite video switches between viewing modes such as those depicted in
In screenshot 850, a listing of options for criteria for presenting ads is presented. For example, the user may indicate the types of ads desired, such as banner ads or interstitial video ads, using area 852. The user may indicate the frequency with which ads are presented using area 854. The user may also upload a video or image file containing a desired ad at area 856. Screenshots 800 and 850 may be useful for a commercial or other entity desiring a media vehicle for delivering advertising, allowing the entity to sponsor a composite video that may be customized according to the desires of the entity. The composite video generated is easily transferred and does not require proprietary systems or software to present, allowing the entity to easily display the composite video on its own platforms such as a website. The entity may also search for and/or select video assets related to the products and/or services it is advertising. For example, Nike may have Tiger Woods as a spokesperson, and would like the create a composite video of video assets containing Tiger Woods sinking golf holes in which Nike ads would be interspersed.
At step 908, video assets of the lists of video assets and advertisements may be received, for example, from database 608 and/or advertisement scheduler 606. At steps 910 and 912, the received video assets are processed to form a composite asset and to generate graphics, respectively, according to the received lists, which may be performed by video processor 614 and graphics generator 616, respectively. The composite asset formed at step 910 and the graphics generated at step 912 may be combined to form a composite video asset at step 914, which may be performed by compositer 618. In practice, one or more steps shown in process 900 may be combined with other steps, performed in any suitable order, performed in parallel (e.g., simultaneously or substantially simultaneously), or removed.
Applicants consider all operable combinations of the embodiments disclosed herein to be patentable subject matter.
This application claims the benefit of U.S. Provisional Patent Application No. 60/790,182 filed Apr. 7, 2006 and entitled “System and Method for Enhanced Graphical Video Navigation” and U.S. Provisional Patent Application No. 60/872,736 filed Dec. 4, 2006 and entitled “Systems and Methods of Searching For and Presenting Video and Audio,” both of which are herein incorporated by reference.
Number | Date | Country | |
---|---|---|---|
60790182 | Apr 2006 | US | |
60872736 | Dec 2006 | US |