Patent Grant
9870128
Electronic devices may be used to consume content, such as audio books, television shows, movies, and music. In a common application, a user may view video content or listen to audio content on a home entertainment system. Home entertainment systems can include any number of purpose-built media devices, such as televisions, stereos, video game systems, and desktop computers that provide rich content consumption experiences. Users may also use personal computing devices to consume content. For example, a user may use a smart phone, tablet computer, portable media player or laptop computer to consume content. Such devices may include both audio and video output components, or they may include only an audio output component.
Television shows, movies, audio books and the like can include topics and items that may be associated with supplementary or otherwise related content. The items of related content often contain useful or interesting information, such as additional background or explanatory text regarding an event, location or person referenced in the primary content, commentary regarding the primary content in general or a closely related topic, and the like. For example, a television show may reference a historical event, or it may cover only part of a subject. Viewers may be invited to consume additional content related to, e.g., the historical event or other areas of the subject. In some cases, viewers may be provided with a network address of a content server at which they can access the content, or viewers may be shown a commercial or other promotion for another television show with related content.
Embodiments of various inventive features will now be described with reference to the following drawings. Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.
Introduction
Generally described, the present disclosure relates to managing presentation of multiple related content items, in some cases simultaneously or substantially simultaneously. Conventionally, a user may use a media device such as a television or computing device to present a content item. The content may include, but is not limited to, audio books, music, television shows, movies, games or some other electronic content. The content may be associated with supplemental or other related content, such as background content regarding a character or event. In order for the user to access such related content, the user may have to perform a manual process of stopping or pausing presentation of the primary content item in order to access the supplemental content. In some cases, users can access supplemental or related content simultaneously with consumption of the primary content item, such as by using a different device (e.g., a computing device) to access the related content while the primary content is presented on the television.
Aspects of this disclosure relate to identifying and automatically presenting supplemental or related content during presentation of a primary content item. Presentation rules, also referred to herein simply as rules, may be defined which specify which types of related content may be presented, under what circumstances the related content is to be automatically presented or suppressed, which media elements or other portions of a primary content item (e.g., audio or video tracks of a movie) are to be overridden by or mixed with related content, and the like. In some embodiments, a primary content item may contain embedded references to supplemental content that is related to the primary content item as a whole or to a specific portion of the primary content item, such as a person, place, thing, event or period of time. When such an embedded reference is encountered during presentation of the primary content item, the referenced supplemental content item can be accessed and automatically presented according to one or more rules. In some cases, the rules may indicate that the related content item (or some portion thereof) is to override the primary content item (or some portion thereof). For example, when the primary content item is a televised sporting event and the related content item is a local radio broadcast of the sporting event, a rule may specify that the media device is to automatically present the audio element or other audio portion of the local radio broadcast instead of the audio element or other audio portion of the primary content.
The example rules described herein are illustrative only, and not intended to be limiting. In some embodiments, rules may be defined with respect to various aspects or elements of content items (e.g., some rules apply to audio elements or other audio portions and some rules apply to video elements or other video portions), genres of content (e.g., some rules apply to televised sporting events while other rules apply to audio books), and the like. Generally described, a rule can specify which content type and/or media element may override another content type and/or media element, and under what circumstances such overrides may be implemented. Rules may also specify which content type and/or media elements may be overridden, and under what circumstances. In addition, rules may specify that certain content is not to be overridden, but rather is to be paused or should continue to be presented simultaneously with other content using audio mixing or split screen video display techniques. Further, rules may specify when overrides may be applied automatically, and when users may be notified of the presence of a related content item and may therefore make an affirmative decision regarding the presentation of each content item.
Additional aspects of the disclosure relate to obtaining references to related content from a network-accessible content management system. Rather than references to related content being embedded in a primary content item, the references may be maintained by and obtained from a content management system. The references may be obtained before or during playback of the primary content item. For example, when a user initiates presentation of a content item (e.g., a conventional DVD without embedded references to related content) on a media device, the media device may contact the content management system to obtain references to related content and data regarding portions of the primary content item to which the related content applies. As another example, a user may initiate presentation of a live content item (e.g., a live broadcast of a sporting event) in which some or all related content references may be unknown at the time presentation is initiated. In such cases, the media device can obtain related content references from the content management system in substantially real-time, or periodically during presentation of the primary content.
Further aspects of the disclosure relate to user interaction with the content items and user configuration of presentation rules. For example, the user may access an interface to define presentation rules, set defaults, and the like. In addition, rather than automatically overriding portions of the primary content item with related content, a user may be prompted as to whether the primary content is to be overridden or paused, or whether the related content is to be ignored. For example, when a place is mentioned during a news cast, the place may be associated with a related documentary. A notification may be displayed to the user that the documentary is available. The user may choose to override the video element of the news cast with the video element of the documentary while the audio element of the news cast continues to be presented. Alternatively, the user may choose to pause the news cast in order to view the documentary, or the user may wish to view the documentary and designate the documentary as the primary content item. If a user designates a related content item as a primary content item, then content that is related to the new primary content item—the documentary in this case—can be obtained rather than content that is related to the previous primary content item—the news cast in this case. The previous primary content item may then be a related content item for the new primary content item. Rules may be applied to the new primary content item and the newly-designated related content item, and those rules may result in a different playback configuration than rules applied to the content items under their previous designations.
Although aspects of the embodiments described in the disclosure will focus, for the purpose of illustration, on a media device obtaining related content from network-accessible servers, one skilled in the art will appreciate that the techniques disclosed herein may be applied to any number of processes or applications. For example, related content may reside on the media device itself, or it may be obtained from sources other than network-accessible servers, such as via satellite or cable television systems and the like. Various aspects of the disclosure will now be described with regard to certain examples and embodiments, which are intended to illustrate but not limit the disclosure.
With reference to an illustrative example, a user may wish to consume a content item and also access separate but related content items simultaneously. The user may initiate presentation of a primary content item, such as an audio book, on a media device, such as a smart phone or portable media player. The audio book may be associated with supplemental or related content that may be presented during presentation of the audio book. For example, when the user is listening to an audio book about a songwriter's life, the audio book may contain embedded references to related content, such as network addresses at which the media device can stream excerpts of songs written by the songwriter. In such cases, the excerpts of the songs can be mixed with the audio book (e.g., played simultaneously) at a relevant playback time or location within the audio book, such as during portions describing the conception or recording of the songs.
Rules for how content of one type may override other content types may be defined, either by default or by the user. In the present example, rules may specify that recordings of songs written by the songwriter may be mixed with the audio book, while other related audio (e.g., interviews with the songwriter) may be presented while the audio book is paused. As another example, a video clip may relate to the audio book. If the media device includes video presentation capabilities, a rule may specify that the video clip may be presented with its audio muted such that the audio book may continue to be presented during playback of the video clip.
Content may include both video and audio elements. The audio and video elements, collectively referred to herein as media elements or simply elements, may include elements such as audio tracks, video tracks, or other sensory elements of a movie, audio book, song, video game, or some other electronic content. As one example, a content item may be a sports broadcast that includes both audio elements (e.g., spoken commentary regarding the sporting event) and video elements (e.g., video clips or feeds of the sporting event).
While the sports broadcast is presented, or when presentation is initiated, references to related content, such as a local radio broadcast of the sporting event, may be obtained from a separate content management system or service. Rules can be implemented which allow for or require replacement of the sports broadcast's audio element with the local radio broadcast in the case of a live sports broadcast; otherwise, the rules may allow the media device to offer to pause the video programming to listen to the related audio.
In some cases, the initial content that the user wishes to consume, also referred to as the primary content, may include a video element or other video portion and the related content may also include a video element or other video portion. For example, when the user is watching a television program and there is a documentary related to a portion of the television program, rules may be applied in which the media device replaces the video element of what is currently being watched with the video element of the related content, but retain the audio element of the primary content whenever the primary content is news or “informational” programming. When the primary content is not news, rules may be applied in which video elements of both content items are presented simultaneously in a picture-in-picture (“PIP”) or split-screen format. If multiple displays are available, the two video elements may be presented substantially simultaneously on different displays. For example, the media device may control two or more video monitors, or the media device can display one video element on a video monitor and transmit the second video element, or information about the second video element, to a different device such as a tablet computer.
In some embodiments, a user may configure rules for individual content types or genres. For example, the user may configure rules directed to talk shows, news shows, live baseball games, and a specific television series such that any related video content may automatically override the video of the primary content (e.g., the video element of the related content is presented without any prompt to the user), but the related audio content is not presented (e.g., the audio element from the primary content item remains the only audio element that is presented). The user may configure separate rules for audio books, such that when the related content is music, the music is to be mixed or otherwise simultaneously presented with the audio book content, while related audio content that is not musical is suppressed and the audio book continues to be the only content item that is presented audibly. Alternatively, the user may configure rules such that the audio book is paused and the related audio content is automatically played or the user is prompted as to whether the related audio content is to be played.
In some embodiments, a supplemental or related content item may be associated with multiple portions of a primary content item. For example, a music clip may be associated with a position within a chapter of an audio book, and also associated with the end of the same chapter. In this case, a user may have multiple opportunities to hear the related item. In embodiments where associations of related content would cause the related item to be presented or made available multiple times during playback of the primary content, rules may be applied such that presentation of the related content or an indication of the availability of that related content may be suppressed after the first presentation of the related content or after the first indication that the related content is available. Under other rules, presentation or indication of related content may be allowed even after the related content has been fully played.
Related content may be accessed from a network-accessible content server. Alternatively, or in addition, related content may be stored on a media device or local storage component associated with a media device. Related content may be retrieved at substantially the same time as presentation of a primary content item is initiated. Alternatively, a media device may be configured to retrieve related content periodically. For example, a network enabled television may query a remote content server associated with related content every n seconds or minutes during presentation of the primary content item in order to determine whether new related content is available, particularly in the case of live or substantially live broadcasts. In still other embodiments, a content server or a content management system may be configured to notify a media device of available supplemental information.
In some embodiments, key portions of the primary content item may be designated such that the user is presented with those key portions regardless of any available related content. For example, crucial plot developments in a movie or audio book may be presented to the user regardless of any available related content item or rule that may be applicable at the time. Otherwise, the user may lose track of the development of the primary content item and the user's continued enjoyment of the primary content item may be jeopardized.
In some embodiments, a user may designate a related content item as a primary content item, thereby replacing the previous primary content item. For example, if the user is watching a broadcast of a sporting event and the score is such that the game is uninteresting, the user may wish to make a currently presented related content item (e.g., a documentary, currently being displayed in the secondary presentation window of a PIP display, regarding one of the teams) as the primary content item. In such cases, content that is related to the new primary content item—the documentary in this case—will be obtained, rather than content that related to the previous primary content item—the sporting event in this case.
Networked Content Consumption Environment
Prior to describing embodiments of the related content presentation and rule application process in detail, example networked content consumption environments in which the process may be implemented will be described.
At (2), a user may use a media device 102 to present a content item, such as an audio book. In this case, the audio book is considered the primary content item, because it is the content item for which the user initiated presentation. The media device 102 may be associated with a content management system 104 for obtaining references to related content items, or the primary content item itself may be associated with the content management system 104. In either case, the media device 102 can retrieve related content references from the content management system 104 at (3). The related content references may be obtained in bulk (e.g., all content references obtained in one transmission at or substantially close to initiation of primary content presentation), or they may be obtained during presentation of the primary content item.
At (4), the media device 102 may, during playback of the audio book, encounter a portion of the audio book that is associated with a reference to related content. In this example, the related content item may be a music clip hosted by the content server 106. The media device can therefore retrieve or stream the related content item from the content server 106 at (5). In some embodiments, the related content item may be hosted by the content management system 104 or a different content server, or it may be stored on the media device 102.
At (6), the media device 102 can apply presentation rules to the primary content item and the related content item (e.g., the audio book and the music clip) to determine how the related content item should be presented. The media device 102 may then present the related content item according to the rules. In the present example, the user may have configured a rule that music clips related to audio books may be mixed or overlaid with the audio of the audio book at the proper point within the audio book. However, if the related content was a different type of content (e.g., a related audio book), different rules may apply. For example, the applicable rules may specify that the user is to be prompted regarding presentation of the related content rather than automatically presenting both items simultaneously.
In some embodiments, related content may be hosted by and/or retrieved from multiple content servers, rather than from a single content server 106. The content management system 104 can determine content relationships between various content items hosted by multiple content servers, and the media device can retrieve related content from the appropriate content server 106.
Turning now to
At (1), the media device 102 can present a primary content item with embedded references to related content items. Returning to the example above, the media device may present an audio book. In some embodiments, the primary content may be associated with related content information that is not embedded into primary content item. The primary content item may be accessed from a storage device associated with the media device 102, from removable media, from a network-accessible content server, etc.
At (2), the media device 102 may encounter a portion of the primary content item that is associated with a related content item (e.g., a reference to a related content item, such as a music clip, is associated with the portion of the audio book). At (3), the media device 102 may retrieve the related content item from a content server 106a. The media device 102 may apply presentation rules and present the related content item, as described above and in detail below (e.g., the media device 102 may mix or overlay the audio of the audio book and the music clip for simultaneous presentation).
Subsequently, the media device may encounter a reference to a second related content item at (5). For example, the media device may encounter a reference to video footage of the subject of the audio book. The reference may indicate that the second related content item is hosted by a different content server 106b. In some embodiments, multiple content items may be hosted by and/or retrieved from a single content server as described above with respect to
At (6), the media device 102 can retrieve the second related content item from the content server 106b. The media device 102 can apply presentation rules regarding the second related content item at (7), and present some portion of or the entire second related content item as appropriate. For example, a rule may be applied such that because the primary content item is an audio book, video content may be presented with the audio element of the video content suppressed so that the audio may continue to be presented.
The media device 102 can correspond to a wide variety of electronic devices. In some embodiments, the media device 102 can include audio or visual output components, such as speakers or video screens. For example, the media device 102 may be a television, stereo, digital video recorder (DVRs), digital media receiver, set-top box, desktop computer, server computer or the like. In some embodiments, a media device 102 may also be a personal device, such as a smart phone, mobile media player, laptop computer or tablet computer that is optionally configured with a video display and/or an audio output component, or that may be connected to external displays, speakers, stereos, or the like. Some media devices 102 may include one or more processors and a storage or memory which may contain software applications executed by the processors. The software of the media device 102 may include modules or applications for establishing communications over the network 110. In addition, the software applications may include multimedia applications which play or otherwise execute audio programs such as music or audio books, video programs such as movies or television shows, and video games. The storage of the media devices 102 may also contain copies of content to play on the speakers or video screens.
As further shown in
The content management system 104 illustrated in
The content management system 104 can include a content processing module 140, a content library 142, and a related content data store 144. The content processing module 140 can process potential primary content items and determine whether supplemental or related content is available for portions of the primary content or for the primary content as a whole. For example, the content processing module can access content searches, data from content developers and providers, transcripts of content presentations, etc. The content management system 104 can store data regarding the relationships (e.g., an identifier of the primary content, an identifier of the related content, a network address of the related content, and data regarding the portion of the primary content to which the related content applies) in the related content data store 144. In some cases, the content management system 104 may store primary and/or related content in a content library 142. For example, a content developer may establish a content management system 104 that hosts primary content items in the content library 142 and includes predetermined references to related content in the related content data store 144.
The content servers 106a-106n can correspond to logical associations of one or more computing devices for hosting content and servicing requests for the hosted content over the network 110. For example, a content server 106a can include a web server component corresponding to one or more server computing devices for obtaining and processing requests for content (such as streaming video or audio) from the media device 102, the content management system 104, or other devices or service providers. In some embodiments, one or more content servers 106 may be associated with a CDN service provider, an application service provider, etc.
In some embodiments, the networked content consumption environment may include additional or fewer components that those illustrated in
Process for Rule-Based Presentation of Multiple Content Items
The process 300 begins at block 302. For example, if the process 300 is implemented by a media device 102, the process 300 may begin automatically upon power up or it may be manually initiated by a user wishing to consume content. The process 300 may be embodied in a set of executable program instructions stored on a computer-readable medium, such as one or more disk drives, of a computing system with which the media device 102 is associated. When the process 300 is initiated, the executable program instructions can be loaded into memory, such as RAM, and executed by one or more processors of the computing system. In some embodiments, the computing system may include multiple computing devices, such as servers, and the processes may be executed by multiple servers, serially or in parallel.
At block 304, the primary content item may be presented. As described above, the primary content may be any type of electronic content. The primary content may be streamed from a network-accessible server, loaded from a local storage device or removable media, etc. The primary content item may be associated with various related content items, and data regarding the related content items may be obtained as described above.
With continuing reference to
At decision block 308, the media device 102 may determine whether to present to the user an option, prompt or some other notification regarding the related content item. The media device 102 may store or access rules regarding which related content items and types to automatically present to the user. For example, if the related content item is a video and the primary content is a video, rules or policies may apply such that the user should be prompted rather than automatically pausing or overriding the video element of the primary content item 410. However, if the media device 102 is capable of presenting multiple video elements simultaneously (e.g., PIP or split-screen), then no user prompt may be needed.
In some embodiments, the media device 102 may not have information about the specific characteristics of the content item, such as whether it is has both video and audio elements or only an audio element. Rather, the information available to the media device 102 (e.g., embedded into the primary content item or received separate from the primary content item) may only indicate the location from which to obtain the related content item and/or an identifier of the related content item. In such cases, the media device 102 may prompt the user for a command regarding the related content item or passively notify the user that related content is available should the user wish to consume it. The related content item may be obtained from its source in order to determine its nature (e.g., whether it is video or audio content) in order to provide the user with additional information, or the nature of the related content item may be determinable from the reference itself (e.g., a file extension included in the reference).
In the present example, at time T1 the media device 102 may determine that no user notification should be presented regarding related content item 420. Such a determination may be based, for example, on a rule regarding the presentation of related content during a preliminary portion of a sporting event broadcast. Under different circumstances (e.g., after the sporting event has begun), different rules may apply and may specify that the user is to be notified of the related content item 420 rather than automatically presented with the related content item 420. In such cases the process 300 may proceed to block 314, where the related content item is retrieved, and subsequently to block 316, where presentation rules are applied.
The related content item 420 includes a video element. In addition, the primary content item 410, while also including a video element, is in a preliminary period (e.g., pregame). In such cases, the rules applied in block 316 may specify that the video element of both items is to be presented via PIP or split-screen display.
As seen in the content timeline of
As another example, during a subsequent iteration of the process 300 at time T2 the media device 102 may determine at block 306 that a second related content item 430—the local radio broadcast of the sporting event—is available. The rules applied at block 316 may indicate that related content of this nature (e.g., an audio-only local broadcast of the sporting event) overrides the audio element of the primary content item 410. As seen in
Returning to
As seen in
Turning now to
With respect to timeline 450, various attributes of audio narration 460 may be identified at various points in time (T1, T2, T3, T4 and T5), and related content items associated with the identified attributes may be selected. An attribute of primary content item 460 may include a person, place, object, event, activity, time period, or any other thing that is described, portrayed, mentioned, discussed, or referenced in the primary content item. An attribute may also relate to a contextual aspect of primary content item 460, and may include, e.g., a genre, mood, location, historical setting, language or time of day. The presence or relevance of an identified attribute in the primary content item 460 may not be limited to a single point in time. Rather, an identified attribute may continue to be relevant for a period of time following the point at which it is first identified, and it may become relevant repeatedly at various points thereafter. For example, an attribute such as a historical landmark may be mentioned briefly at the beginning of a travel article, discussed in detail for an extended duration in the middle of the article, and referred back to at the end of the article. In this example, the same attribute is identified at least three times within a single primary content item, and the relevance of the attribute continues for various periods of time at each point. Thus, any related content items associated with an identified attribute may be presented substantially within the various periods of time at which the attribute is relevant to the primary content item, but the precise timing may vary according to presentation rules.
The attributes of a primary content item may be identified by analyzing the primary content item itself, e.g., the audio narration in the case of primary content item 460, or by analyzing a textual representation of the primary content item, such as the magazine article that audio narration 460 is based on. The related content items 470-498 associated with the identified attributes of primary content item 460 might come from a source that is closely related to the primary content item, or it might come from an unrelated source. For example, the related content items depicted in
Referring now to timeline 450, at time T1 the audio narration 460 begins with a discussion of transportation options that a traveler might use to reach Paris. Based on this discussion, attributes such as transportation generally or a specific airline that flies to Paris may be identified. Related content associated with these attributes may then be selected. Related content items may include the text excerpt “Getting to Paris,” from the magazine article on which audio narration 460 is based, or image 490, showing an airplane belonging to the specific airline that was identified from the narration. At time T2, audio narration 460 may turn to a discussion of recommended accommodations in the Paris area. Based on this discussion, attributes such as accommodations generally or a specific recommended hotel may be identified, and related content such as the text “Where to Stay” or an image of a room from the recommended hotel may be selected. At time T3, discussion may turn to notable tourist sights in Paris, beginning with the Eiffel Tower. This discussion may continue through times T4 and T5 with discussion of Notre Dame and the Louvre, respectively. The discussion of sightseeing may include details about the sights themselves, as well as topics like navigating Paris by walking or public transportation. Accordingly, attributes such as Paris streets and Paris landscapes may be identified, along with the Eiffel Tower, Notre Dame and the Louvre. Related content items selected from these attributes may include text 474 and images 480-484, all from the magazine article on which audio narration 460 is based, along with map 494, audio clip 496, and image 498, all from other sources. Image 480 depicts the Eiffel Tower, image 482 depicts Notre Dame, and image 484 depicts the Louvre. Map 494 may depict the streets of Paris, including a highlighted route between each of the sights that correspond to identified attributes, obtained from a publicly accessible mapping service. Audio clip 496 may comprise a recording of the cathedral bells at Notre Dame, obtained from a library of stock recordings. Image 498 may comprise a photograph of a Parisian park, tagged with GPS location information indicating that it is in the vicinity of the Louvre, and obtained from a social media web site.
As described previously with respect to
The presentation rules that are applied to a primary content item and related content items may be configured by a user, as described below with respect to
Terminology
Depending on the embodiment, certain acts, events, or functions of any of the processes or algorithms described herein can be performed in a different sequence, can be added, merged, or left out altogether (e.g., not all described operations or events are necessary for the practice of the algorithm). Moreover, in certain embodiments, operations or events can be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors or processor cores or on other parallel architectures, rather than sequentially.
The various illustrative logical blocks, modules, routines, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. The described functionality can be implemented in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure.
The steps of a method, process, routine, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of a non-transitory computer-readable storage medium. An exemplary storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.
Conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without other input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list.
Conjunctive language such as the phrase “at least one of X, Y and Z,” unless specifically stated otherwise, is to be understood with the context as used in general to convey that an item, term, etc. may be either X, Y, or Z, or a combination thereof. Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of X, at least one of Y and at least one of Z to each be present.
While the above detailed description has shown, described, and pointed out novel features as applied to various embodiments, it can be understood that various omissions, substitutions, and changes in the form and details of the devices or algorithms illustrated can be made without departing from the spirit of the disclosure. As can be recognized, certain embodiments of the inventions described herein can be embodied within a form that does not provide all of the features and benefits set forth herein, as some features can be used or practiced separately from others. The scope of certain inventions disclosed herein is indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8699862 | Sharifi | Apr 2014 | B1 |
| 20010039571 | Atkinson | Nov 2001 | A1 |
| 20040090390 | Mason | May 2004 | A1 |
| 20040117825 | Watkins | Jun 2004 | A1 |
| 20070208766 | Malik | Sep 2007 | A1 |
| 20080082922 | Biniak | Apr 2008 | A1 |
| 20090024922 | Markowitz | Jan 2009 | A1 |
| 20090199117 | Yamamoto | Aug 2009 | A1 |
| 20100077433 | Beyabani | Mar 2010 | A1 |
| 20100205628 | Davis | Aug 2010 | A1 |
| 20110153047 | Cameron et al. | Jun 2011 | A1 |
| 20110261259 | Phillips | Oct 2011 | A1 |
| 20120221936 | Patterson et al. | Aug 2012 | A1 |
| 20120272185 | Dodson et al. | Oct 2012 | A1 |
| 20120303742 | Hines | Nov 2012 | A1 |
| 20130346906 | Farago | Dec 2013 | A1 |
| 20140304731 | Dhawan | Oct 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| 202502737 | Oct 2012 | CN |
| Entry |
|---|
| Cees et al., “Multimodal Video Indexing: A Review of the State-of-the-art,” Multimedia Tools and Application, Jan. 2005, vol. 25, Issue 1, pp. 5-35. |
