Patent Application
20170094354
The present disclosure relates to multimedia processing. More specifically, it relates to methods and systems for managing content distribution and gating content in a multimedia stream.
Streaming media assets such as audio and/or video streams from a remote server to a client device over a communication network is an increasingly popular way for retrieving and viewing multimedia. In one aspect, a media asset may include an individual multimedia file. In another aspect, a media asset may include a media channel having several multimedia files, which may be rendered in sequence. Media content can be streamed for both Video On Demand (VOD), live TV (LIVE) streaming services, and the like. HTTP Live Streaming (HLS) is one such protocol for transferring streams of media data for such services. Media assets may be retrieved by a client device from one or more distribution servers.
Advertisements may be distributed with the media assets to monetize the content. In traditional broadcasting formats, an “advertisement spot” corresponding to a short time during broadcasting is typically sold to an advertisement provider. VOD and LIVE streaming allows for content to be adapted to audiences more dynamically compared with traditional broadcasting formats. For streaming video, navigation such as playback, fast forwarding, and the like may be conditioned upon viewing at least a portion of an advertisement.
Viewing habits associated with VOD, LIVE streaming, and the like may differ from traditional consumption of multimedia. For example, a user of VOD and LIVE streaming may pause and resume at a later time sometimes on a different device. A user may also provide detailed information about interests, usage habits, etc. based on a client device or user profile. Thus, advertising content for VOD and LIVE streams may be distributed in more sophisticated manners. The inventors perceive a need in the art for collecting information and controlling media playback based on advertisement providers' interest in knowing and/or selecting a most effective advertisement, where advertisements appear relative to the remainder of the content, and when an advertisement has been viewed or otherwise consumed.
Embodiments of the present disclosure provide techniques for managing playback of media items in a computer system. A media item may include metadata that identifies portions of the media item as “gates,” that govern access to other portions of the media item, called “scopes.” When playback reaches a scope portion, a player may determine a state of its associated gate(s). If the gate is locked, the scope portion cannot be played until the gate is unlocked. When playback reaches a gate portion, the player may determine its state and either play the gate or skip playback of the gate depending on its state and definitions supplied for the gate. Gates may be collected into larger constructs, called “pools,” which may be locked and unlocked based on definitions supplied for the pools. The present disclosure describes a syntax from which content providers may define controls for the gates and scopes.
The media source 110 may include a server or network of servers (not shown) and a storage system 112 that stores one or more media items 114. The media items 114 represent content that can be downloaded and displayed by a client device 120. A media item 114 may include a manifest file 116, and coded video streams STR1, STR2, and STR3. The different streams STR1-STR3 may represent content of a common media item that has been coded according to different coding techniques. The streams STR1-STR3 may differ based on, for example, the protocol used to code the media item, the network data rates that are necessary to convey the streams between the media source 110 and a client 120, the frame sizes and/or frame rates of the video that each stream STR1-STR3 supports or the types of client device (e.g., smart phone vs. tablet computer vs. large screen projector) that the streams STR1-STR3 are intended to support. Typically, each stream may be stored as a plurality of individually-retrievable segments (e.g. STR1 as segments 117.1-117.N, STR2 as segments 118.1-118.N, etc.).
The manifest file 116 may store data describing the different streams that are available for retrieval. The manifest file 116 may describe parameters of the streams STR1-STR3 and network locations from which the corresponding segments 117.1-117.N, 118.1-118.N, and 119.1-119.N may be retrieved. In an embodiment, the manifest file 116 may store data defining gates as discussed herein.
The architecture of the media items 114 may be used both for pre-produced video that remains persistently stored at a media source and also for “live” streaming data, media that is captured, coded and distributed within the system 100 on a real time basis. For pre-produced data, a manifest file 116 and streams STR1-STR3 may be coded and stored by the storage system 112 with little or no alteration. For live streaming data, contents of the manifest file 116 and the streams STR1-STR3 may be provided on a rolling basis. The media source 110 may store data representing a predetermined portion of media (say a most recent 10 minute portions of the content), which is replenished on an ongoing basis and from which older portions of the content may be evicted. For the purposes of the present discussion, any distinction between pre-produced content and live streaming content is immaterial unless noted herein.
In an embodiment, the media source 110 may store a playlist 150 that identifies the media items that are available for download. In a simple implementation, a playlist 150 may identify media items 114 individually as single pieces of media content for consumption, say a video short. The principles of the present disclosure also find application with content applications that aggregate several media items into larger collections for consumption, for example, as a television program that integrates a video short with other media items. In such a case, the playlist 150 may identify the collection as single item of media content that, when played, would cause retrieval of several media items such as item 114. The principles of the present disclosure find application with such media items as well. In such an embodiment, metadata that defines gates may be provided within the playlist 150 rather than in the manifest file 116 of media items.
The client device 120 may include a media player to download, decode and display streaming media from the distribution server 110. As such, the client device 120 may include a video decoder (not shown) that decodes coded video data and a display device. The client device 120 also may include a processing system that may interpret gate definitions provided in metadata that accompanies a media item and applies access controls among portions of the media item as determined by those definitions. Although the client device 120 is illustrated as a tablet computer, it may be provided as a variety of computing platforms, including servers, personal computers, laptop computers, smart phones, media players and/or dedicated video conferencing equipment.
The network 130 represents any number of networks that convey coded video data among the media source 110 and client device 120, including, for example, wireline and/or wireless communication networks. A communication network 130 may exchange data in circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the purposes of the present discussion, the architecture and topology of the network 130 are immaterial to the present disclosure unless discussed hereinbelow.
Generally speaking, a gate 210 may govern access to a corresponding scope 215 within a media item. Thus, in an example where advertisements are provided as gates, a player may prevent playback of a given scope (say, scope 235) until the access requirements defined by its corresponding gate 230 are met. For example, the gate 230 may indicate that an advertisement must be played before playback may reach the scope 235.
In one embodiment, a gate may have one of two states: locked or unlocked. When a gate 230 is locked, its associated scope 235 may not be played. Such a gate 230 may become unlocked when its media is played. Thereafter, the media of scope 235 is played. Thus, locked gates effectively prevent a player from jumping around content of the gate and playing only the content of the scopes 215, 225, 235 within a media item.
In an embodiment, an unlocked gate may be replayed if it is encountered during playback. For example,
In another embodiment, gates may have three states: locked, unlocked and collapsed. As before, locked gates prevent access to an associated scope and unlocked gates permit access to an associated scope. A collapsed gate also may permit access to an associated scope. Collapsed gates differ from unlocked gates, however, in the way they may be handled on repeat plays. An unlocked gate may be replayed if playback reaches it a second time whereas a collapsed gate need not be replayed.
The method 400 of
Embodiments of the present disclosure provide a suite of tools that allows content publishers to define gates with a variety of different access parameters. These tools may permit content developers to define, for example, which elements in a media timeline are controlled by a gate, whether a gate must be played anew each time its scope is to be played, and how to handle trickplay events (e.g., scrubbing/scanning along a timeline, jumps between locations of the timeline, etc.).
Table 1 illustrates a set of attributes that may be used to define a gate in an embodiment of the present invention. Gates may be defined in metadata that accompanies a media item.
The example of Table 1 illustrates syntactic elements using JavaScript Object Notation but such elements may employ alternate notations, if desired for a given application.
As shown above, gates may be defined by their start times and durations. The start times may be expressed either as an offset into the media item's timeline, which may be convenient for pre-produced media items, or with reference to a defined timing reference within a network, which may be convenient for live streamed data. For example, several networking protocols define clock references for networked elements, including the Network Time Protocol (NTP), IEEE 1588 and IEEE 802.1 as standards. Gates for live streamed data may be defined with reference to times defined by any of these standards.
The following examples may help to explain the syntactic elements illustrated in Table 1.
In this example, the onset of the first gate is defined to begin at the onset of the media item (time=0.0 sec) and have a duration of 30.0 seconds. The onset of the second gate is defined to begin 10 minutes into the media item (time=600.0 sec) and also have a duration of 30.0 seconds. The second gate is defined to unlock at 15 seconds, which means it will unlock its corresponding scope if the gate is played for 15 seconds, even if an operator jumps out of the gate at some point thereafter. In an alternative embodiment, a scope may be controlled by more than one gate. Accessing the scope may be conditioned upon playback of the controlling gates. For example, playback may proceed from a first gate to a second gate before jumping to content of the associated scope.
In this example, the gate-duration definition precedes the timeline marker, which may indicate that the gate-duration is a global parameter that can be applied to all gates within the timeline, unless overridden by an occurrence of gate-duration in another gate definition. In example 1, the gate-duration value appeared for each of the gate definitions in the body of the timeline. Those gate-duration definitions, therefore, applied only to the gates to which they are associated.
In example 2, each instance of the gate-date field identifies a network time on which the respective gates begin. This embodiment finds application in a live streaming implementation where gates are applied to scopes that are distributed as they are produced.
In example 3, all gates are defined to have a default duration of 30 seconds but the second gate includes a gate-duration (15 seconds) that overrides the default duration. Moreover, all gates are defined to be collapsible when they are unlocked. As in
In example 4, all gates are defined to have a default duration of 30 seconds. The gate-unlocks-at value is set to 1000 seconds, which causes the gates to unlock once they are played for 1000 seconds or at their conclusion, whichever is shorter. In this example, because the default duration is 30 seconds, the gates will unlock at 30 seconds. In one aspect, it may be advantageous to set the gate-unlocks-at value higher than the gate-duration value if a largest duration of that a subsequent gate is unknown or uncertain.
Example 4 also employs a pool attribute, which allows content providers to define relationships among gates assigned to a common pool. For example, for any pool that is unlocked, all gates in the pool may be treated as unlocked, regardless of their per-gate rule evaluation. Similarly, for any pool that is collapsed, all gates in the pool may be treated as collapsed, regardless of their per-gate rule evaluation. For any pool that is locked, each gate in the pool obeys its own lock evaluation.
In Example 4, the pool-skip-value is set to 3, which causes all gates in the pool to be unlocked once three gates have been unlocked. Since the gate-unlocks-at value is set higher than the gate-duration in this example, the pool will be unlocked once three gates have been played in their entirety.
In this example, the gate-grace-period is set to 5 seconds. This field indicates that the gates will be unlocked until a content-played-since-gate reaches 5 seconds, which is reached when the player is in a normal playback mode (e.g., not fast-forward or fast-reverse) and has gone at least 5 seconds since playing a gate. This mode permits viewers to preview a portion of a scope before requiring playback of a gate.
Gate definitions also may constrain a client's response to user playback commands.
For example, the “gate-allows-if,” “gate-allows-rw,” “gate-allows-jump-forward,” “gate-allows-jump-backward” and “gate-allows-pause” parameters may dictate to a client its response when a user attempts to terminate ordinary playback in favor of an alternate viewing mode while playing content of a gate. Similarly, the “if-scan-forward,” “if-scan-backward,” “if-jump-ahead” and “if-jump-behind” parameters may dictate to a client its response when a user attempts to engage alternate viewing modes while playing content of a scope.
Gate definitions also may constrain a client's response to user playback commands. For example, the “gate-allows-if,” “gate-allows-rw,” “gate-allows-jump-forward,” “gate-allows-jump-backward” and “gate-allows-pause” parameters may dictate to a client its response when a user attempts to terminate ordinary playback in favor of an alternate viewing mode while playing content of a gate. Similarly, the “if-scan-forward,” “if-scan-backward,” “if-jump-ahead” and “if-jump-behind” parameters may dictate to a client its response when a user attempts to engage alternate viewing modes while playing content of a scope.
In an embodiment, a client, e.g., a playback service, may develop a state record that reflects its usage of the gates within a media item. For example, for each gate that is played-through, the client may store data identifying the maximum gate duration the user has watched and update the state record. Thus, individual gates in a media item may be marked with a gate-id and the state record may associate each gate's usage with these gate-ids.
In an embodiment, a state record may have a layout as follows:
In implementation, the exact form of a state record will be determined by the gate definitions that are provided for a media item.
Returning to
The principles of the present disclosure also find application in distribution systems that employ proxy caches and the like. Although not illustrated in
The principles of the present disclosure, of course, find application with distribution systems in which a media item 114 is a unitary file (not shown), without use of streams as illustrated in
The display 530 may include a display device and associated driver circuitry, including graphics processors and associated memories (not shown). The display 530 may output a user interface 532 that is generated by the processor 510, which may include various user controls 534 for altering playback, including trickplay modes such as scrubbing, jumps, fast forward and fast reverse playback. As discussed, the client device 500 may disable various playback modes as dictated by gate definitions. In doing so, the client device 500 may alter the user interface to present or remove select playback controls depending on which playback modes are authorized or disabled by the gate definitions.
The input/output system 540 may represent a suite of controls through which an operator may enter commands to the device. The client device 520 may include an array of buttons, keyboard, mice and/or touchscreen devices for entry of such commands. The nature of such input device may be tailored to fit the type of client device 500 in which the principles of the present disclosure are applied. For example, as illustrated in the user interface 532 of
The transceiver 550 may support communication between the client device 500 and a communication network (
Although
Although the foregoing description includes several exemplary embodiments, it is understood that the words that have been used are words of description and illustration, rather than words of limitation. Changes may be made within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of the disclosure in its aspects. Although the disclosure has been described with reference to particular means, materials and embodiments, the disclosure is not intended to be limited to the particulars disclosed; rather the disclosure extends to all functionally equivalent structures, methods, and uses such as are within the scope of the appended claims.
As used in the appended claims, the term “computer-readable medium” may include a single medium or multiple media, such as a centralized or distributed database, and/or associated caches and servers that store one or more sets of instructions. The term shall also include any medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor or that cause a computer system to perform any one or more of the embodiments disclosed herein.
The computer-readable medium may comprise a non-transitory computer-readable medium or media and/or comprise a transitory computer-readable medium or media. In a particular non-limiting, exemplary embodiment, the computer-readable medium may include a solid-state memory such as a memory card or other package that houses one or more non-volatile read-only memories. Further, the computer-readable medium may be a random access memory or other volatile re-writable memory. Additionally, the computer-readable medium may include a magneto-optical or optical medium, such as a disk or tapes or other storage device to capture carrier wave signals such as a signal communicated over a transmission medium. Accordingly, the disclosure is considered to include any computer-readable medium or other equivalents and successor media, in which data or instructions may be stored.
The present specification describes components and functions that may be implemented in particular embodiments which may operate in accordance with one or more particular standards and protocols. However, the disclosure is not limited to such standards and protocols. Such standards periodically may be superseded by faster or more efficient equivalents having essentially the same functions. Accordingly, replacement standards and protocols having the same or similar functions are considered equivalents thereof.
The illustrations of the embodiments described herein are intended to provide a general understanding of the various embodiments. The illustrations are not intended to serve as a complete description of all of the elements and features of apparatus and systems that utilize the structures or methods described herein. Many other embodiments may be apparent to those of skill in the art upon reviewing the disclosure. Other embodiments may be utilized and derived from the disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. Additionally, the illustrations are merely representational and may not be drawn to scale. Certain proportions within the illustrations may be exaggerated, while other proportions may be minimized. Accordingly, the disclosure and the figures are to be regarded as illustrative rather than restrictive.
For example, operation of the disclosed embodiments has been described in the context of servers and terminals that implement video compression, coding, and decoding. These systems can be embodied in electronic devices or integrated circuits, such as application specific integrated circuits, field programmable gate arrays and/or digital signal processors. Alternatively, they can be embodied in computer programs that execute on personal computers, notebook computers, tablets, smartphones or computer servers. Such computer programs typically are stored in physical storage media such as electronic-, magnetic- and/or optically-based storage devices, where they may be read to a processor, under control of an operating system and executed. And, of course, these components may be provided as hybrid systems that distribute functionality across dedicated hardware components and programmed general-purpose processors, as desired.
In addition, in the foregoing Detailed Description, various features may be grouped or described together the purpose of streamlining the disclosure. This disclosure is not to be interpreted as reflecting an intention that all such features are required to provide an operable embodiment, nor that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, subject matter may be directed to less than all of the features of any of the disclosed embodiments. Thus, the following claims are incorporated into the Detailed Description, with each claim standing on its own as defining separately claimed subject matter.
Also, where certain claims recite methods, sequence of recitation of a particular method in a claim does not require that that sequence is essential to an operable claim. Rather, particular method elements or steps could be executed in different orders without departing from the scope or spirit of the invention.
