Patent Application
20130124664
The present invention is related generally to data-delivery systems and, more particularly, to systems that send or receive media presentations.
More and more users are downloading more and more media presentations to more and more devices. (Here, “media presentations” generally include just about any kind of digital content, and, more specifically, sound, video, and interactive files.) These media presentations are often enormous, and downloading them can consume a significant amount of available bandwidth and battery power on the user's device.
In order to manage download requests, download servers often divide a large media presentation into consecutive “chunks” where each chunk represents, for example, a few seconds of video. When a user wishes to consume a media presentation, his device begins by requesting a “playlist” for the presentation from the download server. (Note that here “consume” is meant as a general term for any type of human interaction with a medium. It can include watching television, listening to radio, playing a computer game, talking or texting on a telephone, interacting with a web site, and the like. To simplify the present discussion, a media consumer is generally called a “user” or a “viewer,” even when his medium of choice does not have a visual portion.) The playlist includes a list of descriptions of the chunks into which the presentation is segmented on that server (including alternative resolutions). With the playlist in hand, the user's device asks the server to download the first chunk of the presentation. While the user is viewing the first chunk, his device attempts to “keep ahead” of the user's viewing (and thus avoid “video freeze”) by requesting subsequent chunks of the presentation. The chunks are received and buffered on the user's device so that the user can continue to view the media presentation while subsequent chunks are still being delivered.
Now consider the situation where a small number of people, say a group of friends, wishes to share the experience of simultaneously watching a video presentation in the same room, but each person wants to watch on his own device. (The room might not have a large-screen television, for instance.) The chunked-download model described above does not work so well in this case. While each person can download chunks of the presentation in the manner discussed above, the separate devices can have different download characteristics which will alter exactly when the next chunk becomes available for playing on a given device. Also, the download requests made by one device will compete with the requests of another device. The net result is that each person may be watching at a slightly different point in the presentation. This lack of shared timing can ruin the shared experience when, say, the friends are watching a soccer match together and one person reacts loudly to an exciting goal while another person still has a few seconds to watch before he can see the goal.
The above considerations, and others, are addressed by the present invention, which can be understood by referring to the specification, drawings, and claims. In a chunked-download environment, according to the present invention, a group of “servant” devices (e.g., smart phones or tablet computers) each downloads a media presentation and renders the presentation to a local user. However, a “master” device directs the servants as to which chunks to download and when to render the presentation. In this way, the master keeps all of the presentations on all of the separate servants in synchrony.
In some embodiments, the master is a peer of the servants, and the group of them may choose which is to be the master. In other embodiments, the master is a network server dedicated to supporting synchronized playback.
The master can request status reports from the servants including, for example, buffer status, download progress, reception quality, and the like. The master uses this information to coordinate the downloads of the servants, which may have very different download capabilities. If, for example, one servant is having a difficult time keeping up with the presentation (as reflected in the status reports it sends to the master), then the master may choose to direct that servant to download a reduced-resolution version of the presentation. This version requires less bandwidth and may thus allow this servant to keep up with the others.
If the user of one of the servant devices enters a playback command (e.g., play, stop, pause, fast-forward, skip), then the command is not executed directly on the servant device but is instead sent to the master device. The master then sends the command to all of the servants so that they may execute the playback command in synchrony.
While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:
a and 4b together form a flowchart of a method for a master device to coordinate servant devices in receiving and rendering a media presentation in the representational environment of
Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein.
Aspects of the present invention may be practiced in the representative communications environment 100 of
The web server icon 110 in
(Note that the flowcharts are primarily intended to support the following discussion. The “steps” in the flowcharts are, in some embodiments and in some situations, optional and may be performed in a different order, if at all.)
However the media presentation is selected, step 300 optionally chooses one device, either an end-user device 104, 108 or the server 110, to be the master that coordinates the presentation. If the end-user device 104 is chosen as the master, then, because it also serves as the servant for its participant 102, the device 104 performs the methods of both
In step 302, each participating servant device 104 receives from the master a command to download the playlist for the chosen media presentation. (The playlist may also be called a “manifest” or a “media-presentation description.”) The playlist contains information (such as the number of chunks, playing time duration of each chunk, supported resolutions, and the like) about the chosen media presentation. Note that, in general, the servant device 104, 108 does not request the playlist from the master: Instead, the playlist is requested from a download server 110. Different servant devices 104, 108 may choose to request the playlist from different download servers 110 to avoid contention.
Under the direction of the master, the participating servant device 104 receives a download command in step 304 and performs that command in step 306. There are several types of download commands. The most common specifies which chunks the end-user device 104 should request from a download server 110. The download command could also specify what resolution to download, when to download, and an identification of where within a downloaded chunk the servant device 104 should begin decoding the chunk or queuing the chunk for playback. If the download command includes a value for the resolution, then the servant device 104 can interpret this value as the maximum resolution to download, because the servant device 104 may decide that it needs to download at a lower resolution than specified.
(Note: There is some confusion in the art about the meaning of a “chunk” that is relevant here. Sometimes, a “chunk” is equated with a given time segment of a video presentation, regardless of the coding resolution of that time segment. That is to say, the first two-second segment is a “chunk” that can be encoded at different resolutions. Other times, each resolution of that first two-second segment is considered to be a different “chunk.” In the present discussion, the master generally decides both which chunk to download next and at what resolution, so the distinction is meaningless to the servant device 104.)
The servant device 104 may receive a playback command from the master in step 308. The master sends these commands to synchronize the playback on the separate participating servants 104, 108. In addition to the straightforward play, stop, and pause commands, the master could send a playback command for a “trick play” such as fast-forward, rewind, skip to a specified place in the media presentation, play in slow motion, highlight, and play an alternate audio track. As discussed below, the master in some embodiments monitors the buffer status of all of the participating servant devices 104, 108, so that the master knows that all of the servants 104, 108 are ready to respond to the playback command. In this way, the master synchronizes the disparate playbacks of the servants 104, 108.
In step 310, the servant device 104 also receives a playback command, but in this case the command is from its local user 102. The user 102 may enter the same types of commands (e.g., play, fast-forward) that the servant device 104 receives from the master in step 308. However, in this case the servant device 104 does not directly perform the command. Instead, it sends the command to the master device. Discussed below is what the master does with the received playback command.
a and 4b present a method for a master device, whether the master is an end-user device 104, 108 or a web server 110. Many of the steps in this method are the other end of the communications described above in relation to the servant end-user device 104 and thus need only a brief discussion here.
In optional step 400 (as in step 300 of
As appropriate, in step 402 the master tells the servant end-user devices 104, 108 to download the playlist of the chosen media presentation. (See the discussion of step 302 of
In step 404, the master tells each servant device 104, 108 to download some of the media presentation (step 304 of
A servant 104 may also send status information to the master without being queried. For example, the servant 104 may experience a “video freeze” when its playback reaches the end of the amount of the presentation that it has already downloaded. The servant 104 tells the master of this fact. The master may choose to send a “pause” playback command to all servants 104, 108 to allow the servant 104 to catch up with its downloading. The master may also lower the resolution of the downloads if a lack of bandwidth seems to be the problem. When all of the servants 104, 108 are ready to proceed, the master can issue a “play” command.
In another alternative, the master determines the status of the servant 104 by passively monitoring the download environment. For example, by “sniffing” the air (that is, by examining download requests and responses that the master reads on the airwaves), the master can see that the servant 104 has not yet begun to receive a chunk that is scheduled for playing very soon. The master can respond to this situation, by, for example, querying the servant 104 for a more detailed status report, or sending a particular download command to the servant 104, or even pausing all servants 104, 108 until the servant 104 has received that chunk.
In step 408 (see also step 308 of
Step 410 emphasizes the fact that the master device may also be playing the media presentation to its local user, that is, the master can also be its own servant. In that case, the master also performs the essentials of the method of
In step 412 of
If the master receives conflicting commands in step 412, then it can decide which, if any, to implement. It might also prevent confusion by ignoring a command that comes too closely after another. It is anticipated that users will become accustomed to the fact that not all of their commands can be immediately performed in this shared-presentation environment.
Some embodiments allow a servant to join the group after the playback has begun. This new servant notifies the master of its intent, and the master calculates when that servant will have enough material downloaded to begin playing back in synchrony with the existing members of the group. The master then sends download commands (step 404) to the joining servant device and, when the new servant is ready, sends it a playback command synchronizing it with the other servants 104, 108 (step 408).
In view of the many possible embodiments to which the principles of the present invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the invention. For example, the master can choose to use any of various communications protocols in synchronizing the playback of the servant devices. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.
