1. Field of the Invention
The present invention relates to multimedia content and more specifically to a system and method of providing a shared experience with multimedia content over a communication system.
2. Introduction
Many media devices such as VCRs, portable video recorders, cable set top boxes, CD players, DVD players and so forth, allow users to control the playback of multimedia content. For example, common control features include pause, slow motion, frame advance and repeat. Currently, these media devices are used as stand alone units, which limits their use to a small number of users at a single location. For example, many users will have a home theater in which a DVD player and a cable set top box are utilized at one location in which they can manipulate and manage their multimedia viewing. Often many users in their homes will have more than one television or cable set top box in different rooms and will desire, while in the middle of watching a multimedia selection, to move to another location. The current status of stand-alone multimedia units prevents an ease of use at multiple locations. In another scenario, there may be multiple people in multiple different locations, who each desire to watch the same multimedia presentation. There is a lack of ability which enables synchronization of these various presentations at the various locations. Accordingly, what is needed in the art is an improved system and method for enabling the sharing of multimedia content at different locations.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The features and advantages of the invention may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as set forth herein.
The present invention addresses the limitations set forth above and provides a system and method which allows multiple users to co-watch content at multiple locations and further provides multimedia real time communications among the users which can provide a rich shared experience for business or home use. The invention includes systems, methods and computer-readable media for controlling one or more computing devices which can enable the shared experience of multimedia content between multiple users at different locations. An example method embodiment of the invention includes a method of controlling a shared experience of multimedia content comprising scheduling a recording of a multimedia event at all group member's devices, recording the multimedia event and optionally analyzing differences between group member's devices and/or locations, generating a recording of the multimedia event such that it is ready for viewing by all group members and providing a shared viewing experience by distributing control using optimal real time communication and/or audio-video links. The principles of the present invention may be utilized in many media devices, such as digital video recorders, TiVo devices, cable set-top boxes provided by cable companies and so forth. Other local exchange carriers are moving into the multimedia space beyond just voice services as well as other content management and search engine providers such as Google and Yahoo.
Where commercial breaks and length of presentation differences occur in various time zones or geographically disparate areas, the method may include utilizing synchronization points, adding new content in time gaps at a computing device and/or modifying the length of the presentation to maintain synchronization among the computing devices in a group. This is preferably achieved by segmenting the multimedia data at each location by segments that are common to all groups, segments that not all participants have and/or default or insertion segments to fill in gaps where necessary.
Another aspect of the invention involves receiving audio from a microphone associated with at least one of the plurality of computing devices and transmitting the audio to each of the plurality of computing devices for playback on at least one surround sound speaker associated with at least one of the plurality of computing devices. This feature enables spatial delineation between sound from the multimedia presentation and sound from other participants in the group.
In order to describe the manner in which the above-recited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Various embodiments of the invention are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the invention.
With reference to
Although the exemplary environment described herein employs the hard disk, it should be appreciated by those skilled in the art that other types of computer readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital versatile disks, cartridges, random access memories (RAMs), read only memory (ROM), a cable or wireless signal containing a bit stream and the like, may also be used in the exemplary operating environment.
To enable user interaction with the computing device 100, an input device 190 represents any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth. The input may be used by the presenter to indicate the beginning of a speech search query. The device output 170 can also be one or more of a number of output means. In some instances, multimodal systems enable a user to provide multiple types of input to communicate with the computing device 100. The communications interface 180 generally governs and manages the user input and system output. There is no restriction on the invention operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.
For clarity of explanation, the illustrative embodiment of the present invention is presented as comprising individual functional blocks (including functional blocks labeled as a “processor”). The functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software. For example the functions of one or more processors presented in
The various embodiments of the invention generally relate to providing a shared experience with multimedia content over a communications system. An aspect of the invention relates to providing multiple media players or recording devices that are linked in a data network. The control of the media by a user at one device is distributed to all other devices within the group and thus the media playing at each location can be synchronized. The devices are also capable of participating in a multimedia conference while playing media content.
The concept disclosed herein links multiple computing devices in similar or different network types and provides user control from a single device that is then distributed to all devices such that the viewing of multimedia content is synchronized. Each computing device within the network of
There are many ways that the principles of the invention may be implemented, such as using digital video recorders linked in an IP wide area network. The control signal may be broadcast directly to all other devices in a peer-to-peer model or via the centralized server model of
The devices may also act as endpoints in a multiparty audio or audio-video conference. The conference audio may be mixed in with the multimedia presentation currently being watched. In this context, the video showing the parties watching the conference may be shown as a picture-in-picture image together with the presented multimedia content.
As an example, if there are three parties, each at their own location watching a multimedia presentation, each computing device in this location may have a camera that records and transmits the video of each respective party. The system presents in a corner or other location of the display of each of the parties one or more picture-in-picture windows wherein each window includes a realtime image of one of the respective parties that is watching the multimedia content. As shown in
In the case of content that is recorded from broadcast sources, the system may coordinate the recording at each device, taking into account time zone and scheduling differences. It can then schedule the earliest time when the shared experience session can take place. The system can then deploy analysis of the audio and video or both streams of information to detect any time shift due to variation of broadcast content by different providers to maintain synchronization. In other words, the control information shown in
The content may be recorded from a broadcast source. The system can coordinate the recording at each device taking into account time zone and broadcast scheduling differences. Thus, as is shown in
As another example of the synchronization that may occur, consider users at three different locations in which a prerecorded program was scheduled to play at 8:00 pm. However, if the three recording devices are in three different time zones, then the actual viewing of the program will occur at different times. Using the synchronization approach disclosed herein, a controlling system may instruct local recording devices to each record the scheduled program at a local time. Inasmuch as the program is stored, for example, locally on a digital video recorder, later when a synchronized viewing of the stored content is desired, the system may cause the multimedia playback to occur simultaneously, although the recording of the event occurred at disparate times. In this regard, an aspect of the invention involves performing an analysis on the broadcast program, the timing of the broadcast, the local scheduling according to members of the group, and so forth, such that the local recording of the broadcast content can be coordinated. Having synchronized the local recording of the content, the system can then control synchronized playback and control of the multimedia content with or without user controls disclosed herein.
As an example of when analysis of different broadcasts is desirable, assume that a broadcast event is recorded in two different time zones by two different content providers. In one time zone, a 2.5 minute commercial break occurs at 25 minutes into the program. In another time zone, a 3 minute commercial break occurs at 20 minutes. The analysis will include in this case finding the common audio/video content and the corresponding synchronization points. In a simple case, the synchronization of the media streams involves just time shifting the media rendering so that all the participants were watching the show simultaneously. If there were no commercial or other breaks in the multimedia presentation, then this simple aspect is accomplished (leaving aside the user remote control functions) by starting the multimedia streams simultaneously on the users' screens. However, the audio and/or video is preferably analyzed before the group watches the stream because of the possibility of different lengths of the commercial breaks embedded in the local recordings of the multimedia presentation. Based on the analysis, a simultaneous presentation of the recorded media event can occur at each location for members of the group.
Further, in some cases the overall length of the recorded program may also be different. For example, one time zone may cut out 30 seconds of the presentation in order to facilitate another commercial. The analysis of the different presentations may cause the system to also skip those 30 seconds when simultaneously presenting the multimedia event to the group or fill in those 30 seconds of time with alternate programming or a message to those members of the group who do not have those 30 seconds of time locally recorded. A commercial or advertising may be also sold and presented in this time gap or conferencing video may be inserted. Another aspect would be to transmit those 30 seconds from one local recording device in the one time zone to the other local recording device in advance of the simultaneous presentation such that all gaps are filled in for all members of the group.
Thus, where commercial breaks and length of presentation differences occur in various time zones or geographically disparate areas, the method may include utilizing synchronization points, adding new content in time gaps at a computing device and/or modifying the length of the presentation to maintain synchronization among the computing devices in a group.
One example of how the system manages synchronization is by segmenting each local recording into (1) segments that are common to all participants in the group and (2) unique segments that not everyone in the group has. With the segmentation, the system can then handle the various possible needs in multiple ways. For example, the system may concatenate all common segments to form a program common to all participants. Playback is then a shared experience and straightforward. The system may transfer the non-shared segments such that each local unit has all the segments and then concatenate them for a shared experience. Alternately, the system may have default segments (such as commercials or other data) that may be inserted from a central location or stored locally (and perhaps periodically updated) that are inserted where a local unit is missing units that are to be played on other devices.
Several exemplary implementations of the concept disclosed herein are provided. In a single host scenario, for example, it may be assumed that the user of computing device 402A is the single host in the context of
Also referenced above is the opportunity for a number of users to be co-hosts. In this context, in addition to the user of computing device 402A, for example, the user of computing device 402B may also be a host controller. In this regard, each of these users may have a remote control and whenever one of these users were to press a control button such that the viewing of the multimedia presentation would be altered by way of pausing slow motion and so forth, then the control from each of these users would cause the system to create a control signal that is sent to the other peer-to-peer or server/client computing devices such that the change is simultaneously viewed by the other users. There may be priorities of co-hosts such that a senior co-host's control signals may take priority over other co-hosts or there may be other types of negotiated selections of which control signals to implement in the case of conflicts.
In the context of democratic control, each user may be given host authority such that control selections may be implemented to all other group members in the group. There may be a number of optional features in which the system will determine what actions to take. For example, if there are ten group members and, under democratic control, six members decide to pause the multimedia program whereas a minority of members selects fast forward, the multimedia program, then under democratic control, may operate to analyze the various input from the various members such that the control feature selected by a majority of the group members may be implemented.
Other control analysis and options may be implemented as well. For example, over time each member may be granted more authority or be able to gain or lose their authority in the context of who gets to control the presentation of the multimedia program. For example, users may gain more authority the longer they are a part of the group. Thus, a user that has been in the group for two years when they implement a control of the multimedia program they may have their input count for ten points where new users may have their input count for one point and thus the system may analyze the various user input and implement the input that gains more than 5 points which may be the input of six users or it may be the input of 70 first year users such that the appropriate threshold is passed and the particular input is implemented in a synchronized manner to each group member.
Also as introduced above, in addition to providing a simultaneous control of the presentation of multimedia content may also be used in the context of an audio/video conference. For example, if the group members are each watching a movie and the computing devices also have a microphone and camera connected to them, each user may be able to provide audio or commentary on the multimedia presentation and the audio may be mixed with the audio of the content being watched. Furthermore, the audio that is spoken by users at the various computing devices, in addition to being mixed, may also be assigned a particular channel. For example, in a conference room or in a home theater there may be a 7.1 audio system with one sub-woofer and surround sound speakers and the audio from various players may be assigned to one or more of the speakers such that that audio comes from behind or from the side of the user rather than from the front. For example, a virtual geographic configuration may be presented such that, for example, using
Appropriate adjustments may be made to the quality of such transmitted video inasmuch as, in the preferred embodiment, the multimedia presentation is stored locally rather than transmitted over a possibly bandwidth limited network. The picture-in-picture information, as part of video conference, will in realtime be transmitted over a network to the other users within the group. Accordingly, in addition to modifying the video conferencing information that is transmitted over a network to other members of the group, an analysis may be performed of that content and appropriate adjustments may be made to the multimedia presentation such that synchronization of both the multimedia presentation and the video conferencing information as well as the audio conferencing information may be maintained.
Also as noted above, in the server/client context there may be other adjustments that may be made to maintain synchronization. For example, if the system determines that the media bandwidth may have problems in enabling synchronized viewing of multimedia content, the system that is responsible for distributing content from a device such as a server or from one device to another in a shared experience session may begin distribution in advance in scenarios such as where the media bandwidth is greater than the connection bandwidth. The difference in the bandwidth of a multimedia connection may also vary from user to user and thus a network analysis associated with each individual user may be performed such that distribution may begin ahead of time or at varying intervals on a per user basis.
A service provider of the invention disclosed herein may also a variable fee scheme such that an additional subscription fee or per use fee may be implemented for the capability of controlling of the synchronization of a multimedia presentation and/or audio/video conferencing feature as disclosed herein. For example, in one aspect of the invention, a particular user may be able to increase their control in, for example, a democratic control context by paying more money to enable their control choices to have more impact in the context of control decisions of other users.
Embodiments within the scope of the present invention may also include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or combination thereof) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of the computer-readable media.
Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Computer-executable instructions also include program modules that are executed by computers in stand-alone or network environments. Generally, program modules include routines, programs, objects, components, and data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.
Those of skill in the art will appreciate that other embodiments of the invention may be practiced in network computing environments with many types of computer system configurations, including personal computers, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Embodiments may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination thereof) through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Although the above description may contain specific details, they should not be construed as limiting the claims in any way. Other configurations of the described embodiments of the invention are part of the scope of this invention. For example, the principles disclosed herein are applicable to any networked set of computing devices. Thus, the invention may apply to such arrangements as Internet gaming. Accordingly, the appended claims and their legal equivalents should only define the invention, rather than any specific examples given.
The present application is a continuation of U.S. patent application Ser. No. 11/685,509, filed Mar. 13, 2007, the content of which is incorporated herein by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
7317791 | Carlson | Jan 2008 | B2 |
7424545 | Ducheneaut et al. | Sep 2008 | B2 |
7620659 | Novik et al. | Nov 2009 | B2 |
7669219 | Scott, III | Feb 2010 | B2 |
7716376 | Price et al. | May 2010 | B1 |
20010044835 | Schober et al. | Nov 2001 | A1 |
20040236832 | Morris et al. | Nov 2004 | A1 |
20060242309 | Damick et al. | Oct 2006 | A1 |
20070255788 | Troung | Nov 2007 | A1 |
20070271338 | Anschutz | Nov 2007 | A1 |
20080209021 | Shamma | Aug 2008 | A1 |
20090019374 | Logan et al. | Jan 2009 | A1 |
20090228911 | Vrijsen | Sep 2009 | A1 |
20100138512 | Morris et al. | Jun 2010 | A1 |
Number | Date | Country | |
---|---|---|---|
20140258411 A1 | Sep 2014 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11685509 | Mar 2007 | US |
Child | 14284986 | US |