The present system includes methods and apparatus for controlling the presentation of audio and video information (A/V information) to individuals at an A/V Display.
When the standard A/V signals are to be displayed, a viewer may exercise control over the set-top-box and the A/V display in much the same way that a cable TV or Satellite TV viewer operates his or her TV system. Set-top-box 103 has additional capabilities. For example, ancillary sources of A/V information 107 such as sound microphones and video cameras may be connected as signal inputs to the set-top-box 103 and selected for presentation.
Set-top-box 103 is also used to insert substitute programming into the video stream being sent to the Display 101. For example, by agreement with the standard video providers, an advertisement for dietary supplements might be substituted for an advertisement for snack foods in an exercise establishment. In order to perform such substitution the set-top-box must have access to the substitute A/V signal and it must be informed where and when to play the substitute A/V signal. One source of substitute signals is the normal video stream running from the Distribution system 105 to the set-top-box 103. One or more of the video “channels” or “streams” running from the Distribution system 105 may convey substitute A/V signals. Advantageously, these may be recorded within the set-top-box for later play back to the A/V display. Also, the substitute signals may be received at the set-top-box 103 via a network connection 109 which in
As an additional base for features, the set-top-box 103 is connected to the cellular telephone and text messaging network via the internet 111 and wireless servers 117. A cell phone 119 can then be used, as described below to interact with software programs run on the set-top-box 103 and the A/V signals presented at display device 101.
The set-top-box is capable of receiving a plurality of possible standard video signals from a plurality of different sources such as over the air, Cable TV and Satellite TV. In one embodiment the set-top-box includes one or more receivers, each for one of the major video sources such as cable TV and satellite TV. In a preferred embodiment receivers for these signal sources are provided separately from the set-top-box 103 and connected to the set-top-box by a path for video signals such as DVI or S-Video.
CPU 145 controls the operation of set-top-box 103 in response to programs and data stored in a memory 157. CPU 145 communicates with memory 157 via north PCI bridge 159 which is also connected to exchange information with the south bridge circuit 151 and arrangements connected to it. Through the south bridge circuit 151, CPU 145 also communicates with a DVD device 161, a hard drive 163 and a digital input-output multiplexor 165. An assortment of digital inputs including Keyboard, USB, fire wire, RS-232 and Ethernet, which are represented at 133, are available to exchange information with the set-top-box 103. CPU 145 identifies which of the plurality of possible A/V sources is to be played through to its connected A/V display 101 and controls the involved multiplexor and bridges needed to complete the connection. For example, when a cable receiver output on S-video 131 is to be connected to the display 101, multiplexer 147 is controlled to connect the S-Video input to the south bridge 151 via the ADC 149. The south bridge 151 in turn provides a connection, via one of two MPEG processor P137, VRAM 127 to graphics processor 123 which outputs the video signal to A/V display 101. In the preceding example the communication medium within set-top-box 103 was stated to include PCI bridges such as 151, 159 and 124. Other embodiments may employ other well known arrangements for communicating digital data at high data rates. Additionally, other high speed manipulation devices may be used to provide the functions of MPEG processors.
Digital signals for controlling the functions of set-top-box 103 can be received from any of the inputs 133. In the present example, the Ethernet input is connected to exchange control information and data with the internet 171.
When a given A/V display e.g.,101a is to display a “channel”, the associated distribution receiver e.g., 183 is set up to select a channel in the well known manner and the set-top-box 103a is controlled to pass the received A/V signal on to its connected display 101a. When the channel is to be changed the distribution receiver e.g., 183 is controlled to select a new channel which is passed on to the display 101a by the set-top-box 103a. All of the set-top-boxes are capable of substituting a secondary A/V signal for the regular A/V signal conveyed by its distribution receiver 183-189. To do so the set-top-box must have the appropriate A/V signal for substitution and have the necessary information identifying when the substitution is to occur. In the present example, the substitute video source may be one of the distributed video streams such as provided by distribution receiver 183 or it may be data received over the internet 171. Further, the identification of timing for video substitution is done out-of-band, over for example, the internet 171 and, advantageously, may be sourced by the broadcast facility 173 creating the original video being substituted.
The system of
The event triggers which initiate the substitution of video at the set-top-boxes are generated by each broadcast facility 173 producing the A/V programming in which a substitution is to occur. A broadcast facility includes an automation controller or BAC of which BAC 201 is shown in
The automation controller 201 operates by reading a database provided by the traffic department identifying each program segment that is to be assembled and played in real time. Logically, each program segment is recorded on a tape that is in a separate machine. As time approaches to play a particular segment, the automation controller 201 instructs the assigned tape machine to start playing. Exactly how it instructs the machine is known and depends on the type of system provided by the broadcast facility and how it is configured. In general each type of video source machine requires a period of time to start playing and synchronize itself with the rest of the broadcast facility. This is called a “pre-roll” and can take some number of seconds. Accordingly, the automation controller assembles the final video by properly starting the operation of the video sources.
Each broadcast facility includes an event trigger head end 211 which generates and transmits event triggers to the various set-top-boxes 103 to initiate the display of substitute signals as needed. Each event trigger is sent to the set-top-boxes 103 via the internet and identifies the A/V signal into which substitution is to occur and the time of that substitution. The set-top-boxes 103 respond to such event triggers from the event trigger head end 211 by selecting substitute signals from its possible sources and performing substitution needed. The event trigger head end 211 is itself triggered to send event triggers by signals from the automation controller 201 of the broadcast facility. A stimulation signal to the event trigger head end 211 is sent from the automation controller 201 with sufficient “pre-roll” or lead time to permit timely transmission of the event triggers to the set-top-boxes 103.
The event triggers from a plurality of broadcast facilities 173 are distributed via the internet to the set-top-boxes 103 of the system as represented in
The event triggers are sent in the form of who: when: what: id. “Who” is the name of a specific distribution system and broadcaster. For example, “who” may represent Direct TV-CHI-WGN or Comcast-CHI-ESPN1. “When” is a time stamp representing precisely when the substitution operation is to take place. In one embodiment, the time stamp represents Unix-seconds since epoch ‘.’ microseconds. “What” is a command identifying the action to perform. The “what” commands include CUE, PLAY and STOP. “id” is a key representing an application or file upon which an action is to be performed.
The CUE command identifies to a set-top-box that the file identified in the “id” portion will be played in the near future. The PLAY command identifies to the set-top-box 103 that the file identified in “id” is to be played at the specified time. The STOP command specifies that the set-top-box 103 is to stop playing the specified file at an identified time.
Advantageously, the CPU 145 of all set-top-boxes and all of the various event trigger head ends 211 are operating on the basis of the same time such as that provided by the network time protocol NTP. Similarly, all devices could be connected to the National Bureau of Standards time service so that consistent time exists at all units. A given event trigger may be delivered to different set-top-boxes at slightly different times due to different message delays through the internet. Accordingly, the automation controllers initiate the sending of an event trigger sufficiently early so that the event trigger will normally be delivered to all set-top-boxes with sufficient time to respond to the trigger. Should a set-top-box receive a trigger too late to properly respond the set-top-box ignores the event trigger and does not perform the requested action.
The set-top-boxes 103 receive and respond to the event triggers by performing signal substitution or other event. The A/V signal to be provided in substitution will have been requested from the service provider's network server 113 and stored in the set-top-box hard drive163. The substitute signal has a specific id with which an event trigger can identify and access it. Upon receipt of an event trigger the set-top-box must determine whether the event trigger relates to that set-top-box function. Namely whether the set-top-box is presently sending the A/V signal identified in the “who” portion of the event trigger to the display unit. When the set-top-box is, for example, presenting HG-TV to the display unit and the event trigger identifies ESPN1, the event trigger is ignored. In embodiments where the set-top-box 103 is responsible for channel selection, the particular channel being presented to the display is known because it is being controlled by the local CPU 145. In embodiments where the distribution receiver e.g., 183 is separate from the set-top-box 103, the set-top-box must identify the channel being presented by other methods.
The set-top-box identifies the channel being presented to the display so that the system is ready to substitute signals needed. Some broadcast signals, such as NTSC signals include the source identity in a portion of the signal such the vertical blanking interval. Thus, the set-top-box surveys the vertical blanking interval from time to time when such signals are being displayed so that a current reading of the channel being displayed kept. When other types of signals are being received, no such channel identity is conveyed with the signal and the channel must be identified from the signal content. When signal content must be recognized a significant portion of the identification is carried out by methods and apparatus at the Web services server 113. For example, the server 113 may include receivers for each channel in which signal substitution is expected. A predetermined portion of each received signal is periodically sampled and signature data, such as a hash sum, is extracted from the sample. Thus, signature data for each possible channel is created and updated by the Web services server 113. When a set-top-box needs to identify the channel being presented to its associated display, a sample of the predetermined portion of signal is taken by the set-top-box 103 and presented to the web service server 113 along with a request for identification. The web service server 113 then compares the set-top-box sample with the signatures stored at the web services server to identify a match. When the sampled channel is matched a message is returned to the set-top-box to identify the channel. The set-top-box can then store the identity of the regular signal (channel) being presented until a change is detected.
The following is an example of a signal substitution at all set-top-boxes presenting Direct TV Chicago ESPN2. At the appropriate pre-roll time for substitution, automation controller 201 sends to its associated event trigger head end 211 a trigger message which identifies the substitution. The head end 211 generates an event trigger identifying who: when: what: id which is transmitted to the internet 171 and multicast to all set-top-boxes capable of displaying Direct TV-Chicago-ESPN2. The set-top-boxes then receive the trigger and identify whether they are presently display Direct TV Chicago ESPN2. If not, the event trigger is ignored. If Direct TV Chicago ESPN2 is being displayed and the id of the file to be substituted matches information stored in the set-top-box, CPU 145 substitutes the data at id for the standard data being received in band as a part of ESPN2 at the time specified in the event trigger. A stop event trigger may then be sent to terminate the substitution of the end of the file id may cause reversion back to the standard channel signal.
As described above the set-top-boxes 103 and support system provide a capable arrangement for displaying standard or substitute A/V signals on a display device. The capabilities of the set-top-box coupled with the internet communication capability are a vehicle by which additional features and services can be provided. The set-top-box 103 can access A/V signals from DVD 161, hard drive 163 or any of the inputs connected to digital I/O 165 and display those signals to the exclusion of incoming commercial video stream such as Cable TV or Satellite TV. Accordingly, a large number of special programs can be produced and presented. Further, the CPU 145 acting in conjunction with PCI Express Controller 124 and graphics processor 123 can present graphics stored in memory such as memory 157 on the display 101 and present graphics overlays on video from any of the previously discussed video sources. Such graphics might include locally produced and/or locally relevant advertising or games or contests for the enjoyment of the viewers of the display.
The internet connection with the set-top-boxes 103 also enhances the capabilities of the system as a whole. The set-top-box 103 can access via the internet, various services, web pages and data sources and use the information assessed to provide A/VV display information. For example, a connection may be completed to sports news services 215 or to fantasy sports web site server 217 to display accessed information on A/V display 101. Significantly, the set-top-box 103 has internet access to the cell phone/text message network servers 219. Such access allows interactive A/V features to be provided for the viewers. For example, a video camera may be used to present images of “cute couples”. Fellow viewers are then permitted to vote on the cutest by sending wireless text messages of a type defined by overlay graphics on the presented video via the text message network and the results displayed as another overlay.
While there has been illustrated and described particular embodiments of the present invention, it will be appreciated that numerous changes and modifications will occur to those skilled in the art, and it is intended in the appended claims to cover all those changes and modifications which fall within the true scope of the present invention.