Patent Grant
7093275
a. Field of the Invention
The present invention pertains generally to digital data transferred over a network in conjunction with video or audio data and more specifically to synchronization of data transferred over a network with video or audio data transferred on that network or on an independent network or channel.
b. Description of the Background
Television and audio broadcast may employ a range of broadband media including cable, satellite and high speed Internet. The term broadband refers to the capability of the media or network to carry multiple channels at one time. Video and audio programs are typically allocated a constant bandwidth such that dropouts or distorted output is not produced. A channel that provides a constant minimum bandwidth rate may be referred to as isochronous or as a constant bit rate (CBR) channel. Other services that transfer data on the network, such as Internet, have typically been asynchronous, wherein bandwidth may vary depending on network load. New standards are emerging that provide for isochronous transfer of data on a network. Such standards include voice over DSL (Digital Subscriber Line) and DOCSIS (Data Over Cable System Interface Specification) and more specifically by PacketCable that provides a method for isochronous data transfers over cable.
The advent of isochronous data transfer over broadband networks enables streaming audio and video programs other than those provided by a cable or television operator. Additionally, isochronous data transfers allow the possibility for data from one channel to be synchronized with an audio/video broadcast stream or with data from another channel. Synchronized data transfer may be employed to provide modification and enhancement of a broadcast program. For example, subtitles in a number of spoken languages, or sign language, may be added to televised programs. Such additions are outside the capabilities of captioning facilities of televisions. Additionally, synchronized data transfers may be used to add local content to advertising, such as local automobile dealer information presented in conjunction with a national automobile advertisement. Data transfers must be synchronized in some manner such that subtitles appear at the correct time or advertising message enhancements appear at the correct time. Synchronized data transfers utilized in conjunction with an audio/video stream may provide increased viewer enjoyment and may provide additional opportunities for revenue. As such, a new method is needed to control the transfer of data and the utilization of that data in synchrony with an audio/video stream or data from another channel.
The present invention overcomes the disadvantages and limitations of the prior art by providing a system and method wherein data transferred over an isochronous channel may be synchronized with an audio or video stream. Time information contained within an audio or video stream, such as an MPEG audio/video stream, is employed to synchronize the utilization of data from the isochronous channel. MPEG formats are employed in cable, satellite, DVD and other systems. Information describing the stream, time and nature of combination with a data transfer may be contained within the audio/video stream, the data transfer, or may be supplied by other methods.
The present invention may therefore comprise a method for synchronizing data transfer with an audio/video stream comprising: receiving an audio/video stream; receiving a data transfer including a descriptor of when the data transfer may be combined with the audio/video stream; obtaining time information from the audio/video stream; and combining the data transfer and the audio/video stream when the time information is in substantial agreement with the descriptor.
The invention may further comprise a system for combining a data transfer with an audio/video stream comprising: a video server; a data server; a client system operable to receive the audio/video stream from the video server and the data transfer from the data server; a network management component operable to provide isochronous data transfer from the data server to the client system; and a processing component in the client system operable to monitor the audio/video stream for time information and to combine the data transfer with the audio/video stream.
Latencies incurred between requests for data and reception of data over the channel may be measured and the time at which data is requested may be adjusted in order to minimize local memory requirements. Advantages of the present invention include the ability to overlay images from a data transfer with an audio/video stream. Such overlays may be used for advertising, entertainment or education. Advertising may employ an overlay to identify products for sale in a video image, or to display a product ordering menu or product information. An entertainment application may employ the present invention to combine an audio/video stream, used for background images, with a data transfer used to overlay game characters or other images. Further, the invention may be employed to display a musical score or other information such as guitar chords when a musical audio/video stream is presented. In addition, the present invention allows personalization of advertising at the customer level whereas other personalization methods are often implemented at the broadcast site, limiting the number of versions that may be produced. In other words, the present invention allows modification of an audio/video stream by the client system in contrast to methods where modification is performed by a server at the head end of a cable or satellite system and is therefore limited by the number of broadcast channels.
Further yet, the synchronization of data transfers and an audio/video stream may be employed to reduce processing requirements of a client system for television based advertising and purchasing. A client system may be a set top box, satellite receiver or personal computer, or other system capable of networking and audio/video output. In contrast to executing menu software on a client system, the system of the present invention may be employed to return audio/video stream identification, frame number, and mouse click position from the client system to a server. The server then may generate menus, product information, or other displays and transfer them to the client system for combination with an audio/video stream.
Further advantages of the present invention include that an audio/video presentation need not be modified to include markers or triggers, as is the case for ATVEF (Advanced Television Enhancement Foundation). By using timing information already contained in a broadcast or streaming presentation, the present invention allows greater flexibility in content enhancement.
In the drawings,
The present invention provides a system and method for the transfer of data and use of the data in synchrony with an audio/video stream. Data transfers may be on a separate channel or network and may employ protocols as defined by DOCSIS (Data Over Cable System Interface Specification) versions including PacketCable or other standards such that the network or channel provides isochronous data transfer.
An audio/video stream may be in MPEG-2 format, as may be used in satellite and digital cable television systems. MPEG-2 employs data packets of 188 bytes each. At the start of each packet is a package identifier (PID). There are four types of PIDs used by satellite receivers. A VPID is the PID for the video stream and an APID is the PID for an audio stream. A PCR PID (program clock reference) may be used to synchronize the video and audio packets. Clock reference data is often embedded into the video stream. The present invention may employ timing information from the PCR PID or the timing information contained within the video stream or audio stream to synchronize utilization of transferred data. A fourth PID is the System Information PID and may be used for data such as a program guide, and information about other frequencies. In one embodiment of the present invention, the System Information PID may be employed to indicate that data is available in conjunction with the present audio/video stream.
Data transfers may be broadcast in conjunction with a program, or may be transferred in response to a request from a client system, such as a set top box, satellite receiver, or personal computer. A software program in the client system controls the utilization of data transfers with an audio/video stream. When data transfers are broadcast in conjunction with an audio/video stream, the data transfers may contain information that identifies the audio/video stream with which the data may be utilized, and the time at which the data may be utilized. Further, the data transfer may contain information describing data type and how the data is to be used. For example, a broadcast data transfer containing names of local automobile dealers to be used in conjunction with an advertising message may include a descriptor that text data of a predetermined font size is to be overlaid at a specified position on the display screen. Further, information contained within the data transfer or audio/video stream may define the type of combination to be performed between the transferred data and the audio/video stream. The combination may employ a number of methods including logical operations, blending, or conditional combination, for example. Further, combination may include windowing wherein information from an audio/video stream may be placed in a window and information generated using a data transfer may be placed in another window or combination of windows. Combination may be performed using digital data in a display memory or may employ analog signal mixing.
As previously mentioned, data transfers may be broadcast in conjunction with a program, or may be transferred in response to a request from a client system. When data transfers are performed in response to a request from a client system, the client system may include methods to estimate or measure the amount of time required to establish a connection and receive a data transfer. Advantageously, these methods may reduce the amount of memory needed in the client system to save data transfers prior to utilization. A software program in the client system may contain downloaded information describing which audio/video streams may have associated data transfers. The downloaded information may be received as part of a data transfer employing an isochronous data channel or may be transferred by other methods such as an asynchronous Internet connection. Alternately, an audio/video stream may contain information indicating that a data transfer may be utilized with the audio/video stream.
Cable modem 108 communicates with cable modem termination system 106 and with host processor 110. Cable modem 108 is a modem (modulator/demodulator) that communicates across cable television cable. Cable modem 108 may be an internal or external cable modem, as may be employed with personal computers, or may be part of an interactive set top box. An internal cable modem, or a cable modem that is part of an interactive set top box, communicates data through the system's internal bus. An external cable modem may employ Ethernet or USB (Universal Serial Bus) to transfer data between the modem and the system.
Synchronized service unit 122 comprises demodulator demultiplexer 118, host processor 110 and A/V decoder 120. Synchronized service unit 122 may be part of a set top box, satellite receiver or personal computer. Host processor 110 may comprise a CPU, program memory and system memory. Host processor 110 is operable to send and receive data to and from data server 100, to process data in system memory, and to output processed data to A/V decoder 120. Demodulator demultiplexer unit 118 may be typical of that found in a cable set top box receiver and may comprise an A/D converter, QAM-64/256 demodulator, MPEG frame synchronization, and Reed Solomon error correction for receive operations. Demodulator demultiplexer unit 118 serves as a tuner, receiving information contained within a frequency range. Information supplied by demodulator demultiplexer unit 118 to host processor 110 and A/V decoder 120 may comprise MPEG data, control or other information. Host processor 110 may process information from demodulator demultiplexer 118, possibly employing data received from data server 100, and supply processed information to A/V decoder 120, or, host processor 110 may allow information from demodulator demultiplexer 118 to pass directly to A/V decoder 120.
As shown in
Demodulator demultiplexer unit 118 supplies signals to host processor 110 and to A/V decoder 120. As noted previously, host processor 110 may process information from demodulator demultiplexer 118 and data from cable modem 108 to produce processed information that is sent to A/V decoder 120. Processing by host processor 110 may include combining audio and video information and scaling of images. For example, host processor 110 may vertically compress a video image to produce a banner area at the bottom of the screen, and then display information from the cable modem 108 in the banner area. Further, host processor may be employed to parse information from demodulator demultiplexer 118 and selectively replace data with data from cable modem 108. For example, audio information received by demodulator demultiplexer 118 may be replaced with audio information received by cable modem 108, thereby allowing output of a foreign language, or language corresponding to a selected audience rating. The information transferred from host processor 110 to A/V decoder 120 may be MPEG data, bitmap data, or data in other formats.
A/V 120 decoder may support analog or digital signal inputs, or both. Output from A/V decoder 120 may be in formats such as composite video, S-video, RGB, and component audio. Output from A/V decoder 120 may be supplied to display unit 124 which may comprise a television, computer monitor or other display device. Alternatively, output from A/V decoder 120 may be supplied to a recording device such as a video tape recorder, for example. In an alternate embodiment, A/V decoder 120 may include processing functions such as overlay and windowing, for example, such that the images and audio provided to display unit 124 are processed in part by host processor 110 and may be processed in part by A/V decoder 120. A/V decoder may comprise MPEG decoding circuitry, a frame buffer, and circuitry supporting other functions such as windowing, menu display, captioning and other functions, for example.
The flowcharts of
The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and other modifications and variations may be possible in light in the above teachings. The embodiment was chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the appended claims be construed to include other alternative embodiments of the invention except insofar as limited by the prior art.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6324694 | Watts et al. | Nov 2001 | B1 |
| 6487543 | Ozaki et al. | Nov 2002 | B1 |
| 6493872 | Rangan et al. | Dec 2002 | B1 |
| 6530084 | Del Sesto et al. | Mar 2003 | B1 |
| Number | Date | Country | |
|---|---|---|---|
| 20020162122 A1 | Oct 2002 | US |
