Related subject matter is found in a co-pending U.S. patent application, having application No. 09/823,646 , filed Mar. 20, 2001, entitled “ADAPTIVE BANDWIDTH FOOTPRINT MATCHING FOR MULTIPLE COMPRESSED VIDEO STREAMS IN A FIXED BANDWIDTH NETWORK,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 09/864,524, filed May 24, 2001, entitled “METHOD AND APPARATUS FOR A MULTIMEDIA SYSTEM,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 09/864,602 , filed May 24, 2001, entitled “METHOD AND APPARATUS OF MULTIPLEXING A PLURALITY OF CHANNELS IN A MULTIMEDIA SYSTEM,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 09/864,476 , filed May 24, 2001, entitled “METHOD AND APPARATUS FOR MANAGING RESOURCES IN A MULTIMEDIA SYSTEM,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 09/990,976 , filed Nov. 21, 2001, entitled “SYSTEM AND METHOD FOR MULTIPLE CHANNEL VIDEO TRANSCODING,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 09/990,737 filed Nov. 21, 2001, entitled “METHOD AND SYSTEM FOR RATE CONTROL DURING VIDEO TRANSCODING,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/137,151 , filed May 2, 2002, entitled “METHOD AND SYSTEM FOR PROTECTING VIDEO DATA,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/174,371, filed Jun. 18, 2002, entitled “DYNAMICALLY ADJUSTING DATA RATE OF WIRELESS COMMUNICATIONS,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/376,581 , filed Feb. 28, 2003, entitled “METHOD AND APPARTUS FOR NON-INTRUSIVE TRANSCEIVER PROPERTY ADJUSTMENT,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/376,853 , filed Feb. 28, 2003, entitled “SYSTEM FOR PROVIDING DATA TO MULTIPLE DEVICES AND METHOD THEREOF,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/345,710 , filed Jan. 16, 2003, entitled “METHOD OF MOTION VECTOR PREDICTION AND SYSTEM THEREOF,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/345,847, filed Jan. 16, 2003, entitled “METHOD OF VIDEO ENCODING USING WINDOWS AND SYSTEM THEREOF,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/375,582 , filed Feb. 24, 2003, entitled “METHOD AND SYSTEM FOR TRANSCODING VIDEO DATA,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/461,095 filed Jun. 13, 2003, entitled “SYSTEM AND METHOD FOR PROCESSING AUDIO FRAMES,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/683,062 , filed Oct. 10, 2003, entitled “METHOD AND APPARATUS FOR ACCURATELY DETECTING VALIDITY OF A RECEIVED SIGNAL,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/673,693 , filed Sep. 29, 2003, entitled “METHOD AND SYSTEM FOR SCALING IMAGES,” which is hereby incorporated herein by reference in its entirety. Related subject matter is found in a co-pending U.S. patent application, having application No. 10/673,612 filed Sep. 29, 2003, entitled “METHOD AND SYSTEM FOR NOISE REDUCTION IN AN IMAGE,” which is hereby incorporated herein by reference in its entirety.
The present disclosure relates generally to multimedia data delivery and more particularly to multimedia delivery in wireless systems.
In order to achieve reliable broadcast-quality motion picture experts group (MPEG) audio/video (A/V) distribution in an MPEG A/V delivery system, the audio and video received and encoded at an MPEG server typically must be played back at an MPEG client at substantially the same rate at which the MPEG data was received at the server. This matching rate significantly contributes to the reliable delivery of A/V data as it allows the decoder at the MPEG client to consume A/V data at the rate at which it was generated, thereby reducing or avoiding buffer underflow or overflow and thus reducing or eliminating artifacts in the displayed video and audio. In view of the benefits of the matching rate requirement, the MPEG standards, e.g., ISO/IEC 13818, typically institute a quality requirement that the jitter between the server and the client remain within +/−500 nanoseconds (ns). Accordingly, the MPEG standards implement elaborate mechanisms in an effort to fulfill this jitter requirement in standard broadcast systems. These mechanisms typically are satisfactory in satellite, cable or terrestrial delivery systems due to the near constant delay between the server and the client. However, in wireless networks and other similar networks, these mechanisms often fail because the wireless links typically do not exhibit a substantially constant delay between wireless devices. To illustrate, in IEEE 802.11 applications, the frequency, and thus the symbol rate, used to transmit information is dependent on the distance between the transmitter and the receiver. Accordingly, as the distance between the server and client varies, so does the jitter in the transmission of MPEG data between the server and client. Similarly, obstructions between the server and the client may cause dynamic changes in the symbol rate, thereby resulting in dynamic changes in the transmission time and therefore the jitter of MPEG data.
The magnitude of jitter also may increase as a result of the implementation of MPEG delivery systems within a network, such as a local area network (LAN), a wide area network (WAN), a metropolitan area network (WAN), the Internet and the like. In these instances, the transmission of MPEG data from a server to a client typically would require the processing of the MPEG data at various layers of the network stack (e.g., the telecommunications protocol/internet protocol (TCP/IP) stack) at the server, the transmission of the processed MPEG data via a wireless medium, and then the subsequent processing of the MPEG data at various layers of the network stack at the client. Considerable additional jitter maybe introduced due to variances in the software and hardware implementing these network stacks. The overall jitter resulting from these variances may be of such magnitude so as to prevent the client from synchronizing to the server. Accordingly, improved techniques for synchronizing devices in wireless environments so as to reduce jitter would be advantageous.
The purpose and advantages of the present disclosure will be apparent to those of ordinary skill in the art from the following detailed description in conjunction with the appended drawings in which like reference characters are used to indicate like elements, and in which:
The following description is intended to convey a thorough understanding of the present disclosure by providing a number of specific embodiments and details involving synchronized multimedia data delivery. It is understood, however, that the present disclosure is not limited to these specific embodiments and details, which are exemplary only. It is further understood that one possessing ordinary skill in the art, in light of known systems and methods, would appreciate the use of the disclosure for its intended purposes and benefits in any number of alternative embodiments, depending upon specific design and other needs.
Referring now to
At the MPEG server 102, the A/V encoder 108 receives input A/V data 132 and provides as an output an encoded or transcoded representation of the input A/V data 132. The RF transceiver 110 packetizes the encoded/transcoded A/V data and provides it for transmission over the wireless link 106 via the antenna 118. The A/V data 132 may represent, for example, MPEG audio and/or video data provided by a satellite feed, a cable head end, an Internet data stream, a DVD player and the like. At the MPEG client 104, the packetized A/V data is received by the RF transceiver 120 via the antenna 119. The A/V data is depacketized and provided to the input buffer 122 for storage. The A/V data is provided from the input buffer 122 to the A/V decoder 124 for decoding or transcoding and the resulting decoded/transcoded A/V data 134 may be provided for display on one or more display devices, stored on a hard disk or optical drive, retransmitted to another device for further processing, and the like.
As illustrated, the A/V encoder 108 is clocked by clock signal 136 (having frequency ƒA) provided by clock source 114 and the transceiver 110 is clocked by clock signal 138 (having frequency ƒB) provided by clock source 116. At the MPEG client 104, the A/V decoder 124 is clocked by clock signal 144 derived from a clock signal 140 (having frequency ƒD) provided by the clock source 128 and the transceiver 120 is clocked by clock signal 142 (having frequency ƒC) provided by the clock source 130. For ease of discussion, the frequencies ƒA and ƒD are assumed to be approximately 27 megahertz (MHz) in accordance with the MPEG standards and the frequencies ƒB and ƒC are assumed to be approximately 40 MHz. However, clock signals having other frequencies may be used, as appropriate, without departing from the spirit or the scope of the present disclosure.
Ideally, the clock sources 114 and 128 are perfectly synchronized and the clock sources 116 and 130 are perfectly synchronized, thereby reducing or eliminating jitter between the encoding of the A/V data 132 at the MPEG server 102 and the subsequent decoding of the A/V data 134 at the MPEG client 104. However, process variations, changes in temperature, voltage, and manufacturing defects, typically result in the production of clock sources that do not maintain an exactly constant frequency. This frequency drift typically is not mirrored between the clock sources of the MPEG server 102 and the MPEG client 104, and therefore potentially results in unacceptable jitter in the absence of a technique for adjusting the clock sources of the MPEG client 102 to more closely match the clock sources of the MPEG server 104. Accordingly, in at least one embodiment, the MPEG distribution system 100 employs a synchronization technique based on the frequency offsets between multiple clocks.
In at least one embodiment, the transceiver synchronization module 112 receives as inputs the clock signals 136 and 138, and from these clock signals determines a clock offset value (herein referred to Δppm1) between the clock signals 136 and 138. The clock offset value Δppm1 preferably is represented as parts-per-million (ppm), but in other instances may be represented as a frequency value or clock tick value. An exemplary technique for determining the clock offset value Δppm1 is discussed below with reference to
The synchronization module 112 then provides the clock offset value Δppm1 to the RF transceiver 110 for transmission to the MPEG client 104. In at least one embodiment, RF transceiver 110 packetizes the clock offset value Δppm1 by embedding it within a maintenance message between the MPEG server 102 and the MPEG client 104. To illustrate,
It will be appreciated that the use of the clock offset values Δppm1 and Δppm2 may result in substantial frequency drift over time as there is no feedback loop between the clocks 128, 130 and the clocks 114 and 116. Accordingly, in at least one embodiment, the synchronization module 126 monitors the fullness of the input buffer 122 to gauge the effective synchronization of the clocks 128, 130 to the clocks 114, 116. Generally, when the clock signal of an MPEG decoder is adequately synchronized to the clock signal of an MPEG encoder, the buffer supplying MPEG data to the decoder typically remains within a certain buffer fullness because the data typically is output from the buffer at the same rate it is input due to the adequate clock synchronization. However, when the clock signals are not adequately synchronized, buffer overflow or underflow may result. Accordingly, when the synchronization module 126 detects that the input buffer 122 may be heading toward a buffer underflow or overflow condition, the synchronization module 126 may implement a clock correction procedure as described in detail below.
Referring to
At step 155, the MPEG data stream is packetized and provided to the MPEG client 104 via the wireless network 106. At step 156, the relative timing reference values in the MPEG data stream are converted back to absolute timing reference values. As described below with reference to
Referring now to
At step 172, a representation of the first offset is provided to the MPEG client 104. To illustrate, the first offset representation may be packetized and transmitted as one or more packets via the wireless transceivers 110 and 120. At step 173, a second offset between the clock 116 of the MPEG server 102 and the clock 130 of the MPEG client 104 is determined. The second offset may be determined using any of a variety of techniques known to those skilled in the art, such as, for example, by using time domain correlation on a short synch training period to determine rough time and frequency alignment and then using time domain correlation in conjunction with frequency domain analysis to determine fine timing and frequency alignment. Time domain correlation and phase information from the OFDM pilot signal and OFDM constellation then may be used to drive an increasingly finer frequency offset between the transceivers 110 and 120. The second offset therefore may represent a relative synchronization between the clocks 116 and 130.
As noted above, the first offset represents the relative synchronization between the clocks 114 and 116 and the second offset may represent the relative synchronization between the clocks 116 and 130. Accordingly, at step 174, the first and second offsets may used to adjust the timing reference of the A/V decoder 124 (e.g., clock signal 144) so as to synchronize it with the clock 114 of the MPEG server 102, as described in greater detail with reference to
Referring now to
It will be appreciated that the current count of the counter 206 is representative of the number of clock cycles of the clock signal 136 during a predetermined number of clock cycles of the clock signal 214 (which was scaled to improve its resolution). From the current count of the counter 206, the clock offset calculation 210 may determine the clock offset value Δppm1 in accordance with the following equations:
C1=Bit_Count/FreqE*(FreqA+FreqA*ppmmax
C2=Bit_Count/FreqE*(FreqA−FreqA*ppmmin
Gain=ppmspread/(C1−C2) {EQ. 3}
Δppm1=(FreqA−C2)*Gain−ppmspread/2 {EQ. 4}
where Bit_Count represents the maximum unsigned number that may be represented by the number of bits in registers 204, 206 and 208 (e.g., Bit_Count=230 for a 30-bit register), FreqA represents the frequency of clock signal 136 (e.g., 27 MHz), FreqE represents the frequency of the scaled clock signal 214 (e.g., 160 MHz), ppmmax
Referring now to
Upon reception of a wireless signal, an orthogonal frequency division multiplexing (OFDM) demodulator 314 of the RF transceiver 120 demodulates the wireless signal and provides the resulting A/V data to the MAC processor 304, which in turn buffers the A/V data in the input buffer 122. As discussed in detail below, the MAC processor 304 (or, alternatively, the PHY processor 302) may convert the timing reference values (e.g., PCRs or SCRs) embedded in the A/V data from relative values back to absolute values. As noted above, the data also may include a representation of the clock offset value Δppm1, which the MAC processor 304 uses to control the NCO 306. The OFDM demodulator 314 further provides a phase offset or frequency offset to the PHY processor 302, where the phase/frequency offset is determined from timing circuits of the OFDM demodulator 314 (such as detectors, synchronizers and PLLs) that continuously run to correctly recover the data. Typically, this phase/frequency offset is averaged over a number of packets to obtain a frequency or phase offset accurate within fractions of a ppm. The PHY processor 302 determines the clock offset value Δppm2 from this phase/frequency offset and provides the clock offset value Δppm2 to control the NCO 306. The buffer monitor 312, in one embodiment, monitors the fullness of the buffer 122 to determine whether the fullness of the buffer has fallen below a lower threshold or has risen above a higher threshold. Based on this comparison, the buffer monitor 312 provides an error e to the NCO 312.
The NCO 312 output frequency is adjusted based on the clock offset values Δppm1 and Δppm2 and the error e. It will be appreciated that the resulting signal 316 output by the NCO 312 will be “dirty” (i.e., having a periodic clock cycles) and therefore is smoothed by the PLL 308. The resulting clock signal 318 then may be provided to the transport module 310, which uses the clock signal 318 for timing the removal of A/V data from the buffer 122. The clock signal 318 is also provided to the A/V decoder 124 for use in timing the decoding and display of the A/V data provided by the transport module 310.
Referring now to
Referring now to
Method 600 initiates at step 602 whereupon the MPEG client 104 receives the clock offset value Δppm1. At step 604, the clock offset value Δppm2 is determined. Meanwhile, at step 606, the fullness of the buffer 122 is monitored by comparing the fullness with one or more low and high thresholds to detect a potential buffer underflow or overflow. Based on this comparison, an error value e is generated. At step 608, the clock signal 144 provided to the A/V decoder 124 is adjusted based on the clock offset values Δppm1, Δppm2 and the error value e so as to more closely synchronize the clock signal 144 to the clock signal 136.
Referring now to
As illustrated, an MPEG transport stream 702 comprising a plurality of PCR values 703-706 (also referred to herein as PCR1-PCR4, respectively) is provided from the A/V encoder 108 (
PCR′x=PCRx−RTPServer {EQ. 5}
where RTPServer represents the value of the RTP counter incremented by local clock 114.
The resulting MPEG TS stream 712 then is packetized and transmitted between the RF transceiver 110 of the MPEG server 102 and the RF transceiver 120 of the MPEG client 104. At the MPEG client 104, the MPEG stream 712 is depacketized and a component of the receiver synchronization module 126 converts the relative PCR values 713-716 back to absolute PCR values 723-726 (also referred to herein as PCR″1-PCR″4). In one embodiment, the synchronization module 126 converts the PCR values back to absolute values by adding the client RTP counter value (incremented by the clock 128) of the MPEG client 104 to generate the PCR values in accordance with the following equation:
PCR″x=PCR′x+RTPClient {EQ. 6}
where RTPClient represents the value of the RTP by the local clock 128.
Note that when EQ. 6 is substituted into EQ. 7, the following equation results:
PCR″x=PCRx−RTPServer+RTPClient {EQ. 7}
Thus, presuming that RTPServer and RTPClient are synchronized (i.e., RTPServer˜=RTPClient), EQ. 8 simplifies to:
PCR″x˜=PCRx {EQ. 8}
Thus, as EQ. 9 demonstrates, the conversion from absolute PCR values to relative PCR values at the MPEG server 102 and the subsequent conversion of the relative PCR values back to absolute PCR values recovers the original PCR values (assuming the server and client RTP counters are synchronization) while reducing or eliminating the effects of jitter and propagation delay between the MPEG server 102 and the MPEG client 104 on the PCR values in the resulting MPEG transport stream 722.
Referring now to
Accordingly, in one embodiment, the MPEG server 102 implements method 800 in order to adjust the value of absolute timing references in retransmitted packets so as to reduce the occurrence and magnitude of such timing issues. At step 801, a packet is transmitted from the MPEG server 102 to the MPEG client 104. At step 802, the MAC processor 304 (
Other embodiments, uses, and advantages of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. The specification and drawings should be considered exemplary only, and the scope of the disclosure is accordingly intended to be limited only by the following claims and equivalents thereof.
Number | Name | Date | Kind |
---|---|---|---|
4866395 | Hosteller | Sep 1989 | A |
5027203 | Samad et al. | Jun 1991 | A |
5093847 | Cheng | Mar 1992 | A |
5115812 | Sano et al. | May 1992 | A |
5253056 | Puri | Oct 1993 | A |
5475434 | Kim | Dec 1995 | A |
5563950 | Easter et al. | Oct 1996 | A |
5602589 | Vishwanath et al. | Feb 1997 | A |
5635985 | Boyce et al. | Jun 1997 | A |
5644361 | Ran et al. | Jul 1997 | A |
5652749 | Davenport et al. | Jul 1997 | A |
5732391 | Fiocca | Mar 1998 | A |
5737020 | Hall et al. | Apr 1998 | A |
5740028 | Sugiyama et al. | Apr 1998 | A |
5844545 | Suzuki et al. | Dec 1998 | A |
5850443 | Van Oorschot et al. | Dec 1998 | A |
5940130 | Nilsson et al. | Aug 1999 | A |
5996029 | Sugiyama et al. | Nov 1999 | A |
6005623 | Takahashi et al. | Dec 1999 | A |
6005624 | Vainsencher | Dec 1999 | A |
6014694 | Aharoni et al. | Jan 2000 | A |
6040863 | Kato | Mar 2000 | A |
6081295 | Adolph et al. | Jun 2000 | A |
6141693 | Perlman et al. | Oct 2000 | A |
6144402 | Norsworthy et al. | Nov 2000 | A |
6167084 | Wang et al. | Dec 2000 | A |
6182203 | Simar, Jr. et al. | Jan 2001 | B1 |
6215821 | Chen | Apr 2001 | B1 |
6219358 | Pinder et al. | Apr 2001 | B1 |
6222886 | Yogeshwar | Apr 2001 | B1 |
6236683 | Mougeat et al. | May 2001 | B1 |
6259741 | Chen et al. | Jul 2001 | B1 |
6263022 | Chen et al. | Jul 2001 | B1 |
6300973 | Feder et al. | Oct 2001 | B1 |
6307939 | Vigarie | Oct 2001 | B1 |
6314138 | Lemaguet | Nov 2001 | B1 |
6323904 | Knee | Nov 2001 | B1 |
6327273 | Van der Putten et al. | Dec 2001 | B1 |
6366614 | Pian et al. | Apr 2002 | B1 |
6385248 | Pearlstein et al. | May 2002 | B1 |
6438168 | Arye | Aug 2002 | B2 |
6480541 | Girod et al. | Nov 2002 | B1 |
6526099 | Chistopoulos et al. | Feb 2003 | B1 |
6549561 | Crawford | Apr 2003 | B2 |
6584509 | Putzolu | Jun 2003 | B2 |
6714202 | Dorrell | Mar 2004 | B2 |
6724726 | Coudreuse | Apr 2004 | B1 |
6748020 | Eifrig et al. | Jun 2004 | B1 |
7263110 | Fujishiro | Aug 2007 | B2 |
20010026591 | Keren et al. | Oct 2001 | A1 |
20020106022 | Takahashi et al. | Aug 2002 | A1 |
20020110193 | Kyoon et al. | Aug 2002 | A1 |
20020138259 | Kawahara | Sep 2002 | A1 |
20020145931 | Pitts | Oct 2002 | A1 |
20020196851 | Lecoutre | Dec 2002 | A1 |
20030093661 | Loh et al. | May 2003 | A1 |
20030148785 | Mangal et al. | Aug 2003 | A1 |
20030152148 | Laksono | Aug 2003 | A1 |
20030169776 | Reesor | Sep 2003 | A1 |
20040090994 | Lockridge et al. | May 2004 | A1 |
20040174907 | Van Der Putten | Sep 2004 | A1 |
20060009246 | Marinier et al. | Jan 2006 | A1 |
20060075133 | Kakivaya et al. | Apr 2006 | A1 |
Number | Date | Country |
---|---|---|
0661826 | Jul 1995 | EP |
0739138 | Oct 1996 | EP |
0805599 | Nov 1997 | EP |
0855805 | Jul 1998 | EP |
0896300 | Feb 1999 | EP |
0901285 | Feb 1999 | EP |
0955607 | Nov 1999 | EP |
1032214 | Aug 2000 | EP |
1087625 | Mar 2001 | EP |
1 089 482 | Apr 2001 | EP |
1 339 182 | Aug 2003 | EP |
07-210670 | Aug 1995 | JP |
WO 0195633 | Dec 2001 | WO |
WO 0209331 | Jan 2002 | WO |
WO 02080518 | Oct 2002 | WO |
Number | Date | Country | |
---|---|---|---|
20060161675 A1 | Jul 2006 | US |