1. Field of the Invention
The present invention relates to encoding of data to provide for robust error recovery due to data losses typically incurred during transmission or storage of signals.
2. Art Background
A number of techniques exist for reconstructing lost/damaged data due to random errors that may occur during signal transmission or storage. However, these techniques are limited at handling the loss of consecutive packets of data. Consecutive loss of packets of data is described in the art as burst error. Burst errors may result in a reconstructed signal with such a degraded quality that it is easily apparent to the end user.
Additionally, compression methodologies used to facilitate high speed communications compound the signal degradation caused by burst errors, thus adding to the degradation of the reconstructed signal. Examples of burst error loss affecting transmitted and/or stored signals is seen in high definition television (“HDTV”) signals, mobile telecommunication applications, as well as video storage technologies including compact disc (CD), video disk (e.g., DVD), and video cassette recorders (VCRs).
In one application, the advent of HDTV has led to television systems with a much higher resolution than the current standards proposed by the National Television Systems Committee (“NTSC”). Proposed HDTV signals are predominantly digital. Accordingly, when a color television signal is converted for digital use the luminance and chrominance signals can be digitized using eight bits. Digital transmission of NTSC color television so encoded requires a nominal bit rate of about two hundred and sixteen megabits per second. The transmission rate is greater for HDTV which would nominally require about 1200 megabits per second. Such high transmission rates are well beyond the bandwidths supported by current wireless standards. Accordingly, an efficient compression methodology is required.
Compression methodologies also play an important role in mobile telecommunication applications. Typically, packets of data are communicated between remote terminals in mobile telecommunication applications. The limited number of transmission channels in mobile communications requires an effective compression methodology prior to the transmission of packets. A number of compression techniques are available to facilitate high transmission rates.
Adaptive Dynamic Range Coding (“ADRC”) and Discrete Cosine Transform (“DCT”) coding provide image compression techniques known in the art. Both techniques take advantage of the local correlation within an image to achieve a high compression ratio. However, an efficient compression algorithm can result in compounded error propagation because errors in an encoded signal are more prominent when subsequently decoded. This error multiplication can result in a degraded video image that is readily apparent to the user.
The present invention provides a method for compressing data by determining a central value that is greater than the minimum value and less than the maximum value of the range of data. In one embodiment, the central value is chosen to be a value that substantially reduces a decoding error in the event that the range of values is subsequently estimated. In one embodiment, the central value is the value that minimizes the expected mean square error during reconstruction when there is an error. In one embodiment, the maximum and minimum values represent intensity data for pixels of an image. In another embodiment, the compression process is Adaptive Dynamic Range Coding, and the central value is a value within the dynamic range, excluding the maximum and minimum values.
The objects, features and advantages of the present invention will be apparent from the following detailed description in which:
The present invention provides a method for coding and arranging a signal stream to provide for a robust error recovery and methods for performing error recovery. In the following description, for purposes of explanation, numerous details are set forth, in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that these specific details are not required in order to practice the present invention. In other instances, well known electrical structures and circuits are shown in block diagram form in order not to obscure the present invention unnecessarily.
The following is described in the context of Adaptive Dynamic Range Coding (ADRC) encoded video images, and more particularly the recovery of lost or damaged (lost/damaged) compression parameters such as dynamic range (DR). However, it is contemplated that the present invention is not limited to video, not limited to ADRC encoding, and not limited to the particular compression parameters generated. Rather, it will be apparent that the present invention is applicable to different compression technologies, and different types of correlated data, including, but not limited to, two-dimensional static images, hologram images, three-dimensional static images, video, two-dimensional moving images, three dimensional moving images, monaural sound, and N-channel sound. The present invention is also applicable to different compression parameters including, but not limited to, the central value (CEN) which may be used in ADRC processes. In addition, the present invention is applicable to different types of ADRC processes including edge-matching and non edge-matching ADRC. For further information regarding ADRC, see “Adaptive Dynamic Range Coding Scheme for Future HDTV Digital VTR”, Kondo, Fujimori, Nakaya, Fourth International Workshop on HDTV and Beyond, Sep. 4–6, 1991, Turin, Italy.
The signal encoding, transmission, and subsequent decoding processes are generally illustrated in
Encoder 110 can follow the Adaptive Dynamic Range Coding (“ADRC”) compression algorithm and generate Packets 1, . . . N for transmission along transmission Media 135. Decoder 120 receives Packets 1, . . . N from Transmission Media 135 and generates Signal 130. Signal 130 is a reconstruction of Signal 100.
Encoder 110 and Decoder 120 can be implemented a variety of ways to perform the functionality described herein. In one embodiment, Encoder 110 and/or Decoder 120 are embodied as software stored on media and executed by a general purpose or specifically configured computer or data processing system, typically including a central processing unit, memory and one or more input/output devices and co-processors, as shown in
Embodiments of circuits for encoding and recovering lost/damaged compression parameters are shown in
In another embodiment, the output 185 outputs the recovered compression parameter. The recovered compression parameter is then input to other circuitry to generate the decoded data.
An alternate embodiment of the circuits for encoding compression parameters and recovering lost/damaged compression parameters are shown in
Packet structure 300 is generated by encoder 110 and is transmitted across transmission/storage media 135. For one embodiment, packet structure 300 comprises five bytes of header information, eight DR bits, eight CEN bits, a motion flag bit, a five bit threshold index, and 354 bits of Q codes. The packet structure described herein is illustrative and may typically be implemented for transmission in an asynchronous transfer mode (“ATM”) network. However, the present invention is not limited to the packet structure described and a variety of packet structures that are used in a variety of networks can be utilized.
In one embodiment, data structure 300 is stored in a computer readable memory, so that data structure 300 can be accessed by a program being executed on a data processing system. The data structure 300 stored in the memory includes a dynamic range data object (DR) and a central value data object (CEN) associated with the dynamic range data object. The central value data object has a value that is greater than a minimum value of the dynamic range data object, and less than a maximum value of the dynamic range data object. The central value data object substantially reduces a decoding error in the event that the dynamic range data object is estimated. Each data structure 300 may be a packet structure.
As noted above, the exemplary system and apparatus described above may be used to encode images, such as video or moving images, using ADRC. ADRC has been established as a feasible real-time technique for coding and compressing images in preparation for constant bit-rate transmission.
The discrete data points that make up a digital image are known as pixels. Each pixel may be represented independently using 8 bits, but other representations can also be used for the purposes of compression or analysis. Many representations begin by dividing this raw data into disjoint sets. For historical reasons, these sets, which may be composed of one or more pieces of data or pixels, are referred to as “blocks”, even though they may not have a traditional block shape. The data may then be characterized by compression parameters. In one embodiment, these compression parameters include block parameters and bitstream parameters.
A block parameter includes data that describes how an image looks. The block parameter therefore may be used to define one or more attributes of a block. For example, in ADRC, block-wide information can include the minimum pixel value (MIN), the maximum pixel value (MAX), the central pixel value (CEN), the dynamic range of the pixel values (DR), or any combination of these values.
A bitstream parameter may include data that describes how an image is encoded. In one embodiment, a bitstream parameter may indicate the number of bits used to encode data. For example, in ADRC, the bitstream parameters may include Qbit and motion flag (MF) values. In this embodiment, the bitstream parameter therefore can indicate how the data is encoded that represents where a pixel value lies within the range specified by the global information.
In one example in which ADRC encoding is used, the block data is comprised of the MIN, DR and Qbit number (defined below), and the pixel data is comprised of Qcodes. DR may be defined as MAX−MIN or MAX−MIN+1. In the present embodiment, as will be explained below, CEN may be defined as a value between MIN and MAX. For example, CEN may be equal to MIN+DR/2.
A Q code is an integer in the range [0,2Q−1] that identifies one value in the set {MIN, MIN+1, . . . , CEN, . . . , MAX}. Since the Qbit number, Q, is generally small and the DR value may be relatively large, it is generally not possible to represent all pixel values exactly. Therefore, some quantization error is introduced when pixel values are reduced to Qcode values. For instance, if the Qbit number is 3, then it is generally possible to represent 23=8 values from the set {MIN, MIN+1, . . . , CEN, . . . , MAX} without any error. Pixels with other values are rounded to one of these eight values. This rounding introduces quantization error.
Temporal compression is feasible for a sequence of like images spanning more than one instance in time. An image frame is defined as the 2-dimensional collection of pixels arising within a given time period. It is well known that data from corresponding locations of temporally close image frames is likely to contain similar values. When this is true, compression is improved by encoding each of these like values only once.
In a second example, multiple image frames are encoded by adding a motion flag (MF) to the block information of the first example. This MF indicates whether or not data from each frame is encoded using separate Qcodes. If no motion is indicated, the same Qcodes are used to represent each frame of data. If motion is indicated, then separate Qcodes are used to encode each frame.
Two methods of ADRC coding can be used: non-edge-matching ADRC, and edge matching ADRC. These methods differ in the precise formula used to generate the quantization code (Qcode) value. On the other hand, the methods have much in common. Both methods begin by segmenting the image into blocks, and then determining the maximum (MAX) and minimum (MIN) pixel value for each block. In 2D ADRC, one quantization code (Qcode) value is determined for each pixel. In 3D ADRC, a motion flag (MF) value (1 if motion, 0 otherwise) is determined for each block. When the motion flag is 1, a unique Qcode can be determined for each block. When the motion flag is 0, then corresponding pixel values can be averaged for each block, the block parameter values are updated accordingly, and a single Qcode can be determined that will represent the corresponding pixels from each frame.
Non-edge-matching ADRC can define the DR value as
DR=MAX−MIN+1 (1)
and a quantization code as
where Q is the number of quantization bits, and xi is the original pixel value (or averaged pixel value, in the case of non-motion blocks in 3D ADRC). Pixel values can be reconstructed or recovered according to the following formula:
where MAX represents the maximum level of a block, MIN represents the minimum level of a block, Q represents the number of quantization bits, qi represents the quantization code (encoded data), x′i represents the decoded level of each sample, and where it is expected that x′i≈xi.
Edge-matching ADRC can define the DR value as
DR=MAX−MIN (4)
and a quantization code as
where Q is the number of quantization bits, and xi is the original pixel value (or averaged pixel value, in the case of non-motion blocks in 3D ADRC). Pixel values can be reconstructed or recovered according to the following formula:
where MAX represents the maximum level of a block, MIN represents the minimum level of a block, Q represents the number of quantization bits, qi represents the quantization code (encoded data), x′i represents the decoded level of each sample, and where it is expected that x′i≈xi.
Although the exemplary quantization code and reconstruction formulae above for ADRC uses the MIN value, any value greater than or equal to MIN, and less than or equal to MAX, may be used, along with DR, to encode and decode pixel values. For both edge matching and non-edge matching ADRC, the DR value may be lost during transmission. If DR is lost, then pixel values are reconstructed using an estimate for DR.
The maximum decoding error when DR is overestimated (or underestimated) is related to the value, e.g., block parameter, that is used to encode and decode the pixel values.
For example,
The axis 210 on the left of
The performance illustrated in
The central value may be selected as a value that substantially reduces, and even minimizes, the expected mean square error for reconstruction when DR is estimated, and has a constant DR estimation error. This value may be determined by the following process.
A general form of the ADRC decoding equation without truncation error is:
where values for zi, M and K are provided in Table 1. The general form of Eq. (7) both simplifies the ADRC notation and allows for simultaneous derivation of formulas for non-edge-matching and edge-matching ADRC.
If instead of transmitting the MIN value, some other value is transmitted and the DR value is strictly positive, the other value may be expressed as:
VAL=MIN+αDR (8)
where α is a constant. The ADRC decoding equation may therefore be:
Let DRe represent erroneous estimate of the dynamic range. The erroneous decoding may be represented as:
where xerror(i) represents an erroneous decoding, and so the decoding error errori=xi−xerror(i) can be written:
Therefore, the mean square error (MSE) may be expressed as a function of α:
where errors represents the decoding error, N represents the number of erroneously decoded pixels, and α is a non-negative real number.
The expected mean square error may be expressed as a function of α to optimize over α:
where MSE (α) represents the mean square error expressed as a function of α, and E represents the expected mean square error.
The conditions for minimization can be checked by computing the first and second derivatives:
From Eq. (18), the second derivative is strictly positive whenever DRe≠DR; therefore the point at which E′ (MSE(α))=0 is the global minimum. This may be achieved if:
2M2α=2E(zi)M (19)
Eq. (19) therefore implies that:
Assuming a uniform distribution of Qcode values, the expected values may be:
Thus, in the case of non-edge-matching ADRC Eq. (20) may become:
Similarly for edge-matching ADRC, Eq. (20) may become:
Substituting α=½ back into Eq. (8), the optimal value to transmit may be:
VAL=MIN+DR/2=CEN (24)
for either non-edge-matching ADRC or edge-matching ADRC.
Although this derivation assumed a uniform distribution of Qcode values, a non-uniform distribution which favors Qcode values near the middle of the region also supports the use of the CEN value.
The benefit of transmitting the CEN value can be quantified using Eq. (16) and substituting α=0 so that VAL=MIN and α=½ so that VAL=CEN.
Assuming a uniform distribution of Qcode values, E(qi2) may be calculated as:
The ratio of the mean square errors for CEN value decoding to the mean square errors for MIN value decoding is tabulated in Table 2 for various Qbit values Q:
The reduction in mean square error for DR recovery assuming some common types of ADRC encoding is therefore quantified. Thus, the CEN value is the mean square optimal counterpart of DR in ADRC transmission subject to DR loss. This type of encoding is referred to herein as Central Value ADRC.
In Central Value ADRC, the central value (CEN) can be transmitted in place of the MIN value. In one embodiment, as discussed above, the CEN value may be defined as
In this embodiment, the reconstruction formulas for x′i may be found by substituting MIN=CEN−DR/2 in Eqs. (3) and (6). That is, for non-edge-matching ADRC:
and in the case of edge-matching ADRC:
In the error-free case, the performance of Central Value ADRC using CEN is similar to ADRC using a MIN value. However, in the presence of DR loss, Central Value ADRC may provide better lost/damaged data recovery performance compared with the MIN value ADRC.
The invention has been described in conjunction with a preferred embodiment. It is evident that numerous alternatives, modifications, variations and uses will be apparent to those skilled in the art in light of the foregoing description.
This application is a continuation application of Ser. No. 09/342,296, filed Jun. 29, 1999 now U.S. Pat. No. 6,549,672.
Number | Name | Date | Kind |
---|---|---|---|
4361853 | Remy et al. | Nov 1982 | A |
4381519 | Wilkinson et al. | Apr 1983 | A |
4419693 | Wilkinson et al. | Dec 1983 | A |
4509150 | Davis | Apr 1985 | A |
4532628 | Matthews | Jul 1985 | A |
4574393 | Blackwell et al. | Mar 1986 | A |
4586082 | Wilkinson | Apr 1986 | A |
4656514 | Wilkinson et al. | Apr 1987 | A |
4675735 | Wilkinson et al. | Jun 1987 | A |
4703351 | Kondo | Oct 1987 | A |
4703352 | Kondo | Oct 1987 | A |
4710811 | Kondo | Dec 1987 | A |
4722003 | Kondo | Jan 1988 | A |
4772947 | Kono | Sep 1988 | A |
4788589 | Kondo | Nov 1988 | A |
4797653 | Takayama | Jan 1989 | A |
4807033 | Keesen et al. | Feb 1989 | A |
4815078 | Shimura | Mar 1989 | A |
4845560 | Kondo et al. | Jul 1989 | A |
4885636 | Sullivan | Dec 1989 | A |
4890161 | Kondo | Dec 1989 | A |
4903124 | Hoshi et al. | Feb 1990 | A |
4924310 | Von Brandt | May 1990 | A |
4953023 | Kondo | Aug 1990 | A |
5023710 | Kondo et al. | Jun 1991 | A |
5047768 | Yamaguchi et al. | Sep 1991 | A |
5052021 | Goto et al. | Sep 1991 | A |
5086489 | Shimura | Feb 1992 | A |
5093872 | Tutt | Mar 1992 | A |
5101446 | Resnikoff et al. | Mar 1992 | A |
5134479 | Ohishi | Jul 1992 | A |
5142537 | Kutner et al. | Aug 1992 | A |
5150210 | Hoshi et al. | Sep 1992 | A |
5159452 | Kinoshita et al. | Oct 1992 | A |
5162923 | Yoshida et al. | Nov 1992 | A |
5166987 | Kageyama | Nov 1992 | A |
5177797 | Takenaka et al. | Jan 1993 | A |
5185746 | Tanaka et al. | Feb 1993 | A |
5196931 | Kondo | Mar 1993 | A |
5208816 | Seshardi et al. | May 1993 | A |
5228059 | Takegahara et al. | Jul 1993 | A |
5237424 | Nishino et al. | Aug 1993 | A |
5241381 | Kondo | Aug 1993 | A |
5243428 | Challapali et al. | Sep 1993 | A |
5247363 | Sun et al. | Sep 1993 | A |
5258835 | Kato | Nov 1993 | A |
5307175 | Seachman | Apr 1994 | A |
5327502 | Katata et al. | Jul 1994 | A |
5337087 | Mishima | Aug 1994 | A |
5359694 | Concordel | Oct 1994 | A |
5379072 | Kondo | Jan 1995 | A |
5398078 | Masuda et al. | Mar 1995 | A |
5400076 | Iwamura | Mar 1995 | A |
5406334 | Kondo et al. | Apr 1995 | A |
5416651 | Uetake et al. | May 1995 | A |
5428403 | Andrew et al. | Jun 1995 | A |
5434716 | Sugiyama et al. | Jul 1995 | A |
5438369 | Citta et al. | Aug 1995 | A |
5446456 | Seo | Aug 1995 | A |
5455629 | Sun et al. | Oct 1995 | A |
5469216 | Takahashi et al. | Nov 1995 | A |
5469474 | Kitabatake | Nov 1995 | A |
5471501 | Parr et al. | Nov 1995 | A |
5473479 | Takahura | Dec 1995 | A |
5481554 | Kondo | Jan 1996 | A |
5481627 | Kim | Jan 1996 | A |
5499057 | Kondo et al. | Mar 1996 | A |
5528608 | Shimizume | Jun 1996 | A |
5557420 | Yanagihara et al. | Sep 1996 | A |
5557479 | Yanagihara | Sep 1996 | A |
5577053 | Dent | Nov 1996 | A |
5594807 | Liu | Jan 1997 | A |
5598214 | Kondo et al. | Jan 1997 | A |
5602858 | Kitaori | Feb 1997 | A |
5610998 | Kondo | Mar 1997 | A |
5625461 | Okamoto et al. | Apr 1997 | A |
5625715 | Trew et al. | Apr 1997 | A |
5636316 | Oku et al. | Jun 1997 | A |
5649053 | Kim | Jul 1997 | A |
5663764 | Kondo et al. | Sep 1997 | A |
5673357 | Shima | Sep 1997 | A |
5675565 | Taguchi et al. | Oct 1997 | A |
5677734 | Oikawa et al. | Oct 1997 | A |
5689302 | Jones | Nov 1997 | A |
5699475 | Oguro et al. | Dec 1997 | A |
5703889 | Shimoda et al. | Dec 1997 | A |
5724099 | Hamdi et al. | Mar 1998 | A |
5724369 | Brailean et al. | Mar 1998 | A |
5737022 | Yamaguchi et al. | Apr 1998 | A |
5750867 | Williams et al. | May 1998 | A |
5751361 | Kim | May 1998 | A |
5751743 | Takizawa | May 1998 | A |
5786857 | Yamaguchi | Jul 1998 | A |
5790195 | Ohsawa | Aug 1998 | A |
5796786 | Lee | Aug 1998 | A |
5805762 | Boyce et al. | Sep 1998 | A |
5808994 | Tanaka et al. | Sep 1998 | A |
5809041 | Shikakura et al. | Sep 1998 | A |
5809231 | Yokoyama et al. | Sep 1998 | A |
5852470 | Kondo et al. | Dec 1998 | A |
5861922 | Murashita et al. | Jan 1999 | A |
5878183 | Sugiyama et al. | Mar 1999 | A |
5896466 | Kim | Apr 1999 | A |
5903481 | Kondo et al. | May 1999 | A |
5928318 | Araki | Jul 1999 | A |
5936674 | Kim | Aug 1999 | A |
6067636 | Yao et al. | May 2000 | A |
6137915 | Chai | Oct 2000 | A |
6266454 | Kondo | Jul 2001 | B1 |
Number | Date | Country |
---|---|---|
0 398 741 | Nov 1990 | EP |
0 527 611 | Aug 1992 | EP |
0 558 016 | Feb 1993 | EP |
0 566 412 | Apr 1993 | EP |
0 571 180 | May 1993 | EP |
0 592 196 | Oct 1993 | EP |
0 596 826 | Nov 1993 | EP |
0 605 209 | Dec 1993 | EP |
0 610 587 | Dec 1993 | EP |
0 597 576 | May 1994 | EP |
0 651 584 | Oct 1994 | EP |
0 680 209 | Apr 1995 | EP |
0 746 157 | May 1996 | EP |
0 833 517 | Apr 1998 | EP |
2 320 836 | Nov 1997 | GB |
404372779 | Dec 1992 | JP |
7-67028 | Mar 1995 | JP |
410240796 | Sep 1998 | JP |
WO9607987 | Sep 1995 | WO |
WO 9921285 | Oct 1998 | WO |
99 21090 | Apr 1999 | WO |
Number | Date | Country | |
---|---|---|---|
20030133618 A1 | Jul 2003 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 09342296 | Jun 1999 | US |
Child | 10358567 | US |