The present invention relates generally to compression of still image and moving video data, and more particularly to the application of calculation of statistics of the behavior of the quantized transform representation of the video from the measured variances and the measured correlations in pixel space. Once the statistical behavior of the video is modeled, the video streams can be collected into successive refinement streams (progressive mode), and the probabilities of the constructed symbols can be calculated for the purposes of entropy removal. The measured variances and correlations suffice to reconstruct the compressed video streams for any frame or group of frames.
In accordance with one aspect of the invention, a method is provided for the optimal rearrangement of components into a transmission stream based on the calculated variance of individual quantized transform components from the measured variance and correlation of the raw untransformed visual samples.
A second aspect of the invention provides a method for the optimal calculation of entropy reduction tables for a transmission stream based on the calculated symbol probabilities based on the calculated probability distributions of individual quantized transform components.
A final aspect of the invention provides a method for the parallel construction of transmission stream rearrangement, symbol construction and entropy tables between compressing apparatus and decompressing apparatus via communication of the measured variances and correlations of the raw untransformed visual samples.
As depicted in
The JPEG zig-zag transmission order illustrated in
As depicted in
The original JPEG specification provides for a pre-defined entropy pre-shared encoding table as depicted in
As illustrated in
Each JPEG-2000 progressive transmission approach described above in
Much effort has been expended in the incremental increase of efficiency in the communication of entropy coding statistics between compressing and decompressing apparatuses, but no significant advances can be claimed over the prior-are techniques described herein. The current invention discloses a far more efficient means of developing entropy tables independently in compressor and decompessor.
The quantized transform components of said one subblock 6110 of said plurality of multidimensional subblocks processed through a step 6120 to reorder quantized transform components into order of greatest probability of zero (lowest variance). Said step 6120 uses said calculated variances for said quantized transform components from said step 6030 to perform its sort processing.
Said reordered quantized transform components are then processed through a step 6130 of collection of said reordered quantized transform components into symbols. Each said collected symbol is then processed through a step 6140 of entropy coding of said symbol into a short sequence of bits. Said step 6140 uses said calculated relative probabilities for each symbol from said step 6040 in its entropy-removing calculations.
Said short sequence of bits is finally processed through an aggregation step 6150 to concatenate generated bit sequences into a transport stream.
Once said subsampled sequential stream of visual data 7010 has been processed through said compression unit 5150, a higher-resolution sequential stream of visual data less subband data 7110 may be processed. Said higher-resolution sequential stream of visual data less subband data 7110 is comprised of sequential stream of visual data 5110 where each and every coefficient comprising said subsampled sequential stream of visual data 7010 is set to 0 with a variance of 0. Said higher-resolution sequential stream of visual data less subband data 7110 is factored into a plurality of multidimensional subblocks 7120. Said plurality of multidimensional subblocks 7120 is then processed subblock by subblock by said compression unit 5150 to produce a sequence of compressed bits for transmission.
A covariance matrix Apixel,x 8020 is composed of the multiplication of said measured pixel variance in the x direction by the autocorrelation matrix derived from said measured pixel correlation in the x direction. Similar matrices Apixel,y and Apixel,z (in the case of three-dimensional said multidimensional subblocks 7120) will utilize the measured pixel variance, pixel correlations and length of each dimension of the said multidimensional subblocks 7020.
DCT covariance matrix x 8030 is calculated as the product of said matrix DCTx 8010, said covariance matrix Apixel,x 8020, and the transpose of said matrix DCTx 8010.
The variance of the quantized transform component 8040 of index u,v,w within said multidimensional subblocks 7020, a-2u,v,w, is calculated as the product of the trace of said DCT covariance matrix Ax 8030 with the trace of said DCT covariant matrix Av (and with the trace of said DCT covariant matrix Az if said multidimensional subblocks 7020 are three-dimensional) divided by the quantizer value for said quantized transform component 8040 of index u,v,w within said multidimensional subblocks 7020.
The probability Pu,v,w(x==O) 9020 that any quantized transform component of index u,v,w within said quantized transform subblock is 0 is calculated from the Cumulative Distribution Function of a normal distribution with expectation of 0 and variance equal to that of said quantized transform component of index u,v,w within said quantized transform.
The probability Pu,v,w(log 2(x)==n) 9030 that any quantized transform component of index u,v,w within said quantized transform subblock has n bits in its representation is calculated from the Cumulative Distribution Function of a normal distribution with expectation of 0 and variance equal to that of said quantized transform component of index u,v,w within said quantized transform.
A typical symbol Su,v,w(r, b) 9040 comprised of a run length of r zeros followed by a non-zero value of length b is calculated as the conditional probability the each symbol in the order of said rearrangement of said quantized transform component within said quantized transform subblock. The probability of the ith quantized transform component following quantized transform component index u,v,w within said quantized transform subblock being 0 is written P(u,v,w)+i(x==O). The probability of the rth quantized.
While the present invention has been described in its preferred version or embodiment with some degree of particularity, it is understood that this description is intended as an example only, and that numerous changes in the composition or arrangements of apparatus elements and process steps may be made within the scope and spirit of the invention. In particular, rearrangement and recalculation of statistics may be made to support various modes of progressive transmission, including spectral banding or bitwise refinement. Further, pixel statistics may be measured and transmitted on a per-block or global basis, and may be measured in each dimension or averaged across all dimensions. Block sizes may also be taken to be as large as the entire frame, as would be typical when using the wavelet transform.
With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claimed invention.
Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent to those of skill in the art upon reading the above description. The scope of the invention should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the arts discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the invention is capable of modification and variation and is limited only by the following claims.
All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those skilled in the art unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.
This application is a continuation of U.S. application Ser. No. 14/266,645, filed on Apr. 30, 2014, which is a non-provisional and claims benefit of a U.S. Provisional Application No. 61/818,419, filed on May 1, 2013.
Number | Name | Date | Kind |
---|---|---|---|
4780762 | Nagasaki | Oct 1988 | A |
4894713 | Delogne et al. | Jan 1990 | A |
5629780 | Watson | May 1997 | A |
5737020 | Hall et al. | Apr 1998 | A |
5751358 | Suzuki | May 1998 | A |
5768534 | Guillote et al. | Jun 1998 | A |
5933193 | Niesen | Aug 1999 | A |
5936669 | Niesen | Aug 1999 | A |
5953460 | Wu | Sep 1999 | A |
6023531 | Peters | Feb 2000 | A |
6222887 | Nishikawa | Apr 2001 | B1 |
6239811 | Westwater | May 2001 | B1 |
6252989 | Geisler et al. | Jun 2001 | B1 |
6256413 | Hirabayashi | Jul 2001 | B1 |
6314208 | Konstantinides et al. | Nov 2001 | B1 |
6347157 | Chui | Feb 2002 | B2 |
6456744 | Lafe | Sep 2002 | B1 |
6801573 | Zheng | Oct 2004 | B2 |
6894628 | Marpe et al. | May 2005 | B2 |
6990247 | Schwartz | Jan 2006 | B2 |
7006568 | Gu et al. | Feb 2006 | B1 |
7076104 | Keith et al. | Jul 2006 | B1 |
7123656 | Reznik | Oct 2006 | B1 |
7599840 | Mehrotra et al. | Oct 2009 | B2 |
7634145 | Keith et al. | Dec 2009 | B2 |
7684981 | Thumpudi et al. | Mar 2010 | B2 |
7889790 | Sun | Feb 2011 | B2 |
8179974 | Tu et al. | May 2012 | B2 |
8236068 | Aramata et al. | Aug 2012 | B2 |
8306340 | Ceperkovic et al. | Nov 2012 | B2 |
8326067 | Furbeck | Dec 2012 | B2 |
8351500 | Shimofure | Jan 2013 | B2 |
8432967 | Hosaka | Apr 2013 | B2 |
8436755 | Korodi et al. | May 2013 | B2 |
20010004400 | Aoki | Jun 2001 | A1 |
20020168022 | Kim | Nov 2002 | A1 |
20020186772 | Li | Dec 2002 | A1 |
20030090591 | Concion | May 2003 | A1 |
20030112333 | Chen et al. | Jun 2003 | A1 |
20030142098 | Kim | Jul 2003 | A1 |
20030169928 | Stanek | Sep 2003 | A1 |
20030190086 | Kim | Oct 2003 | A1 |
20040001611 | Celik | Jan 2004 | A1 |
20040061709 | Park | Apr 2004 | A1 |
20040062448 | Zeng et al. | Apr 2004 | A1 |
20040066974 | Karczewicz | Apr 2004 | A1 |
20040105590 | Akimoto | Jun 2004 | A1 |
20040151243 | Bhaskaran et al. | Aug 2004 | A1 |
20040190610 | Song | Sep 2004 | A1 |
20040208362 | Suzuki | Oct 2004 | A1 |
20050008224 | Caruso | Jan 2005 | A1 |
20050105608 | Coleman | May 2005 | A1 |
20050157791 | Sun | Jul 2005 | A1 |
20050165517 | Reich | Jul 2005 | A1 |
20050190836 | Lu et al. | Sep 2005 | A1 |
20050207660 | Edgar | Sep 2005 | A1 |
20050281479 | Song | Dec 2005 | A1 |
20050286629 | Dumitras | Dec 2005 | A1 |
20060062304 | Hsia | Mar 2006 | A1 |
20060098897 | Dewaele | May 2006 | A1 |
20060155531 | Miller | Jul 2006 | A1 |
20060164268 | Lee | Jul 2006 | A1 |
20060165311 | Watson | Jul 2006 | A1 |
20060292837 | Gomila | Dec 2006 | A1 |
20070018906 | Visnovsky | Jan 2007 | A1 |
20070071090 | Peng | Mar 2007 | A1 |
20070081589 | Kim | Apr 2007 | A1 |
20070121731 | Tanizawa | May 2007 | A1 |
20070195737 | Walker et al. | Aug 2007 | A1 |
20070230565 | Tourapis | Oct 2007 | A1 |
20070263938 | Lee et al. | Nov 2007 | A1 |
20070273939 | Kishida | Nov 2007 | A1 |
20080071764 | Omi | Mar 2008 | A1 |
20080186348 | Yanai | Aug 2008 | A1 |
20080192824 | Lim | Aug 2008 | A1 |
20080298636 | Rasheed | Dec 2008 | A1 |
20090092099 | Gu et al. | Apr 2009 | A1 |
20090232408 | Meany | Sep 2009 | A1 |
20100054342 | Jang | Mar 2010 | A1 |
20100286991 | Hedelin | Nov 2010 | A1 |
20100322597 | Gharavi-Alkhansari | Dec 2010 | A1 |
20100324850 | Jung | Dec 2010 | A1 |
20100329329 | Reznik | Dec 2010 | A1 |
20110075897 | Dekel | Mar 2011 | A1 |
20110255589 | Saunders et al. | Oct 2011 | A1 |
20110268182 | Joshi | Nov 2011 | A1 |
20110286525 | Kamisli | Nov 2011 | A1 |
20120057629 | Shi et al. | Mar 2012 | A1 |
20120133361 | Gross | May 2012 | A1 |
20120140819 | Kim | Jun 2012 | A1 |
20130003829 | Misra | Jan 2013 | A1 |
20130089137 | Korman | Apr 2013 | A1 |
20130114703 | DeForest | May 2013 | A1 |
20130195206 | McCarthy | Aug 2013 | A1 |
20130287104 | Jeong | Oct 2013 | A1 |
20130336594 | Dorairaj | Dec 2013 | A1 |
20140003728 | Aoki | Jan 2014 | A1 |
20140051108 | Bernardi | Feb 2014 | A1 |
20140115472 | Mochinaga | Apr 2014 | A1 |
20140169407 | Terry | Jun 2014 | A1 |
20140267432 | Dong | Sep 2014 | A1 |
20140270331 | Zhao | Sep 2014 | A1 |
20140286434 | Lee | Sep 2014 | A1 |
20140327737 | Westwater | Nov 2014 | A1 |
20140328406 | Westwater | Nov 2014 | A1 |
20140376620 | Kondo | Dec 2014 | A1 |
20150003707 | Amon | Jan 2015 | A1 |
20160309189 | Westwater | Oct 2016 | A1 |
20160309190 | Westwater | Oct 2016 | A1 |
Entry |
---|
Nadenau, “Wavelet-Based Color Image Compression: Exploiting the Contrast Sensitivity Function”, IEEE Transactions on Image Porcessing, vol. 12, No. 1, Jan. 2003, pp. 58-70. |
Office action for U.S. Appl. No. 14/266,757, dated Sep. 1, 2016, Westwater, “Method and Apparatus to Perform Optimal Visually-Weighed Quantization of Time-Varying Visual Sequences in Transform Space”, 20 pages. |
Office action for U.S. Appl. No. 14/266,645, dated Sep. 28, 2016, Westwater, “Method and Apparatus to Perform Correlation-Based Entropy Removal from Quantized Still Images or Quantized Time-Varying Video Sequences in Transform”, 30 pages. |
Office action for U.S. Appl. No. 15/189,187, dated Sep. 8, 2016, Westwater, “Method and Apparatus to Perform Optimal Visually-Weighed Quantization of Time-Varying Visual Sequences in Transform Space”, 20 pages. |
Number | Date | Country | |
---|---|---|---|
20160309190 A1 | Oct 2016 | US |
Number | Date | Country | |
---|---|---|---|
61818419 | May 2013 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14266645 | Apr 2014 | US |
Child | 15189242 | US |