This application claims the benefit, under 35 U.S.C. §365 of International Application PCT/EP2011/069855, filed Nov. 10, 2011, which was published in accordance with PCT Article 21(2) on Jun. 7, 2012 in English and which claims the benefit of European patent application No. 10306308.7, filed Nov. 29, 2010.
The invention is made in the field of reconstruction of self-similar textured image regions.
Self-similar textured regions in images induce high bit rates in encoding, in particular but not only in high definition encoding, while at the same time raise only limited interest by an observer. The interest of the observer is only focused on such regions in case the region shows discontinuities in structure or luminance.
In principal, the self-similarity of the texture allows for reducing the bit rate by encoding of only a sample part of the textured region and reconstruction of pixels of the non-encoded further part of the textured region by copying pixels of the sample part. Reconstruction is also known as synthesizing.
For instance, Li-Yi Wei and Marc Levoy: “Fast texture synthesis using tree-structured vector quantization”, 2000, Proc. of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH '00), ACM Press/Addison-Wesley Publishing Co., New York, N.Y., USA, 479-488, proposed a pixel-based reconstruction scheme in which a part of a texture region is reconstructed based on a pixel by pixel selection process.
Selection of a sample pixel for use for reconstruction of a given non-encoded pixel can be based on maximal similarity of a neighbourhood of the given non-encoded pixel with a neighbourhood of the sample pixel among sample pixel candidates.
The selection of the sample pixel dependents on the size of the neighbourhood used for selection. Different neighbourhood sizes can result in different sample pixels for reconstruction of a same non-encoded pixel. Neighbourhoods larger than a feature size of the texture in principle are suited for reproducing the texture well but require large sample parts for actually reproducing the texture in good visual quality. Neighbourhoods smaller than the feature size of the texture are resulting in poor reproductions. Thus, it is desirable to provide a method and device for reconstruction of a self-similar textured region of an image which adaptively selects the neighbourhood size close to the feature size of the texture.
The inventors therefore propose the method of claim 1 and the device of claim 3.
Said method comprises determining pixels of a part of the self-similar textured region by copying sample pixels from a sample part of the self-similar textured region, the sample pixels being selected using a neighbourhood matching, wherein a size of neighbourhoods used for matching is selected based on an analysis of descriptors computed from coefficients of DCT transform of differently sized blocks of the sample part.
In an embodiment, the method further comprises the steps of receiving a bit stream in which the sample part of the self-similar textured region is encoded and decoding the sample part
In a further embodiment of the method the size of the neighbourhoods used for matching is adaptively selected by determining the coefficients by applying a DCT transformation on each of the differently sized blocks, determining, for each of the differently sized blocks, an associated sequence of descriptors wherein each descriptor is determined using sums of coefficients summed along parallel paths, selecting that sequence of descriptors which is the sequence associated with the smallest block size among those sequences which are not monotone decreasing and determining the size of the neighbourhoods using a size of the differently sized block associated with determined sequence.
Said device comprises a processing device. The processing device is adapted for determining pixels of a further part of the self-similar textured region by copying sample pixels from a sample part of the self-similar textured region, the sample pixels being selected using a neighbourhood matching, wherein the processing device is adapted for selecting a size of the neighbourhoods used for matching using an analysis of sequences of descriptors computed from coefficients of DCT transform of differently sized blocks of the sample part.
In an embodiment, said device further comprises a receiver and a decoder. The receiver is adapted for receiving a bit stream in which the sample part of the self-similar textured region is encoded and the decoder is adapted for decoding the sample part.
The inventive concept can also be applied on an encoding device for determining suitable sample part size and/or for determining the sized of the non-encoded part or on both.
If applied on encoder side, the selected neighbourhood size can further be transmitted to a decoder.
Exemplary embodiments of the invention are illustrated in the drawings and are explained in more detail in the following description. The exemplary embodiments are explained only for elucidating the invention, but not limiting the invention's disclosure, scope or spirit defined in the claims.
In the figures:
Textures, defined as stationary regions, can vary from regular or near-regular textures, in which a pattern is recopied at different locations (brick wall), to stochastic noisy textures (sand, grass . . . ).
This invention proposes a new characterization step, which can be applied in the field of computer graphics for autoadaptive algorithm and image/video compression scheme. The invention is based on the study of descriptors computed from the 2D Discrete Cosine Transform (DCT) coefficient depicted in
The DCT coefficients are computed from:
where I(m,n) corresponds to the luminance value at position (m,n), N is the size of the support block,
Then
where C corresponds to the DCT coefficient at position (λ cos θ, λ sin θ) in transform domain.
The descriptor vector components characterize the texture of the transformed block. First information lies in the feature sizes of patterns inside a texture region. Synthesis algorithms, especially pixel-based techniques, are based on the comparison between groups of pixels.
The inventors noticed from texture results computed with different sizes of neighborhood that sizes greater than the feature size of the snake skin are able to produce visually good results, while lower sizes are definitely not. Moreover, this size must also not to be too large because it reduces the candidate set of pixels depending of the patch size.
A first component of the descriptor vector is computed from coefficients crossed over by circle C1 shown in
Therefore, if the first component has the greatest value, the descriptor vector is monotone and decreasing and the block size is not sufficient for being used as neighborhood in texture synthesis algorithms. Conversely, if other components are predominant (non monotone curve), the main pattern appears to be included into the block. Thus reconstruction can be based on a comparison of neighbourhoods with a correspondingly selected size.
According to the previous reasoning, an 8×8 size is not enough, while with size 16×16, higher frequencies are predominant, so the main pattern can be include in a 16×16 block. Using a 32×32 neighborhood is also possible but it means that the patch has to be large enough to contain a large set of 32×32 blocks, which is not always possible in video compression domain for example, where segmented texture regions are not large enough.
Texture synthesis can be used as an efficient compression technique for encoding of texture regions. Indeed, detailed texture regions are difficult to deal with, using classical transform-based compression schemes.
In this context, DCT descriptors can be useful for different aspects.
First, after having segmented textures at encoder side, descriptors can be used at encoder side to decide whether texture synthesizer at decoder side is able to reconstruct this kind of texture or not. Further, the encoder can decide how much of the textured region can be omitted from encoding and how much is needed as sample by the decoder. And, the encoder can send the neighbourhood size for this texture region as side information. If no side information regarding the neighbourhood size is received by the decoder, the decoder can determine DCT descriptors from a received sample and then use them for determining an appropriate neighbourhood size for the reconstruction process.
A variety of parallel paths for computing descriptor vector components can be used.
Coefficients correspond to the sum of DCT coefficients crossed by dashed lines depicting exemplary paths. Since DC is not kept, the exemplary descriptor vector gets three components as shown by arrows.
The invention allows getting texture features to parameterize texture synthesis algorithms, depending on type, shape and featured size of texture patterns to synthesize.
Number | Date | Country | Kind |
---|---|---|---|
10306308 | Nov 2010 | EP | regional |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/EP2011/069855 | 11/10/2011 | WO | 00 | 11/14/2013 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2012/072402 | 6/7/2012 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
6185329 | Zhang et al. | Feb 2001 | B1 |
7136072 | Ritter | Nov 2006 | B2 |
7508983 | Enomoto | Mar 2009 | B2 |
8285064 | Haddad et al. | Oct 2012 | B2 |
8498335 | Holcomb et al. | Jul 2013 | B2 |
20050002569 | Bober et al. | Jan 2005 | A1 |
20090003441 | Sekiguchi et al. | Jan 2009 | A1 |
20090086814 | Leontaris et al. | Apr 2009 | A1 |
Number | Date | Country |
---|---|---|
1570977 | Jan 2005 | CN |
101360174 | Feb 2009 | CN |
101573730 | Nov 2009 | CN |
07184158 | Jul 1995 | JP |
11112979 | Apr 1999 | JP |
2004507177 | Mar 2004 | JP |
2004348741 | Dec 2004 | JP |
WO0215586 | Feb 2002 | WO |
WO2009091080 | Jul 2009 | WO |
Entry |
---|
Alparone L et al:“Texture-Based Analysis Techniques for the Classification of Radar Images”, IEEE Proceedings F. Communications, Radar & SignalProcessing, Institution. |
Joan S Weszka et al:“A comparative Study of Texture Measures for Terrain Classification”, IEEE Transactions on Systems, Man and Cybernetics, IEEE Inc. New York, US, vol. SMC-5, No. 4. |
Manian V et al:“On the use of transform features for SAR image classification”, Geoscience and Remote Sensing Symposium Proceedings, IGARSS '98. Seattle, WA, USA. |
Strand J et al: “Local Frequency Features for Texture Classification”, Pattern Recognition, Elsevier, GB, vol. 27, No. 10. |
Wei L-Y et al:“Fast Texture Synthesis Using Tree-Structured Vector Quantization” Computer Graphics.SIGGRAPH 2000, Conference Proceedings. New Orleans. |
Search Report Dated Feb. 17, 2012. |
Racape et al., “Adaptive Texture Characterization/Synthesis for Advanced Compression Scheme”, 19th European Signal Processing Conference, Barcelona, Spain, Aug. 29, 2011, pp. 1-10. |
Ashikhmin, “Synthesizing Natural Textures”, Proceedings of ACM Symposium on Interactive 3D Graphics, Research Triangle Park, North Carolina, USA, Mar. 19, 2001, pp. 277-286. |
Kwatra et al., “Graphcut Textures: Image and Video Synthesis Using Graph Cuts”, Proceedings of ACM Symposium on Interactive 3D Graphics, Monterey, California, USA, Apr. 27, 2003, pp. 277-286. |
Kwatra et al., “Texture Optimization for example-based synthesis”, Proceedings of ACM Symposium on Computer Graphics and Interactive Techniques, Los Angeles, California, USA, Jul. 31, 2005, pp. 795-802. |
Dumitras et al., “A texture replacement method at the encoder for bit-rate reduction of compressed video”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, No. 2, Feb. 2003, pp. 163-175. |
Zhu et al., “Video coding with spatio-temporal texture synthesis”, 2007 IEEE International Conference on Multimedia and Expo, Beijing, China, Jul. 2, 2007, pp. 112-115. |
Ndjiki-Nya et al., “Generic and robust video coding with texture analysis and synthesis”, 2007 IEEE International Conference on Multimedia and Expo, Beijing, China, Jul. 2, 2007, pp. 1447-1450. |
Deforges et al., “Color LAR Codec: A color image representation and compression scheme based in local resolution adjustment and self-extracting region representation”, IEEE transactions on circuits and systems for video technology (TCSVT), vol. 17, No. 8, Aug. 2007, pp. 974-987. |
Sabha et al., “Texture Synthesis using Exact Neighborhood Matching”, Computer Graphics forum, vol. 26, No. 2, Jun. 2007, pp. 131-142. |
Suhring, “H264 AVC software coordination”, http://iphome.hhi.de/suehring/tml/download/KTA/, Jan. 2010, p. 1. |
Amadasun et al., “Textural Features Corresponding to Textural Properties”, IEEE Transactions on Systems, Man and Cybernetics, vol. 19, No. 5, Sep./Oct. 1989, pp. 1264-1274. |
Smach et al., “Generalized Fourier Descriptors with Applications to Objects Recognition in SVM Context”, Journal of Mathematical Imaging and Vision, vol. 30, Dec. 1, 2007, pp. 43-71. |
Hong et al., “Synthesising Textures Using Variable Neighbourhood”, Microcomputer Information, vol. 22, No. 6-1, Jun. 2006, pp. 225-227. English Abstract. |
Number | Date | Country | |
---|---|---|---|
20140064630 A1 | Mar 2014 | US |