The invention relates to a method of processing a digital image encoded and decoded in accordance with a pixel-block encoding technique, said technique being suitable to provide a motion vector per block of pixels and a quantization step per image.
The invention also relates to a post-processing device using such a method.
The invention also relates to a video decoder and a video encoder using such a post-processing device.
The invention further relates to a portable apparatus comprising such a video decoder.
The invention also relates to a computer program using such a method.
Finally, the invention relates to a signal intended to convey such a program.
The invention notably finds its application in the processing of digital images which are encoded and decoded at a low rate in accordance with an encoding technique such as MPEG-4 or JVT (Joint Video Team).
Because of the growing need to transmit and store digital data, the compression techniques or, in other words, the techniques of encoding digital image sequences have become widespread. The most conventional techniques of compressing image sequences such as those defined by the MPEG standard (Motion Picture Expert Group), or ITU-T VCEG, use a motion compensation based on a block-matching algorithm and a block transform, for example, the discrete cosine transform or DCT. The sequence of images is divided into groups of images and such a group comprises an intraframe, or I frame, encoded in an independent manner, followed by several predicted frames encoded in a differential manner with respect to the preceding or subsequent frame. The block transform has the advantage that it provides strong compression rates. In contrast, the subsequent quantization step creates block effects in the decoded digital images, which leads to a degradation of their quality. Indeed, the quantization is coarser as the encoding rate is smaller. Consequently, the degradation due to the quantization step may range from an imperceptible level, when the encoding rate is high, to an annoying level, when it is low.
To remedy this problem, numerous techniques of post-processing said decoded digital images have been developed so as to correct said block effects. The blocks of 8×8 pixels used for encoding a digital image form a grid on this image, which is referred to as principal grid. The block effects appear on this grid. The majority of techniques for correcting blocks use the hypothesis that the position of said grid on the decoded image is known and remains fixed throughout the sequence of images.
Unfortunately, this hypothesis is not always true. For reasons of D/A and A/D conversions (for example, for the transmission of the sequence of encoded images or within a television) and subsequent to the use of possible pre-processing algorithms for the digital image sequence, an original image may turn out to be shifted by several pixels and the size of the principal grid may be modified. Now, the decoder does not have any information on this subject. One consequence of such a phenomenon is that the effectiveness of the block correction techniques is directly affected.
To obviate this problem, techniques for detecting the principal grid of a decoded digital image have been developed, for example, a method of detecting three regular grid sizes based on blocks of 8×8, 10-11×8 and 12×8 pixels, and described in the international application WO 01/20912.
It is an object of the invention to propose a solution for detecting the presence of a secondary grid in a digital image which has been encoded and subsequently decoded at a low rate in accordance with a block encoding technique, which method provides a localization of said grid while keeping a small complexity.
Indeed, it is not rare that a secondary grid appears in addition to the principal grid in a digital image which has been encoded and then decoded at a low rate. This secondary grid is due to a motion compensation per block of pixels, which is susceptible of creating block effects outside the borders of the principal grid, which effects are not corrected by a too coarse quantization.
The block effects which are due to the secondary grid are not generally taken into account in the conventional post-processing techniques, which ignore the existence of said grid.
To solve such a problem, the method as described in the opening paragraph is characterized in that it comprises the steps of.
The method according to the invention relates to digital images which are encoded and decoded at a low rate. As has been elucidated hereinbefore, it is indeed the low rate at which the phenomenon of the secondary grid is most susceptible to appear.
A first advantage of this method is that, while it provides a localization of the secondary grid in a decoded image, it serves for pre-processing before using block-effect correction techniques or techniques of evaluating the quality of said image and provides the possibility of considerably improving the effectiveness of these low-rate techniques. Indeed, knowing the localization of the secondary grid allows a block correction technique comprising, for example, a filtering step using a filter and centering said filter on the block effects due to said secondary grid. In the case of a technique of evaluating the quality, comprising a step of counting block effects, the localization of the secondary grid allows a more precise count and thus a better evaluation of the quality of the decoded image.
A second advantage of this method is that it has a small complexity. Indeed, the selection step allows elimination of the images whose quantization step is smaller than a predetermined threshold, i.e. only the images that have been encoded and decoded in a sufficiently coarse manner are preserved. As far as the step of detecting pixel blocks is concerned, it eliminates a certain number of candidate blocks so as to finally preserve only those which are uniform and whose motion vector is non-zero and with an amplitude which is smaller than a predetermined amplitude threshold. This small complexity renders it possible to perform the method according to the invention in real time and thus use it within a decoder incorporated in a portable apparatus, such as a personal digital assistant or a mobile telephone.
The invention also relates to a device using such a method.
These and other aspects of the invention are apparent from and will be elucidated, by way of non-limitative example, with reference to the embodiment(s) described hereinafter.
In the drawings:
The invention relates to a method of processing a digital image belonging to a group of images and encoded and decoded in accordance with a block encoding technique. It is applicable to any encoding technique which is suitable for supplying a motion vector per pixel block and a quantization step per image. The technique used is, for example, MPEG4 or JVT (H.26L has become JVT, which is the object of a unified standardization effort by the standardization committees ISO/IEC MPEG and ITU-T VCEG.)
It should be noted that
In an encoding scheme of the type MPEG-4 or JVT, the sequence of digital input images IS is divided into groups of images IG such as shown in
It is thus at the level of said quantization stage that encoding errors are created.
It is this quantization stage which is at the origin of the ultimate appearance of a secondary grid in pixel blocks of certain images of the sequence at the moment of decoding. Indeed, as is shown in
The phenomenon may then easily propagate to the subsequent frames. Nevertheless, it attenuates progressively because of the motion compensations and the encoding operations for the successive error images.
It should also be noted that this secondary grid phenomenon particularly occurs in uniform zones of the image. Indeed, if the block B is present in a uniform zone, it is certainly also the case for the block B′, unless the block-matching technique is not associated with them. The probability that their difference would have an intensity which would be smaller than the quantization step is then quite greater if a difference between two textured blocks comprising, for example, object contours were concerned.
It is thus an object of the method according to the invention to detect said secondary grid in the pixel blocks of a decoded digital image. Such a method comprises three steps which are illustrated in
As this is the case in the majority of standards, it will hereinafter be assumed that a block comprises 8×8 pixels which will be denoted B8×8. However, the invention is evidently not limited to this particular case.
The first step is a step SELECT of selecting said decoded digital image DI as a function of its quantization step Q, intended to select said image if its quantization step is larger than a predetermined threshold. In a preferred embodiment of the invention, said threshold is fixed at 25 on a scale of values between 1 and 31, for example, for the MPEG-4 standard, i.e. only the decoded images DI whose quantization step Q is larger than 25 are selected.
Let us consider a selected decoded image SDI satisfying the afore mentioned criterion. The next step is a step DETECT of detecting pixel blocks, intended to detect the blocks B8×8 of said selected decoded image having a secondary grid.
In the preferred embodiment, all the pixels 8×8 of the selected image SDI are subjected to this step, but this is not obligatory. One could, for example, consider that the blocks situated in the center of the selected decoded image SDI are more important to the human eye and, for reasons of complexity, one might choose the processing operation to be limited to the processing of these blocks.
Said detection step DETECT comprises three sub-steps, also shown in
|m1−m2|<S, with:
In the preferred embodiment of the invention, S is chosen to be equal to 3 by virtue of the known properties of the human visual system. The phenomenon of the secondary grid is actually only detectable to a human eye when the block B8×8 responds to the previously mentioned conditions, in other words, when the zone considered is relatively uniform. In this case, said detection step UNI supplies a block of 8×8 pixels, referred to as uniform block B8×8
The next sub-step is a sub-step MV_SELECT of selecting a uniform block, intended to select a uniform block if its motion vector is non-zero and has an amplitude which is smaller than a predetermined amplitude threshold, as given below:
in which vx and vy are the horizontal and vertical components of the motion vector MV associated with the block B8×8
Consequently, a block B8×8
At this stage, we have identified the blocks of 8×8 pixels of the selected decoded image SDI comprising block effects which are due to the presence of a secondary grid.
The next sub-step of the method according to the invention, referred to as the sub-step LOC of localizing a secondary grid within a selected uniform block B8×8
In the preferred embodiment of the invention, the positions of the pixels of the predicted frame P which are present on a secondary grid are “highlighted” in a localization card Loc, i.e. Loc(i0+vx,j0+q)=1, in which q is between 0 and 7 and Loc(i0+p,j0+vy), in which p is between 0 and 7. Other ways of presenting the results of the localization sub-step Loc are of course also envisageable, but this mode has the advantage of being simple.
In the next step, the selected decoded image SDI as a whole is considered. It concerns a step VIS of evaluating a visibility measurement V, which is characterized in that it evaluates the visibility of the secondary grid in said selected decoded image SDI.
Such an evaluation may be made in different ways. It may be particularly based on a local measurement of intensity or contrast variation in the neighborhood of a pixel (i,j) belonging to a block B8×8
However, it should be recalled that such a block of 8×8 selected pixels B8×8
It is also envisageable to resort to the general knowledge about the human visual system for evaluating the visibility of the secondary grid. It indicates, for example, that a variation of contrast is much more visible in a uniform texture zone of average intensity (about 70 to 90 units of contrast out of 255 possible levels) than in very clear or, in contrast, very dark uniform zones. Such considerations may be used for standardizing a scale of visibility measurement of the secondary grid in a block B8×8
In the preferred embodiment of the invention, said evaluation of the visibility relates to the position of the predicted frame P in a group of images IG. It does not take the local extent of contrast into account, nor the knowledge about the human visual system. On the other hand, it depends on the position Pos(SDI) of the image SDI in the group of images IG.
In accordance with an empirical study, shown in
In the preferred embodiment, the visibility measurement is utilized for weighting the values of the pixels of the localization card of the secondary grid in the decoded image. A weighted localization card Ploc of the secondary grid is supplied. In other words, for a pixel (i,j) of a block B8×8
An advantage of this evaluation step VIS as used in the preferred embodiment of the invention is its simplicity. It should be recalled that the object of the method of detecting the secondary grid according to the invention is its small complexity and rapid execution so that it can be accommodated in portable video decoders.
The obtained visibility measurement Vk combined with the localization card Loc is in conformity with the principal object of the method according to the invention, namely localizing the zones of the decoded image in which the presence of a secondary grid is visible to the human eye and disturbing, with a view to, for example, a corrective post-processing operation.
Let us consider, for example, the case of a method of post-processing block effects, which method comprises a primary filtering step. At least one filter is used in said step. As has been stated hereinbefore, such a method generally starts from the following hypotheses:
Said filter is thus applied to the borders between two blocks of pixels. Knowing the localization of the secondary grid, such a method of correcting block effects may integrate a second filtering step, referred to as secondary filtering step, comprising at least one filter. As is shown in
Let us now consider a method of measuring the quality of a decoded image, which method comprises a step of counting a number of block effects in said image. Such a method generally starts from the same hypotheses as the previously mentioned post-processing method, namely that the localization of the principal grid is known and that the block effects are present on this principal grid. Said counting step counts, for example, a block effect per grid segment present in a block, or a number of pixels situated on a grid. Knowing the localization of the secondary grid, such a method of measuring the quality may thus add the block effects due to the secondary grid to the number of block effects due to the principal grid. The extent of quality obtained is then more realistic. If a weighted localization card of the secondary grid is available, such a method may even advantageously refine its extent of quality by weighting a contribution of a block effect with the calculated number of block effects as a function of the visibility associated therewith.
Said post-processed decoded image PPDI is subsequently supplied to a display device DISP suitable for displaying said post-processed decoded image on a screen.
The processing device SEC_GRID according to the invention may thus improve the performance of a post-processing device PP and consequently that of a decoder DEC or a video encoder ENC.
The invention is not limited to the embodiments which have been described by way of example. Modifications or improvements are possible without departing from the scope of the invention. The invention is not limited to the detection of a secondary grid in images which have been encoded and subsequently decoded at a low rate in accordance with the MPEG-4 or H.26L encoding techniques. It is also applicable to images which have been decoded by means of any technique using blocks and motion compensation.
The description above with reference to FIGS. 1 to 12 illustrates rather than limits the invention. It will be evident that there are other alternatives which do not depart from the scope of the appended claims.
There are numerous ways of implementing the described functions by means of software. In this respect, FIGS. 1 to 12 are very diagrammatic and each Figure represents only an embodiment. Although a Figure shows different functions in the form of separate blocks, it does not exclude that a single item of software performs several functions. It neither excludes that a function can be performed by a software assembly.
It is possible to implement these functions by means of a digital decoding circuit incorporated in a portable multimedia apparatus, such as a personal digital assistant or a mobile telephone, which circuit is conveniently programmed. A set of instructions in a programming memory may cause the circuit to perform different operations described hereinbefore with reference to FIGS. 1 to 12. The set of instructions may also be loaded into the programming memory by reading a data carrier such as, for example, a CD-ROM. Reading may also be effected via a communication network such as the Internet. In this case, a service provider will put the set of instructions at the disposal of those interested.
Reference signs between parentheses in a claim should not be construed as limiting the claim. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in the claims. Use of the indefinite article “a” or “an” preceding an element or a step does not exclude the presence of a plurality of such elements or steps.
Number | Date | Country | Kind |
---|---|---|---|
0205435 | Apr 2002 | FR | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IB03/01584 | 4/22/2003 | WO |