Patent Grant
7343040
The present invention relates to a method and a system for modifying a digital image taking its noise into account.
The invention relates to a method for calculating a transformed image from a digital image and formatted information related to defects of an appliance chain. The appliance chain contains image-capture appliances and/or image-restitution appliances. The appliance chain contains at least one appliance. The method includes the stage of automatically determining characteristic data from formatted information and/or from the digital image. The characteristic data are referred to hereinafter as the characteristic noise data.
It results from the combination of technical features that the transformed image does not exhibit any visible or annoying defect, especially defects related to noise, as regards its subsequent use.
Preferably, according to the invention, the method additionally includes the following stages for determining the characteristic noise data:
Preferably, according to the invention, the method additionally includes the following stages for deducing the characteristic noise data:
It results from the combination of technical features that the local brightness variations related to the noise are then obtained.
Preferably, according to the invention, the method additionally includes, for selection of analysis zones over the digital image, the stage of classifying the analysis zones according to their mean brightness, in such a way as to obtain classes. The method additionally includes:
It results from the combination of technical features that characteristic noise data as a function of brightness are then obtained.
Preferably, according to the invention, the formatted information contains the characteristic noise data.
Preferably, according to the invention, the method additionally includes the stage of employing a transformation algorithm for constructing an intermediate digital image. The algorithm has the advantage of making desired modifications to the digital image but suffers from the disadvantage of increasing the noise of the intermediate digital image.
Preferably, according to the invention, to calculate a transformed image from the intermediate digital image obtained from the digital image, the method additionally includes the stage of employing a function whose purpose is to modify the brightness of the digital image and which has, as arguments, at least:
It results from the combination of technical features that there is then obtained a transformed image exhibiting the desired characteristics and a controlled noise level.
Preferably, according to the invention, the intermediate digital image is composed of the digital image.
Preferably, according to the invention, the method is more particularly designed to calculate a transformed image corrected for all or part of the blurring. The method additionally includes the following stages:
It results from the combination of technical features that a deblurred transformed image is then obtained.
Preferably, according to the invention, the formatted information makes it possible to determine, for each image zone to be corrected, an image representation and a reference representation in a base related to the image zone to be corrected.
The method is such that, to construct an enhancement profile from formatted information and noise, it additionally includes the following stages:
The set of values of parameters of the parameterized operator constitutes the enhancement profile PR.
Preferably, according to the invention, the method additionally includes the following stages for correction of each image zone to be corrected as a function of the enhancement profile:
Preferably, according to the invention, the method additionally includes the stage of calculating, from the transformed image, an image having a controlled noise level, by employing a function whose purpose is to modify the brightness of the digital image and which has, as arguments, at least:
It results from the combination of technical features that there is then obtained a deblurred image having a controlled noise level.
The formatted information may depend on values of variable characteristic depending on the digital image, especially the size of the digital image. Preferably in this case according to the invention, the method additionally includes the stage of determining the value or values of the variable characteristics for the digital image.
Thus employment of the method for formatted information including characteristic noise data that depend on variable characteristics depending on the digital image reduces to employment of the method for characteristic noise data that do not depend on any characteristic variable.
Preferably, according to the invention, the method is more particularly designed to calculate a transformed image from a digital image and from formatted information related to the defects of an appliance chain containing at least one image-restitution appliance. The restitution appliance has a dynamic range. The transformed image has a dynamic range. The method additionally includes the stage of adapting the dynamic range of the transformed image to the dynamic range of the said restitution appliance. It results from the combination of technical features that the restitution of the transformed image by the restitution appliance exhibits reinforced high frequencies. It also results from the combination of technical features that the restitution appliance can restitute images of characters with less blurring.
The invention is applicable to the case of a digital image composed of color planes. The application comprises applying the method according to the invention to each color plane. In this way a transformed image is obtained from the digital image. It results from the combination of technical features that the transformed image exhibits the desired characteristics and a controlled noise level.
The invention relates to a system for calculating a transformed image from a digital image and formatted information related to defects of an appliance chain. The appliance chain contains image-capture appliances and/or image-restitution appliances. The appliance chain contains at least one appliance. The system includes data-processing means for automatically determining characteristic data from formatted information and/or from the digital image. The characteristic data are referred to hereinafter as the characteristic noise data.
The transformed image does not exhibit any visible or annoying defect, especially defects related to noise, as regards its subsequent use.
Preferably, according to the invention, the data-processing means for determining the characteristic noise data include:
Preferably, according to the invention, the deducing means additionally include:
Preferably, according to the invention, the system additionally includes, for selection of analysis zones over the digital image, classification means for classifying the analysis zones according to their mean brightness, in such a way as to obtain classes. The system additionally includes data-processing means for:
Preferably, according to the invention, the formatted information contains the characteristic noise data.
Preferably, according to the invention, the system additionally includes data-processing means employing a transformation algorithm for constructing an intermediate digital image. The algorithm has the advantage of making desired modifications to the digital image but suffers from the disadvantage of increasing the noise of the intermediate digital image.
Preferably, according to the invention, to calculate a transformed image from the intermediate digital image obtained from the digital image, the system includes calculating means employing a function whose purpose is to modify the brightness of the digital image and which has, as arguments, at least:
Preferably, according to the invention, the intermediate digital image is composed of the digital image.
Preferably, according to the invention, the system is more particularly designed to calculate a transformed image corrected for all or part of the blurring. The system additionally includes:
The system additionally includes data-processing means for:
Preferably, according to the invention, the formatted information makes it possible to determine, for each image zone to be corrected, an image representation and a reference representation in a base related to the image zone to be corrected. The system is such that the calculating means for constructing an enhancement profile from formatted information and noise additionally include means for determining:
Preferably, according to the invention, the data-processing means for correction of each image zone to be corrected as a function of the enhancement profile include calculating means for:
Preferably, according to the invention, the system additionally includes calculating means for calculating, from the transformed image, an image having a controlled noise level, by employing a function whose purpose is to modify the brightness of the digital image and which has, as arguments, at least:
Preferably, according to the invention, the formatted information depends on values of variable characteristics depending on the digital image, especially the size of the digital image. The system additionally includes calculating means for determining the value or values of the variable characteristics for the digital image.
Preferably, according to the invention, the system is more particularly designed to calculate a transformed image from a digital image and from formatted information related to the defects of an appliance chain containing at least one image-restitution appliance. The restitution appliance has a dynamic range. The transformed image has a dynamic range. The system additionally includes data-processing means for adapting the dynamic range of the transformed image to the dynamic range of the restitution appliance.
Other characteristics and advantages of the invention will become apparent upon reading of the description of alternative embodiments of the invention, provided by way of indicative and non-limitative examples, and of:
a, which illustrates a local brightness variation over an analysis zone,
b, which illustrates a histogram of occurrences of local brightness variations,
c, which illustrates a part of the histogram situated before the first local maximum of the histogram,
a, which illustrates the correction of a transformed image zone as a function of an enhancement profile,
b, which illustrates an example of creation of a deblurred image with controlled noise level,
a and 8b, which illustrate the construction of an enhancement profile from the noise,
a, 9b, 9c and 9d, which present the adaptation of the dynamic range of the transformed image to the dynamic range of a restitution appliance,
Referring in particular to
A more complex appliance P25, such as a scanner/fax/printer, a photo-printing. Minilab, or a videoconferencing appliance can be regarded as an appliance P25 or as a plurality of appliances P25.
Referring in particular to
The following examples constitute appliance chains P3:
Referring in particular to
Referring in particular to
Referring in particular to
To produce the formatted information IF, it is possible, for example, to use the method described in the International Patent Application filed on the same day as the present application in the name of Vision IQ and entitled “Method and system for producing formatted information related to defects of at least one appliance of a chain, in particular blurring”. That application describes a method for producing formatted information related to the appliances of an appliance chain. The appliance chain is composed in particular of at least one image-capture appliance and/or at least one image-restitution appliance. The method includes the stage of producing formatted information related to the defects of at least one appliance of the chain. The appliance preferably makes it possible to capture or restitute an image (I). The appliance contains at least one fixed characteristic and/or one variable characteristic depending on the image (I). The fixed and/or variable characteristics can be associated with one or more values of characteristics, especially the focal length and/or the focusing and their values of associated characteristics. The method includes the stage of producing, from a measured field D(H) measured formatted information related to the defects of the appliance. The formatted information may include the measured formatted information.
To produce the formatted information IF, it is possible, for example, to use the method described in the International Patent Application filed on the same day as the present application in the name of Vision IQ and entitled “Method and system for providing formatted information in a standard format to image-processing means”. That application describes a method for providing formatted information IF in a standard format to image-processing means, especially software and/or components. The formatted information IF is related to the defects of an appliance chain P3. The appliance chain P3 includes in particular at least one image-capture appliance and/or one image-restitution appliance. The image-processing means use the formatted information IF to modify the quality of at least one image derived from or addressed to the appliance chain P3. The formatted information IF includes data characterizing the defects P5 of the image-capture appliance, especially the distortion characteristics, and/or data characterizing the defects of the image-restitution appliance, especially the distortion characteristics.
The method includes the stage of filling in at least one field of the said standard format with the formatted information IF. The field is designated by a field name. The field contains at least one field value.
To search for the formatted information IF, it is possible, for example, to use the method described in the International Patent Application filed on the same day as the present application in the name of Vision IQ and entitled “Method and system for modifying the quality of at least one image derived from or addressed to an appliance chain”. That application describes a method for modifying the quality of at least one image derived from or addressed to a specified appliance chain. The specified appliance chain is composed of at least one image-capture appliance and/or at least one image-restitution appliance. The image-capture appliances and/or the image-restitution appliances being progressively introduced on the market by separate economic players belong to an indeterminate set of appliances. The appliances of the set of appliances exhibit defects that can be characterized by formatted information. For the image in question, the method includes the following stages:
To produce the formatted information IF, it is possible, for example, to use the method described in the International Patent Application filed on the same day as the present application in the name of Vision IQ and entitled “Method and system for reducing update frequency of image processing means”. That application describes a method for reducing the update frequency of image-processing means, in particular software and/or a component. The image-processing means make it possible to modify the quality of the digital images derived from or addressed to an appliance chain. The appliance chain is composed in particular of at least one image-capture appliance and/or at least one image-restitution appliance. The image-processing means employ formatted information related to the defects of at least one appliance of the appliance chain. The formatted information depends on at least one variable. The formatted information makes it possible to establish a correspondence between one part of the variables and of the identifiers. By means of the identifiers it is possible to determine the value of the variable corresponding to the identifier by taking the identifier and the image into account. It results from the combination of technical features that it is possible to determine the value of a variable, especially in the case in which the physical significance and/or the content of the variable are known only after distribution of image-processing means. It also results from the combination of technical features that the time between two updates of the correction software can be spaced apart. It also results from the combination of technical features that the various economic players that produce appliances and/or image-processing means can update their products independently of other economic players, even if the latter radically change the characteristics of their product or are unable to force their client to update their products. It also results from the combination of technical features that a new functionality can be deployed progressively by starting with a limited number of economic players and pioneer users.
A description will now be given of the concept of variable characteristic CC. According to the invention, a variable characteristic CC is defined as a measurable factor, which is variable from one digital image INUM to another that has been captured, modified or restituted by the same appliance P25, and which has an influence on defect P5 of the image that has been captured, modified or restituted by appliance P25, especially:
A measurable factor which is variable from one appliance P25 to another but which is fixed from one digital image INUM to another that has been captured, modified or restituted by the same appliance P25, is not generally considered to be a variable characteristic CC; an example is the focal length for an appliance P25 with fixed focal length.
The formatted information IF may depend on at least one variable characteristic CC.
By variable characteristic CC there can be understood in particular:
A description will now be given of the concept of variable characteristic value VCC. A variable characteristic value VCC is defined as the value of variable characteristic CC at the moment of capture, modification or restitution of a specified image.
A digital image INUM contains a set of image elements defined as pixels PX-num.1 to PX-num.n distributed regularly over the surface of image INUM. In
Intermediate image I-Int contains a set of pixels similar to that of image INUM, but not necessarily so, defined as intermediate pixels Px-int.1 to Px-int.n. Each intermediate pixel is characterized by an intermediate position Px-int and an intermediate value vx-int.
Transformed image I-Transf also contains a set of pixels defined as transformed pixels PX-tr.1 to PX-tr.n. Each transformed pixel is characterized by a transformed position Px-tr and a transformed value vx-tr.
A transformed image is a corrected or modified image obtained by application of a transformation to an image INUM.
This transformation, which may be a photometric transformation, is performed by incorporating, in the calculation,
It will be noted that the formatted information may be related to a limited number of transformed pixels and/or may incorporate values of variable characteristics that depend on the image (such as the focal length, the focusing, the aperture, etc.). In this case there may exist a supplementary stage which, for example, is performed by interpolation in such a way that it is reduced to simple formatted information such as that of an appliance that is not provided with any variable characteristics, so that the case of appliances with variable focal length in particular reduces to the case of an appliance with fixed focal length.
It will be noted that the formatted information can be related to a limited number of transformed pixels and/or of values of variable characteristics depending on the image. In this case it is possible to include a supplementary stage, which is performed, for example, by interpolation. In the example of a function x′,y′=f(x, y, t), where t is a variable characteristic (such as focal length), the formatted information can be composed of a limited number of values (xi, yi, ti, f(xi, yi, ti)). It is then necessary to calculate an approximation for the other values of x, y, t other than the measurement points. This approximation can be applied by resorting to simple interpolation techniques or by using parameterizable models (polynomials, splines, Bezier functions) having greater or lesser order depending on the desired final precision. With an analogous formalism, t could be a vector and could include a plurality of variable characteristics (focal length, focusing, zoom, etc.) simultaneously.
In the case of noise and/or of blurring, the formatted information could be composed if necessary of vectors with which the noise and/or the blurring related to an appliance and/or to an appliance chain can be characterized, for the set of combinations of variable parameters of the device, especially by resorting to characteristic profiles of the defect in special representation bases, especially frequency representations such as Fourier transforms, wavelet transforms, etc. In fact, the person skilled in the art knows that frequency representations are compact domains that are appropriate for representation of physical phenomena related to noise and/or to blurring.
It is additionally possible to combine the formatted information IF related to a plurality of appliances P25 of an appliance chain P3 to obtain formatted information related to a virtual appliance exhibiting the defects of the said plurality of appliances P25; in such a way that it is possible to calculate, in one stage, transformed image I-Transf from image INUM for all of the said plurality of appliances P25; in such a way that the said calculation is faster than if the method according to the invention were to be applied successively to each appliance P25; in the example of a frequency representation such as the Fourier transform, the said combination can be achieved cumulatively from the characteristic profiles of the defect of each appliance, for example by multiplication.
It will be possible for the formatted information to include not only data that were studied in a preliminary phase and that are related to the appliances used, but also all information styled in the Exif or other format that could provide particulars on the adjustments of the appliance at the moment of filming (focal length, focusing, aperture, speed, flash, etc.).
Let us assume that digital image INUM represents, for example, capture of the monochromatic image of a white square on a black background.
We note that the application of algorithm CAPP may, in the case of noise and/or blurring, reduce original image INUM to a perfect or quasi-perfect image. The same algorithm may also reduce image INUM to another image, which may be deformed, albeit differently, in such a way as to produce an image closely resembling a known type of image noise and/or blurring (retro noise effect, etc.). The same method also makes it possible to reduce image INUM to an image that is not perfect (in the sense of a white square on a black background, as in
For certain types of appliance APP, especially for image capture, it is possible to deduce characteristic noise data DcB from formatted information. For example, this is the case in particular for appliances with which it is possible to determine particulars of variable characteristics that influence noise, such as gain, ISO, etc. The dependence between noise and these characteristics will be entered into the formatted information, especially by means of polynomial functions.
To the extent that characteristic noise data cannot be deduced directly or indirectly from formatted information, it will be necessary to deduce these characteristic data. We will therefore describe, within the meaning of the present invention, a practical example with which there can be produced characteristic noise data DcB that are related to an image INUM.
Image INUM is subdivided into a series of analysis zones (ZAN), which are not necessarily joined and which may intersect as the case may be.
An example of measurement of local brightness variation VLL can be achieved by calculating, over an analysis zone ZAN, the maximum brightness deviation among the set of points. In
It is possible to analyze the set of measurements of local brightness variation VLL statistically by creating a histogram of frequencies of occurrence of the variations. In such a histogram, an example of which is illustrated in
The profile of this histogram for a natural image, such as a landscape image containing a random distribution of patterns of different brightness but having homogeneous brightness over small analysis zones, contains a characteristic zone situated before the first local maximum (
The characteristic noise data of image INUM may be composed of the set of values of the histogram up to the first mode. Another way of extracting more synthetic information from the noise characteristic comprises, as illustrated in
In
Let us consider a digital image INUM within the meaning of the present invention, and let us also consider a transformation applicable to an INUM in such a way as to construct an intermediate image which, in certain respects, has the advantage of making the desired modifications but, on the other hand, suffers from the disadvantage of increasing the image noise in certain zones. As we will see hereinafter, it will be possible for this transformation to be a transformation that reduces blurring, a transformation that increases contrast, a transformation with which image mosaics can be created, or any other transformation capable of modifying the noise characteristics between image INUM and I-Int. The method illustrated in
The analysis of the mean brightness and of the local brightness variation VLL in zone ZAN-j makes it possible to determine class Cj to which the noise belongs and to extract the data DcB of noise BM-j. According to one option, it is possible to calculate a normalized ratio Rj between BM-j and VLL. As illustrated in
vx-tr=(Rj)vx-num+(1−Rj)vx-int
where vx-num and vx-int represent the brightnesses of Px-num-j and Px-int-j respectively. In the opposite case (the local brightness variation VLL is large compared with BM-j, thus corresponding to the signal), the ratio Rj tends to 0 and the brightness vx-tr of transformed pixel Px-tr-j is taken for the most part in intermediate image I-Int.
More generally, the brightness value of a transformed pixel can be expressed as a function of the brightnesses of pixel vx-num and its neighbors, of the brightnesses of pixel vx-int and its neighbors and finally of the characteristic noise data.
It will therefore be possible, for example, to deduce the transformed image from the intermediate image by applying a more or less intensive filtering operation in the latter on the basis of noise measured in INUM.
This method has the advantage that it takes, in the intermediate image, only information that is pertinent to the exclusion of points for which the noise analyzed in original image INUM is too large within the meaning of a global statistical study of noise characterized by the data DcB.
It is understood that it is possible, during the clipping operation, to apply any relation, especially linear or nonlinear transformations, for passage between the images INUM and I-Int.
In
We will now describe a practical example of a method designed more particularly to calculate a transformed image corrected for all or part of the blurring. The description of this method is based on the practical example of the system shown in
For a configuration of given arguments (focal length, focusing, zoom, aperture, etc., DcB, zone ZIC), it is possible, by means of the parameterizable model of formatted information, to access characteristic blurring profiles related to an image representation RI and a reference representation RR. These profiles are expressed in a particular base, especially a frequency base B, by using, for example, a Fourier transform, a wavelet transform, etc.
Base B will be implicit or else will be established within the formatted information. Within the meaning of the present invention, the person skilled in the art sees that it is possible to represent a digital image (such as INUM) in a vector space of dimension equal to the number of pixels. By base B there is understood, non-exclusively, a base, in the mathematical sense of the term, of this vector space and/or a vector subspace thereof.
Hereinafter, frequency is defined as an identifier related to each element of the base. The person skilled in the art understands Fourier transformations and/or wavelet transforms as changes of the base of the image space. In the case of an appliance APP for which the blurring defects significantly affect only a subspace of the vector space of the images, it will be necessary to correct only those components of image INUM that belong to this subspace. Thus base B will be chosen preferably as a base for representation of this subspace.
Another way of employing the method within the meaning of the invention is to choose, for representation of the image, a base that is optimal within the meaning, for example, of that of calculation time. It will be possible to choose this base with small dimension, each element of the base having a support of a few pixels spatially localized in image INUM (for example, splines or sets of Laplace operators of local variations, Laplacian of Laplacian, or derivatives of higher order, etc.).
The measurement of the local brightness variation VLL over zone ZIC makes it possible, by virtue of the characteristic noise data DcB of INUM, to calculate a coefficient Rj (device dcb2). This coefficient will be coupled with the representations RI and RR (device pr) to generate a frequency-based enhancement profile PR related to zone ZIC. This profile indicates the gain to be applied at each frequency related to the brightness information contained in zone ZIC to be corrected, in order to suppress all or part of the blurring.
a shows that it is then sufficient to express zone ZIC in a base B, especially an adequate frequency base B(ZIC), to apply the enhancement function for all or part of the frequencies. B(ZIC*)=B(ZIC)*PR, and then, by an inverse transform, to find the transformed image zone. The set of transformed image zones is then combined in such a way as to obtain the deblurred transformed image (I-Transf ID). By means of this combination it is possible, for example, to apply solutions in the case of overlap of ZIC, especially to limit the edge effects.
Creation of the image (I-Transf ID) as described in the foregoing has the advantage of applying the necessary modifications to image INUM from the viewpoint of blurring, but suffers from the disadvantage of increasing the noise in certain zones (especially relatively uniform zones).
A second employment of a method of the present invention is based on the practical example of the system shown in
We see that the ratio between these two profiles can indicate the gain for each frequency to be applied to RI to find RR. On the other hand; it turns out that direct application of the calculated gain between RI and RR can generate undesirable behaviors, especially with high frequencies, when zone ZIC to be corrected contains a high noise level. These phenomena are known by the person skilled in the art as the effect of brightness oscillations defined as “ringing”. According to the invention, the method will estimate, between RR and RI, a profile whose position is parameterized as a function of the noise in analyzed zone ZIC.
a and 8b show two examples of profiles PR that can be generated according to the invention. The deviation between profiles RI and RR shows the frequency loss introduced by the blurring inherent to the device.
a treats the case of high noise level in zone ZIC; it will be advisable to choose, between RI and RR, a profile RH such that its effect is less at the high frequencies (the end of RH will coincide with RI), which in this case carry the information related to the noise in the image.
In contrast,
In any case, RH will not be permitted to exceed RR, which is the ideal profile of the device but does not correspond to an image that can be constructed by a real device. In view of the foregoing description, it is possible to choose multiple functions that parameterize a curve of profile RH between RI and RR. In
Construction of the frequency-based enhancement profile PR is performed immediately by calculation of the ratio RH/RI for all frequencies.
The method of the invention is applicable to the processing of color images. From the viewpoint of image-processing software, a color image is considered to contain as many images (or color planes) as there are basic colors in the image. Thus an image IMrgb is considered to be composed of the three color planes Im-red, Im-green, Im-blue. Analogously, an image IMcmyk may be considered to be composed of four color planes: Im-cyan, Im-magenta, Im-yellow, Im-black. In the method described in the foregoing, each color plane will be processed independently in such a way as to obtain n transformed images, which will recompose the different color planes of the transformed final image.
The method of the invention is applicable to calculation of a transformed digital image I-Transf designed to be displayed via a restitution means of known dynamic range (
Cost reduction is defined as a method and system for lowering the cost of an appliance P25 or of an appliance chain P3, especially the cost of the optical system of an appliance or of an appliance chain, the method consisting in:
The method and system according to the invention can be used to lower the cost of an appliance or of an appliance chain: it is possible to design a digital optical system, to produce formatted information IF related to the defects P5 of the appliance or of the appliance chain, to use this formatted information to enable image-processing means, whether they are integrated or not, to modify the quality of images derived from or addressed to the appliance or to the appliance chain, in such a way that the combination of the appliance or the appliance chain with the image-processing means is capable of capturing, modifying or restituting images of the desired quality at reduced cost.
| Number | Date | Country | Kind |
|---|---|---|---|
| 01 09291 | Jul 2001 | FR | national |
| 01 09292 | Jul 2001 | FR | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/FR02/01908 | 6/5/2002 | WO | 00 | 7/15/2004 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO03/007243 | 1/23/2003 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5499057 | Kondo et al. | Mar 1996 | A |
| 5694484 | Cottrell et al. | Dec 1997 | A |
| 6069982 | Reuman | May 2000 | A |
| 6115104 | Nakatsuka | Sep 2000 | A |
| 6462835 | Loushin et al. | Oct 2002 | B1 |
| Number | Date | Country |
|---|---|---|
| 0 640 908 | Mar 1995 | EP |
| Number | Date | Country | |
|---|---|---|---|
| 20040247196 A1 | Dec 2004 | US |
