The present invention relates generally to image enhancement, and specifically to systems and methods for enhancement of images using multi-resolution decomposition and reconstruction.
Multi-resolution processing is a well-known technique for image enhancement, particularly for enhancing the contrast of fine features in radiological images. The technique, which is related to wavelet transforms, is also known as multiscale processing. For example, U.S. Pat. No. 5,467,404, whose disclosure is incorporated herein by reference, describes a method for enhancing the contrast of a digital image by the steps of:
Koren and Laine provide a useful review of the theory of multi-resolution processing in “A Discrete Dyadic Wavelet Transform for Multidimensional Feature Analysis,” published in Time-Frequency and Wavelet Transforms in Biomedical Engineering (M. Akay, ed., IEEE Press, 1997), which is incorporated herein by reference. To summarize briefly, an input signal, such as a two-dimensional input image s(x,y), is repeatedly filtered using a low-pass filter H to generate a hierarchy of filtered images of successively decreasing resolution (i.e., increasing scale). In the present patent application and in the claims, these filtered images are referred to as “scale images.” According to this nomenclature, scale 0 is simply the input image itself. The wavelet transform of the input image is then computed by filtering each of the scale images (except the residual image at the lowest resolution level) using a decomposition filter (or forward transformation filter) G. The wavelet transform may be represented interchangeably as a hierarchy of transformed scale images in the spatial domain or as a hierarchy of transform coefficients in the frequency domain. The term “transform components,” as used in the context of the present patent application and in the claims, comprehends both of these representations.
After the decomposition filtering step, each of the scale images is processed to enhance certain features. Typically, in systems known in the art, non-linear filtering techniques are applied to enhance edges in each of the scale images. To reverse the wavelet transform, the scale images are filtered using reconstruction filters (or reverse transformation filters) K and L. The K and L filters operate on the transform components in orthogonal directions. The residual image and the reverse-transformed scale images are successively filtered, using a filter with response H*, and added together to reconstruct the enhanced image.
The form of the filters H, G, K and L is dictated by wavelet theory. Koren and Laine provide explicit functional forms and numerical values of the filter kernels (i.e., impulse responses of the filters) for a number of possible choices of filtering functions that meet the theoretical criteria. According to the theory, the frequency responses of the filters must satisfy the relations:
The same filter kernel values for H, G, K and L are applied at each of the different scales, although the filter response is upsampled according to the scale. In other words, for each scale m, the filter kernels are expanded, relative to the scale 0 kernels given by Koren and Laine, by adding 2m−1 zeros between successive coefficients in the scale-0 kernels. When filters obeying these criteria are used, then in the absence of other processing of the transform components (as is used to enhance features in the image), the image reconstructed by K, L and H* will be identical to the original input image. In this sense, when wavelet transforms are used in the conventional manner known in the art, the effect of the reverse transformation filters K and L is inverse to that of the forward transformation filter G.
Embodiments of the present invention provide improved methods and systems for multi-resolution, wavelet-based image enhancement. In these embodiments, a multi-resolution image processor decomposes an input image to generate transform components, and then applies different reconstruction filters to the transform components at different image scales in order to reconstruct an enhanced image. In other words, the values of the coefficients in the K and L filters that are used in reconstruction may differ among the scales, and thus K and L are not necessarily inverse to G.
The K and L filter responses are typically chosen responsively to the characteristics of the input image and to the degree of detail enhancement that is desired. For example, to provide greater enhancement of fine detail, the kernel values of the K filter used for one or more of the high-resolution (low-scale) images in the wavelet transform may be increased relative to the higher scales. Additionally or alternatively, the kernel values may be chosen to reduce noise in the output image, or to degrade certain types of features in the image relative to others. Because the linear K and/or L filter is used not only for reconstruction, but also enhancement (which may include noise reduction and/or feature degradation), image processors operating in accordance with some embodiments of the present invention may omit the additional non-linear filtering step that is typically used to enhance the image in multi-resolution processing methods known in the art. Elimination of the need for computation-intensive non-linear filtering in this manner is useful in accelerating the image processing computation. Alternatively, in other embodiments, the enhanced K and/or L filter may be used in combination with non-linear filtering in order to improve the quality of the output image.
There is therefore provided, in accordance with an embodiment of the present invention, a method for image enhancement, including:
performing a multi-resolution decomposition of an input image, thereby generating multi-resolution transform components associated with different image scales, including at least first and second image scales; and
performing a multi-resolution reconstruction to generate an enhanced image by applying filter coefficients to the multi-resolution transform components, such that different, first and second filter coefficients are respectively applied to the multi-resolution transform components that are associated with the first and second image scales.
In some embodiments, the method includes applying a non-linear transformation to the multi-resolution transform components before performing the multi-resolution reconstruction. In other embodiments, the method does not include applying a non-linear transformation to the multi-resolution transform components before performing the multi-resolution reconstruction.
In disclosed embodiments, performing the multi-resolution decomposition includes applying a wavelet transform to the input image.
In some embodiments, the first image scale has a higher resolution than the second image scale, and a selected one or more of the first filter coefficients are set to values greater than a corresponding one or more of the second filter coefficients.
In another embodiment, applying the filter coefficients includes performing successive one-dimensional convolutions in X- and Y-directions, using different, respective X and Y filter kernels.
In a disclosed embodiment, the input image is a radiological image.
There is also provided, in accordance with an embodiment of the present invention, a method for image enhancement, including:
performing a multi-resolution decomposition of an input image using a forward transformation filter, thereby generating multi-resolution transform components associated with at least one image scale; and
performing a multi-resolution reconstruction to generate an enhanced image by applying a reverse transformation filter to the multi-resolution transform components, such that the reverse transformation filter is not an inverse of the forward transformation filter.
Typically, applying the reverse transformation filter includes reconstructing a succession of scale images, and performing the multi-resolution reconstruction includes summing the scale images to generate the enhanced image. In a disclosed embodiment, performing the multi-resolution reconstruction includes clipping pixel values in one or more of the scale images. Clipping the pixel values may include applying a baseline reconstruction to at least a portion of the multi-resolution transform components using a baseline reverse transformation filter that is the inverse of the forward transformation filter, and determining clipping limits based on the baseline reconstruction.
There is additionally provided, in accordance with an embodiment of the present invention, apparatus for image enhancement, including an image processor, which is operative to perform a multi-resolution decomposition of an input image, thereby generating multi-resolution transform components associated with different image scales, including at least first and second image scales, and which is further operative to perform a multi-resolution reconstruction to generate an enhanced image by applying filter coefficients to the multi-resolution transform components, such that different, first and second filter coefficients are respectively applied to the multi-resolution transform components that are associated with the first and second image scales.
There is further provided, in accordance with an embodiment of the present invention, apparatus for image enhancement, including an image processor, which is operative to perform a multi-resolution decomposition of an input image using a forward transformation filter, thereby generating multi-resolution transform components associated with at least one image scale, and which is further operative to perform a multi-resolution reconstruction to generate an enhanced image by applying a reverse transformation filter to the multi-resolution transform components, such that the reverse transformation filter is not an inverse of the forward transformation filter.
There is moreover provided, in accordance with an embodiment of the present invention, a computer software product, including a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to perform a multi-resolution decomposition of an input image, thereby generating multi-resolution transform components associated with different image scales, including at least first and second image scales, and further cause the computer to perform a multi-resolution reconstruction to generate an enhanced image by applying filter coefficients to the multi-resolution transform components, such that different, first and second filter coefficients are respectively applied to the multi-resolution transform components that are associated with the first and second image scales.
There is furthermore provided, in accordance with an embodiment of the present invention, a computer software product, including a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to perform a multi-resolution decomposition of an input image using a forward transformation filter, thereby generating multi-resolution transform components associated with different image scales, and further cause the computer to perform a multi-resolution reconstruction to generate an enhanced image by applying a reverse transformation filter to the multi-resolution transform components, such that the reverse transformation filter is not an inverse of the forward transformation filter.
The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
Typically, image processor 24 comprises a general-purpose computer, which is programmed in software to carry out the functions described herein. The software may be downloaded to processor 24 in electronic form, over a network, for example, or it may alternatively be supplied on tangible media, such as CD-ROM or DVD. Further alternatively, some or all of the functions of processor 24 may be implemented in hard-wired logic or using pre-programmed or field-programmable logic components.
Although image capture device 22 is shown here to comprise an X-ray camera, the principles of the present invention are by no means limited to this sort of imaging modality. The techniques described hereinbelow are applicable to other radiological modalities as well, such as ultrasound, computed tomography (CT), magnetic resonance imaging (MRI) and substantially any other modality known in the art. These techniques may also be extended, mutatis mutandis, to three-dimensional image enhancement, using multi-dimensional filtering techniques described in the above-mentioned article by Koren and Laine. Furthermore, although the embodiments described herein are directed particularly to enhancing radiological images, in other embodiments of the present invention, similar multi-resolution techniques may be applied to electronic images captured by cameras and imaging devices of other sorts. These techniques may be used both on-line, as shown in
Input image 42 is repeatedly low-pass filtered, at scaling steps 44, to generate the hierarchy of scale images. The scaling step is applied N times to generate N+1 scale images (including the original, scale-0 image). Each of the scale images, from scale 0 to N−1 is decomposed, at a decomposition step 46, so as to generate components of the wavelet transform of the input image. A scale N residual image 54 is not decomposed. Steps 44 and 46 typically use the H and G filters, respectively, as defined by Koren and Laine. Alternatively, other filter realizations, as are known in the art, may be used at these steps. The H and G filters are separable into X- and Y-components, and may thus be implemented by separate, one-dimensional X- and Y-convolutions with the appropriate one-dimensional kernels. Thus, step 46 generates, for each scale m, one-dimensional transform components Smx and Smy. In an exemplary embodiment, H and G use the following filter kernels:
Alternatively, other filter kernels may be used, as defined by Koren and Laine or as are otherwise known in the art.
Optionally, linear or non-linear image enhancement operations may be applied to the transform components, at a non-linear enhancement step 48. Exemplary non-linear filtering methods that may be applied in this step are described by Koren and Laine and in the patents cited in the Background of the Invention. The scale images (whether enhanced or not) are then reverse-transformed, at a reconstruction step 50, which is described below with reference to
In systems known in the art, steps 60 and 64 use the K filter kernel, as defined by Koren and Laine, and steps 62 and 66 use the L filter kernel. For the H and G kernels listed above, the corresponding, standard K and L kernels are as follows:
These kernels are used for all scales (with appropriate upsampling, as described in the Background of the Invention).
On the other hand, in embodiments of the present invention, different kernels are used for different scales at step 50. For example, in order to provide enhancement of fine details in image 42, the following enhanced kernels may be used for scales 0-5 in place of the standard K kernel at steps 60 and 64:
The standard L kernel listed above in Table I is used at steps 62 and 66. The above enhanced K kernels have been found empirically to give good results, particularly in enhancing X-ray images of fine bone structures in images of body extremities. (The image enhancement procedure was applied to images captured by the Quix™ FP-100 Digital Radiography detector, produced by Edge Medical Ltd., Raanana, Israel.)
Alternatively, other kernel values may be used at the low scales and/or other scales, depending on the enhancement required. For example, the inventors have found the following K kernels to be useful at steps 60 and 64 for enhancing chest X-ray images (captured using the above-mentioned Quix detector):
Although the above examples use the same K and L kernels for X-reconstruction (steps 60 and 62) and Y-reconstruction (steps 64 and 66), different kernels may alternatively be used for X- and Y-reconstruction in order to apply different enhancements to X-oriented and Y-oriented image features. Whatever specific kernel is chosen, the enhancement is achieved at no added computational cost because the image enhancement operation is integrated with filtering steps 60 and 64, which are performed in any case as part of the reverse transformation.
Because of the modification made to the filter kernels used in step 50, the reconstruction operation is no longer exactly inverse to the decomposition operation. There may, therefore, be an overflow in some of the pixel values of the reconstructed scale images following step 50. To eliminate the overflow and maintain the proper proportion between different scale images, the pixel values in at least some of the reconstructed scale images may be reduced, at a clipping step 52. This step may involve simply cutting off pixel values that exceed some saturation threshold. Alternatively, a gradual scaling function may be applied, such as a gamma function, as is known in the art of video systems. Note that this step involves a non-linear operation, in contrast to the linear image enhancement operations described above.
Further alternatively, the order of clipping step 52 and some or all of linear filtering steps 60, 62, 64 and 66 may be reversed. For example, the Kmx and Kmy filters may be broken into equivalent pre- and post-clip filter components, which are used in two successive filtering operations, one before clipping is performed and the other afterwards. Other arrangements will be apparent to those skilled in the art of digital filtering, and are considered to be within the scope of the present invention.
Referring back now to
Although the embodiment described above makes use of specific wavelet transformation filters defined by Koren and Laine, the principles of the present invention may similarly be applied in multi-resolution image enhancement systems using linear filters of other types. Furthermore, although the above embodiments, relate specifically to two-dimensional images, the principles of the present invention may also be applied, mutatis mutandis, in multi-resolution processing of one-dimensional signals, as well as of three-dimensional images. It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.