IMAGE FUSION METHOD BASED ON FOURIER SPECTRUM EXTRACTION

Abstract

The present invention discloses an image fusion method based on Fourier spectrum extraction, which comprises: step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with the maximum frequency amplitude in the images at different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at the corresponding spatial frequency, traversing each frequency to generate a frequency domain component of the fused image, finally performing inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; and step 2, outputting the fused image obtained by the processor by an output unit of the computer.

Description

TECHNICAL FIELD

The present invention relates to an image fusion method, in particular to an image fusion method based on Fourier transform spectrum extraction, and belongs to the technical field of image processing.

BACKGROUND

As is an important field of information fusion, image fusion has been widely applied to the aspects of remote sensing, computer vision, medicine, military target detection and identification and the like.

At present, image fusion methods derived from multi-resolution are popular, a large category of which is based on Gaussian pyramid decomposition of images, and then Laplacian pyramid, gray pyramid, gradient pyramid and the like are derived; and another large category of which is an algorithm based on wavelet decomposition, the basic idea is to decompose the image into a series of sub-images at different resolutions, where each resolution comprises a blurred sub-image containing low frequency information and three sub-images with high frequency detail in a row, column, diagonal direction. The two methods have the common feature that images are fused according to a certain rule at different resolutions to obtain a fused image sequence.

SUMMARY

In order to solve the technical problem, the present invention provides an image fusion method based on Fourier spectrum extraction, which extracts a clear image area through Fourier transform, thereby realizing a method for generating a picture containing detailed information of objects at different depths in the shooting direction by fusing a plurality of images under the shooting condition of small depth of field.

In order to solve the aforementioned technical problems, the present invention adopts the following technical scheme:

the present invention provides an image fusion method based on Fourier spectrum extraction, which comprises:

step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with the maximum frequency amplitude in the images at different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at the corresponding spatial frequency, traversing each frequency to generate a frequency domain component of the fused image, finally performing inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; and

step 2, outputting the fused image obtained by the processor by an output unit of the computer.

In the image fusion method based on Fourier spectrum extraction, performing the steps by the processor specifically comprises:

(1) obtaining gray image information of each image from the plurality of to-be-fused images:

f _n(x,y), (x<K,y<L), n=1, 2, . . . , N

wherein, (x, y) is the pixel coordinate of the gray image, K and L are the boundary values of the image in X and Y directions, respectively, and N is the total number of the images;

(2) transforming the N gray images in a spatial domain obtained in the step (1) into the N gray images in a frequency domain by using two-dimensional discrete Fourier transform to obtain the frequency component of each image:

$F_n\left(f_x,f_y\right)=\sum _{x=0}^{K-1}\sum _{y=0}^{L-1}{f}_n\left(x,y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right]$

wherein, (f_x,f_y) is the coordinate of the spatial frequency, and represents the spatial frequency in X and Y directions, and |F_n(f_x,f_y)| is the frequency domain amplitude;

(3) extracting the frequency component corresponding to the image with the maximum frequency amplitude |F_n(f_x,f_y)| in the N images at the spatial frequency (f_x,f_y) as a frequency component of the fused image at the spatial frequency according to the frequency domain amplitude |F_n(f_x,f_y)| obtained in the step (2), traversing each point (i.e., each spatial frequency (f_x,f_y)) in the frequency domain, and finally generating a frequency domain component of the N fused images by adopting the method:

F_n(f_x,f_y)→F(f_x,f_y); and

(4) performing inverse transform on the fused frequency domain component obtained in the step (3) by using two-dimensional discrete inverse Fourier transform to obtain a gray image reconstructed in the spatial domain, i.e., an image of N images after fusion:

$f\left(x,y\right)=\sum _{x=0}^{K-1}\sum _{y=0}^{L-1}F\left(f_x,f_y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right]$

f(x,y) is the gray image obtained after reconstruction.

In the step (1), the number of the plurality of images N is greater than or equal to 2.

In the step (1), the plurality of fused images have the same field of view and resolution.

In the step (1), the plurality of images have different focus depths for objects at different depth positions or the same object.

Beneficial Effects: the image fusion method disclosed by the present invention appreciates that frequency domain signals represent information such as edge, texture and the like of the image in the spatial domain through the Fourier transform on the images at different focus positions, extracts the detailed information at different positions, synthesizes pictures with the detailed information of objects at different positions by using pictures with the same resolution without replacing cameras and lens, and therefore provides a quick and convenient image fusion method for the application fields of computer vision detection and the like, and the method is simple in calculation, and the fused image contains more image details.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a system structure corresponding to the image fusion method based on Fourier spectrum extraction according to the present invention;

FIG. 2 is a flowchart of the image fusion method based on Fourier spectrum extraction according to the present invention;

FIG. 3 are to-be-fused images in the same field of view with the same camera shooting different focal planes in the present invention;

FIG. 4 are images of spatial frequency domain distribution after the two-dimensional discrete Fourier transform corresponding to the three images in FIG. 3;

FIG. 5 is a frequency domain image obtained by fusing the three images of spatial frequency domain distribution in FIG. 4; and

FIG. 6 is an image in the spatial domain reconstructed by two-dimensional discrete inverse Fourier transform on the frequency domain image in FIG. 5.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present invention will be better understood from the following embodiments. However, it is easily understood by those skilled in the art that the descriptions of the embodiments are only for illustrating the present invention and should not and will not limit the present invention as detailed in the claims.

As shown in FIG. 1, the image fusion method based on Fourier spectrum extraction of the present invention comprises:

step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with the maximum frequency amplitude in the images at different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at the corresponding spatial frequency, traversing each frequency to generate a frequency domain component of the fused image, finally performing inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; and

step 2, outputting the fused image obtained by the processor by an output unit of the computer.

The input unit and the output unit are an input interface and an output interface of the processor, respectively, and the input interface and the output interface can be a network communication interface, a USB serial port communication interface, a hard disk interface and the like.

The image fusion method of the present invention extracts a clear image area based on Fourier transform, thereby generating a picture containing detailed information of objects at different depths in the shooting direction by fusing a plurality of images under the shooting condition of small depth of field. The algorithm of the present invention requires to shoot N images in different depth directions (Z direction) within the same field of view by changing focus positions of lens. Because of the limitation of the depth of field of the lens, each image can be clearly focused on the image plane (X, Y direction) only at a small depth in front and back near the focus plane. In order to display three-dimensional (X, Y, Z direction) information of a photographed object (or space) on one picture, N images are fused to generate one image. Detailed information (X, Y direction) of objects at different depth positions can be obtained from the image.

As shown in FIGS. 2-6, in the image fusion method based on Fourier spectrum extraction of the present invention, performing the steps by the processor specifically comprises:

(1) obtaining gray image information of each image from the plurality of to-be-fused images:

f _n(x,y),(x<K,y<L), n=1, 2, . . . , N

wherein, (x,y) is the pixel coordinate of the gray image, and K and L are the boundary values of the image in X and Y directions, respectively; N is the total number of the images, and N is greater than or equal to 2; the plurality of images have the same field of view and resolution; the plurality of images have different focus depths for objects at different depth positions or the same object;

(2) transforming the N gray images in a spatial domain obtained in the step (1) into the N gray images in a frequency domain by using two-dimensional discrete Fourier transform to obtain the frequency component of each image:

$F_n\left(f_x,f_y\right)=\sum _{x=0}^{K-1}\sum _{y=0}^{L-1}{f}_n\left(x,y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right]$

wherein, (f_x,f_y) is the coordinate of the spatial frequency, and represents the spatial frequency in X and Y directions, |F_n(f_x,f_y)| is the frequency domain amplitude, and the larger the amplitude is, the richer the frequency component is, and the more the detailed information of the image is;

(3) extracting the frequency component corresponding to the image with the maximum frequency amplitude |F_n(f_x,f_y)| in the N images at the spatial frequency (f_x,f_y) as a frequency component of the fused image at the spatial frequency according to the frequency domain amplitude |F_n(f_x,f_y)| obtained in the step (2), traversing each point (i.e., each spatial frequency (f_x,f_y)) in the frequency domain, and finally generating a frequency domain component of the N fused images by adopting the method:

F_n(f_x,f_y)→F(f_x,f_y)

(4) performing the spatial domain image reconstruction step, wherein F(f_x,f_y) comprises the detailed information of the images at different positions in the N images because of the directionality of f_x,f_y, in order to obtain the fused image effect from the frequency domain to the spatial domain, performing inverse transform on the fused frequency domain component obtained in the step (3) by using two-dimensional discrete inverse Fourier transform to obtain a gray image reconstructed in the spatial domain, i.e., an image of the N fused images:

$f\left(x,y\right)=\sum _{x=0}^{K-1}\sum _{y=0}^{L-1}F\left(f_x,f_y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right]$

f(x,y) is the gray image obtained after reconstruction.

In FIG. 3, the images in the pictures are clearly displayed only at focus positions, that is, the edge and detailed texture information is richer; in the fused image (FIG. 6), the detailed information of the three focus positions is well fused into one image, that is, the detailed information of the objects at different shooting depth positions can be seen from one picture, so that the image fusion effect is effectively realized.

Claims

1. An image fusion method based on a Fourier spectrum extraction, comprising: step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing a Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with a maximum frequency amplitude in the images at the different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at a corresponding spatial frequency, traversing each of a frequency to generate a frequency domain component of the fused image, finally performing an inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; andstep 2, outputting the fused image, which is obtained by the processor by an output unit of the computer.

2. The image fusion method based on the Fourier spectrum extraction according to claim 1, wherein performing the steps by the processor specifically comprises: (1) obtaining a gray image information of each of the image from the plurality of to-be-fused images: fn(x,y), (x<K,y<L), n=1, 2, . . . , N wherein, (x,y) is a pixel coordinate of a gray image, K and L are boundary values of the image in X and Y directions, respectively, and N is a total number of the images;(2) transforming N gray images in a spatial domain, which is obtained in the step (1) into the N gray images in a frequency domain by using a two-dimensional discrete Fourier transform to obtain the frequency component of each of the image:

3. The image fusion method based on the Fourier spectrum extraction according to claim 2, wherein: in the step (1), a number of a plurality of the images N is greater than or equal to 2.

4. The image fusion method based on the Fourier spectrum extraction according to claim 2, wherein: in the step (1), a plurality of fused images have a same field of view and resolution.

5. The image fusion method based on the Fourier spectrum extraction according to claim 2, wherein: in the step (1), the plurality of images have different focus depths for objects at different depth positions or a same object.