Image processing device, method, image reading device, and recording medium

Abstract

A local variance or an edge amount is calculated as an image feature amount by analyzing a reference image of the same field of view as multiple band images constituting a multispectral image, a filter parameter is calculated from the image feature amount, and smoothing the band images while preserving edges is performed by using the filter parameter. When a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges.

Description

CROSS-REFERENCE TO RELATED APPLICATION

The present application is based on PCT filing PCT/JP2019/019497, filed May 16, 2019, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to an image processing device, a method, an image reading device, a program, and a recording medium.

In particular, the present invention relates to image processing for, in an image reading device that reads an original in different wavelength bands and generates a multispectral image constituted by multiple band images, improving the S/N ratio of the multispectral image.

BACKGROUND ART

In an image reading device that obtains multiple band images by reading an original in different wavelength bands, depending on the combination of the spectrum of the light source and the spectral sensitivity of the image sensor, there is a band image of a wavelength band in which a sufficient signal amount is not obtained. For example, band images obtained by illumination using ultraviolet light sources or near-infrared light sources often have lower S/N ratios than band images of visible light bands.

As another example of generating a multispectral image, in imaging the ground or the like from an optical satellite or aircraft, it is widely practiced to simultaneously obtain both a high-resolution panchromatic image and a low-resolution multispectral image for the same object.

In a system that simultaneously obtains both a low-resolution multispectral image and a high-resolution panchromatic image, a pan-sharpening process that combines the two images to generate a high-resolution color image is performed.

In the pan-sharpening process, removal of noise included in the multispectral image is a problem.

In removing noise of band images with low resolution and low S/N ratio by using a panchromatic image with high resolution and high S/N ratio, a noise removal device described in Patent Literature 1 performs multiresolution decomposition on the panchromatic image, performs multiresolution decomposition on the band images, corrects the decomposed components of the panchromatic image by using the decomposed components of the band images, and reconstructs a panchromatic image by using the corrected decomposed components and another decomposed component.

CITATION LIST

Patent Literature

• Patent Literature 1: International Publication No. 2015/037189 (paragraph 0024)

SUMMARY OF INVENTION

Technical Problem

As described above, in Patent Literature 1, noise correction is performed by a combination of a high-resolution panchromatic image and a multispectral image. In some cases, only a low-resolution multispectral image is obtained, and no high-resolution image is obtained. In a case where smoothing for noise reduction is performed with only a multispectral image, when the multispectral image includes a large amount of noise, the smoothing may destroy edges.

The present invention is to solve the above problem, and is intended to provide an image processing device capable of, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, improving the S/N ratio of the multispectral image without destroying edges.

Solution to Problem

An image processing device of the present invention includes: a parameter calculator to analyze a reference image of the same field of view as a plurality of band images constituting a multispectral image to calculate a local variance or an edge amount as an image feature amount, and calculate a filter parameter from the image feature amount; and a filter processor to perform an edge preserving smoothing process on the band images by using the filter parameter.

Advantageous Effects of Invention

The present invention makes it possible, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, to improve the S/N ratio of the multispectral image without destroying edges.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an image processing device of a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration example of an image reading device including the image processing device.

FIG. 3 is a block diagram illustrating a configuration example of an image combiner of FIG. 1.

FIG. 4 is a diagram illustrating an example of a spectral quantum efficiency of an image sensor.

FIG. 5 is a block diagram illustrating an image processing device of a second embodiment of the present invention.

FIG. 6 is a block diagram illustrating a configuration example of an image combiner of FIG. 5.

FIGS. 7A to 7C are diagrams illustrating examples of an image correlation calculated by a correlation calculator of FIG. 6.

FIG. 8 is a block diagram illustrating an image processing device of a third embodiment of the present invention.

FIG. 9 is a block diagram illustrating an image processing device of a fourth embodiment of the present invention.

FIG. 10 is a diagram illustrating an example of a computer that implements the functions of the image processing devices of the first to fourth embodiments, together with a light source and an image sensor.

FIG. 11 is a flowchart illustrating a procedure of a process when the function of the image processing device of the first or second embodiment is implemented by the computer of FIG. 10.

FIG. 12 is a flowchart illustrating a procedure of a process when the function of the image processing device of the third embodiment is implemented by the computer of FIG. 10.

FIG. 13 is a flowchart illustrating a procedure of a process when the function of the image processing device of the fourth embodiment is implemented by the computer of FIG. 10.

DESCRIPTION OF EMBODIMENTS

First Embodiment

FIG. 1 is a block diagram illustrating an image processing device 3 of a first embodiment of the present invention. FIG. 2 illustrates a configuration example of an image reading device including the image processing device 3 of FIG. 1.

The image reading device illustrated in FIG. 2 includes a light source 1 and an image sensor 2, in addition to the image processing device 3.

The light source 1 is constituted by multiple light sources each having a relatively narrow band. Hereinafter, the light sources having the relatively narrow bands will be referred to as band light sources. The multiple band light sources include, for example, a light source having a visible light wavelength band, a light source having an ultraviolet wavelength band, and a light source having a near-infrared wavelength band. The multiple band light sources are controlled to sequentially illuminate an original one by one.

The image sensor 2 sequentially obtains multiple band images DIN by imaging the original when the original as an object is sequentially illuminated by the above multiple band light sources. In general, the multiple band images DIN have different S/N ratios.

The image sensor 2 may be a one-dimensional image sensor (line sensor) or a two-dimensional image sensor (area sensor). The line sensor may be a contact image sensor.

The image processing device 3 combines the multiple band images DIN to generate a reference image having a higher S/N ratio than each band image, analyzes the reference image to calculate a local variance or an edge amount as an image feature amount, calculates a filter parameter from the image feature amount, and uses the filter parameter to smooth the multiple band images DIN while preserving edges.

The image processing device 3 includes an image combiner 31, a parameter calculator 32, and a filter processor 33.

The image combiner 31 combines the multiple band images DIN to generate a reference image MSr having a higher S/N ratio than each band image.

Use of a band image having a low S/N ratio in the combination may increase the noise amount of the reference image MSr and reduce the S/N ratio. To avoid reduction in the S/N ratio of the reference image MSr, in this embodiment, the combination is performed such that a combination weight of a band image whose noise amount is greater is smaller.

The image combiner 31 includes a noise amount calculator 311 and a weighting combiner 312, for example, as illustrated in FIG. 3.

The noise amount calculator 311 calculates a noise amount N_i of each of the multiple band images DIN. The calculated noise amounts N_i are each represented by a scalar, for example.

When the image processing device forms part of the image reading device illustrated in FIG. 2, a signal amount of each band image can be calculated by multiplication of the quantum efficiency of the image sensor 2 in the corresponding wavelength band and the amount of illumination by the used light source in the corresponding wavelength band. The noise amount N_i can be calculated from a reciprocal of the signal amount.

FIG. 4 illustrates an example of the spectral quantum efficiency of the image sensor. The image sensor has a peak near a center of a wavelength band in which the image sensor is sensitive, and the quantum efficiency decreases farther from the peak position.

When information on the quantum efficiency of the image sensor 2 or the amount of illumination by the light source cannot be used, the noise amount N_i of each band image can be calculated by analyzing each band image DIN.

Examples of the method of calculating the noise amount N_i by analyzing the image include a method of calculating a local variance of each pixel in a flat region of the image and determining an average thereof.

In this case, it is possible to calculate a local variance of each pixel and take, as a flat region, a region formed by pixels whose local variances are not greater than a threshold or a region in which the proportion of the pixels whose local variances are not greater than a threshold is not less than a predetermined value.

The local variances are determined by, for example, the same calculation as Equation (3) to be described later.

The weighting combiner 312 generates the reference image MSr by combining the multiple band images DIN such that the combination weight of a band image whose noise amount N_i is smaller is greater. The combination is performed by weighting and adding pixel values at the same positions of the multiple band images. Specifically, the weighting combiner 312 determines weighted averages of pixel values at the same positions of the multiple band images DIN as pixel values at the same positions of the reference image MSr.

The weighted average for a certain pixel (pixel of interest) is determined by, for example, the calculation represented by the following Equation (1):

$\begin{array}{cc}MSr\left(x\right)=\frac{1}{M}\sum _{i\in M}{W}_i·{\mathrm{MS}}_i\left(x\right).& \left(1\right)\end{array}$

In Equation (1),

• • x is a coordinate representing the position of the pixel of interest,
  • M denotes the number of the band images,
  • MS_i denotes the i-th band image of the M band images,
  • MS_i(x) denotes a pixel value at the position x of the band image MS_i,
  • W_i denotes a combination weight for the pixel value MS_i(x) of the band image MS_i, and
  • MSr(x) denotes a pixel value at the position x of the reference image MSr.

As the combination weight W_i for the pixel value MS_i(x) of each band image MS_i, a normalized value of a reciprocal of the noise amount N_i of the band image MS_i is used. The normalized value of the reciprocal of the noise amount of each band image is a value obtained by dividing the reciprocal of the noise amount of the band image by an average of the reciprocals of the noise amounts of all the band images. Thus, the combination weight W_i is represented by the following Equation (2):

$\begin{array}{cc}{W}_i=\frac{\frac{1}{N_i}}{\frac{1}{M}{\sum }_{i\in M}\frac{1}{N_i}}.& \left(2\right)\end{array}$

Instead of the above method, the weighting combiner 312 may set each weight W_i to 0 or 1 by using a preset noise amount threshold and determine the weighted averages. For example, it may set the weight W_i to 0 when the noise amount is not less than the noise amount threshold and to 1 otherwise, and determine the weighted averages by using the number of band images for which the weight W_i has been set to 1, instead of M.

The reference image MSr obtained by the above combination has a higher S/N ratio than each band image.

The parameter calculator 32 analyzes the reference image MSr to calculate, as an image feature amount, a local variance or an edge amount for each pixel of the reference image MSr, and calculates filter parameters D32 from the image feature amounts.

The local variance var(x) for a position x in the reference image MSr can be determined by, for example, performing the calculation represented by the following Equation (3):

$\begin{array}{cc}\mathrm{var}\left(x\right)=\frac{1}{L}\sum _{y\in \Omega \left(x\right)}MS{r\left(y\right)}^2-{\left(\frac{1}{L}\sum _{y\in \Omega \left(x\right)}MSr\left(y\right)\right)}^2.& \left(3\right)\end{array}$

In Equation (3),

• • Ω(x) denotes a local region centered at the position x,
  • L denotes the number of pixels in the local region Ω(x),
  • y is a coordinate representing a position in the local region Ω(x), and
  • MSr(y) denotes a pixel value at the position y of the reference image MSr.

The edge amount can be determined by, for example, a method using a bilateral weight. For example, the edge amount bw(x) for a position x in the reference image MSr can be determined by performing the calculation represented by the following Equation (4):

$\begin{array}{cc}bw\left(x\right)=\sum _{y\in \Omega \left(x\right)}\left(\exp \left(-\frac{1}{2}{\left(\frac{d\left(y,x\right)}{{\sigma }_1}\right)}^2\right)·\exp \left(-\frac{1}{2}{\left(\frac{MSr\left(y\right)-MSr\left(x\right)}{{\sigma }_2}\right)}^2\right)\right).& \left(4\right)\end{array}$

In Equation (4),

• • Ω(x) denotes a local region centered at the position x,
  • y is a coordinate representing a position in the local region Ω(x),
  • d(y,x) denotes a distance between the position indicated by the coordinate y and the position indicated by the coordinate x,
  • MSr(x) denotes a pixel value at the position x of the reference image MSr,
  • MSr(y) denotes a pixel value at the position y of the reference image MSr, and

σ₁ and σ₂ denote constants. The constants σ₁ and σ₂ are arbitrarily determined.

The parameter calculator 32 calculates and outputs the filter parameters D32 from the calculated image feature amounts.

For example, it may simply output the image feature amounts as the filter parameters, or may output, as the filter parameters, reciprocals of the image feature amounts or normalized values of them.

The filter processor 33 uses the filter parameters D32 to smooth the multiple band images DIN while preserving edges, and generates multiple output images DOUT.

The multiple output images DOUT correspond to the respective multiple band images DIN. A set of the multiple output images DOUT forms a multispectral image.

The filter process can be performed by using a reference smoothing filter to which the filter parameters can be input. For example, a joint bilateral filter, a guided filter, or the like can be used.

The filtering process using a joint bilateral filter is represented by, for example, the following Equation (5):

$\begin{array}{cc}DOUT\left(x\right)=\frac{1}{{\sum }_{y\in \Omega \left(x\right)}\mathrm{Ws}\left(x,y,{\sigma }_s\right)Wc\left(x,y,{\sigma }_c\right)}\sum _{y\in \Omega \left(x\right)}Ws\left(x,y,{\sigma }_s\right)Wc\left(x,y,{\sigma }_c\right)DIN\left(y\right).& \left(5\right)\end{array}$

In Equation (5),

• • x is a coordinate representing the position of the pixel of interest,
  • Ω(x) denotes a local region centered at the position x,
  • y is a coordinate representing a position in the local region Ω(x),
  • DIN(y) denotes a pixel value at the position y of the band image DIN, and
  • DOUT(x) denotes a pixel value at the position x of the output image DOUT.

The range of the above local region Ω(x) need not be the same as the range of the local region Ω(x) in the above Equation (3) or (4).

In Equation (5), Ws is a distance weight, and determined by, for example, the following Equation (6):

$\begin{array}{cc}Ws\left(x,y,{\sigma }_s\right)=\exp \left(-\frac{1}{2}{\left(\frac{d\left(y,x\right)}{{\sigma }_s}\right)}^2\right).& \left(6\right)\end{array}$

In Equation (6),

• • d(y,x) denotes a distance between the position indicated by the coordinate y and the position indicated by the coordinate x, and
  • σ_s is a parameter that determines the distance weight Ws, and denotes a variance.

In Equation (5), We is a pixel value weight, and determined by, for example, the following Equation (7):

$\begin{array}{cc}Wc=\exp \left(-\frac{1}{2}{\left(\frac{\left(MSr\left(y\right)-MSr\left(x\right)\right)}{{\sigma }_c}\right)}^2\right).& \left(7\right)\end{array}$

In Equation (7), σ_c is a parameter that determines the pixel value weight Wc, and denotes a variance.

In this embodiment, var(x) given by the above Equation (3), bw(x) given by Equation (4), or a value obtained by normalizing one of them is used as the parameter σ_c.

In the image processing device 3 of the first embodiment, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges.

Also, since the image combiner 31 includes the noise amount calculator 311 as illustrated in FIG. 3, by determining the combination weights on the basis of the noise amounts of the band images used in the combination, it is possible to generate the reference image MSr having a higher S/N ratio, and thus it is possible to obtain output images having high S/N ratios.

In the above example, the multiple band light sources include the light source having the visible light wavelength band, the light source having the ultraviolet wavelength band, and the light source having the near-infrared wavelength band. The set of the multiple band light sources is not limited to the above example. For example, the visible light wavelength band may be divided. For example, a light source having a red wavelength band, a light source having a green wavelength band, and a light source having a blue wavelength band may be provided instead of the light source having the visible light wavelength band.

In the above example, the variances or edge amounts are determined as the image feature amounts by the parameter calculator 32, and the values or normalized values thereof are used as the parameters σ_c for determining the combination weights. The image feature amounts determined by the parameter calculator 32 may be values other than the variances or edge amounts. In any case, the image feature amounts, reciprocals thereof, or normalized values thereof may be used as parameters for determining the combination weights.

Second Embodiment

FIG. 5 illustrates an image processing device 3b of a second embodiment of the present invention.

The illustrated image processing device 3b is generally the same as the image processing device 3 of FIG. 1, but includes an image combiner 31b instead of the image combiner 31.

The image combiner 31 of the first embodiment performs the image combination such that the combination weight of a band image whose noise amount is greater is smaller. The image combiner 31b of this embodiment performs the image combination by using combination weights determined on the basis of image correlations between the band images.

The image characteristics of each band image depend on its wavelength band, and the farther the wavelength bands of two band images are from each other, the greater the difference in image characteristics between the two band images.

In this embodiment, when it is intended to reduce the noise of a band image of a certain wavelength band, the reference image is generated by performing the image combination such that the combination weight for a band image whose image correlation with the band image of the target wavelength band (the image of the target wavelength band) is greater is greater.

The image combiner 31b combines the multiple band images DIN to generate a reference image MSr having a higher S/N ratio than each band image.

The image combiner 31b includes a correlation calculator 313 and a weighting combiner 312b, for example, as illustrated in FIG. 6.

The correlation calculator 313 calculates, for each of the multiple band images DIN, an image correlation Cor_i with a band image of a target wavelength band.

The image correlation Cor_i between each band image and the band image of the target wavelength band can be determined on the basis of, for example, a difference in wavelength between each band image and the band image of the target wavelength. In this case, it can be considered that the smaller the above difference, the higher the image correlation.

In this case, the above image correlation can be represented by a function of wavelength.

FIGS. 7A to 7C illustrate examples of the image correlation. In FIGS. 7A to 7C, the horizontal axis represents the wavelength of each band image, and the vertical axis represents the image correlation. The image correlation is expressed in an arbitrary unit (a.u.).

λ_t on the horizontal axis denotes a center wavelength or peak wavelength (target wavelength) of the target wavelength band.

In the example illustrated in FIG. 7A, the correlation is highest at the target wavelength λ_t, and linearly decreases as the difference from λ_t increases.

In the example illustrated in FIG. 7B, the correlation keeps a constant value in the range in which the difference from the target wavelength λ_t is not greater than a set difference threshold, and is 0 in the range in which the difference is greater than the difference threshold.

In the example illustrated in FIG. 7C, the correlation varies along a normal distribution curve centered at the target wavelength λ_t.

In the above example, the correlation calculator 313 determines, for each band image, the image correlation on the basis of the difference in wavelength from the band image of the target wavelength band.

Alternatively, it is possible to determine, for each band image, the image correlation on the basis of the similarity with the band image of the target wavelength band.

As an indicator of the similarity, a sum of absolute differences (SAD), a sum of squared differences (SSD), a normalized cross correlation (NCC), a zero-means normalized cross correlation (ZNCC), or the like can be used, for example.

SAD for each band image MS_i (denoted by the symbol SA_i) is determined by the following Equation (8):

$\begin{array}{cc}S{A}_i=\sum _x❘{\mathrm{MS}}_i\left(x\right)-MSt\left(x\right)❘.& \left(8\right)\end{array}$

SSD for each band image MS_i (denoted by the symbol SS_i) is determined by the following Equation (9):

$\begin{array}{cc}S{S}_i=\sum _x{\left({\mathrm{MS}}_i\left(x\right)-MSt\left(x\right)\right)}^2.& \left(9\right)\end{array}$

NCC for each band image MS_i (denoted by the symbol NC_i) is determined by the following Equation (10):

$\begin{array}{cc}N{C}_i=\frac{{\sum }_x{\mathrm{MS}}_i\left(x\right)MSt\left(x\right)}{\sqrt{{\sum }_x{{\mathrm{MS}}_i\left(x\right)}^2{\sum }_x{\mathrm{MSt}\left(x\right)}^2}}.& \left(10\right)\end{array}$

ZNCC for each band image MS_i (denoted by the symbol ZNC_i) is determined by the following Equation (11):

$\begin{array}{cc}ZN{C}_i=\frac{{\sum }_x\left({\mathrm{MS}}_i\left(x\right)-M{S}_{i}m\left(x\right)\right)\left(MSt\left(x\right)-MStm\left(x\right)\right)}{\sqrt{{\sum }_x{\left({\mathrm{MS}}_i\left(x\right)-M{S}_{i}m\left(x\right)\right)}^2{\sum }_x{\left(\mathrm{MSt}\left(x\right)-MStm\left(x\right)\right)}^2}}.& \left(11\right)\end{array}$

In Equations (8) to (11),

• • x is a coordinate representing the position of the pixel of interest,
  • MS_i(x) denotes a pixel value at the position x of the band image MS_i,
  • MSt(x) denotes a pixel value at the position x of the band image MSt of the target wavelength band,
  • MS_im denotes an average of the pixel values MS_i(x) of all the pixels in the image MS_i, and
  • MStm denotes an average of the pixel values MSt(x) of all the pixels in the image MSt.

The higher the similarity, the smaller the values of SAD and SSD. The higher the similarity, the greater the values of NCC and ZNCC.

SAD and SSD are appropriate when it is intended to make a comparison in terms of image characteristics including intensity (brightness, luminance), and NCC and ZNCC, each of which provides a value independent of the image intensities (the magnitudes of the pixel values) due to normalization, are appropriate when it is intended to extract spectral reflection characteristics of the object in a specific wavelength band.

The weighting combiner 312b generates the reference image MSr by combining the multiple band images DIN such that the combination weight of a band image whose image correlation Cor_i is greater is greater. The generation is performed by weighting and adding pixel values at the same positions of the multiple band images. Specifically, the weighting combiner 312b determines weighted averages of pixel values at the same positions of the multiple band images DIN as pixel values at the same positions of the reference image MSr.

The weighted average for a certain pixel (pixel of interest) is determined by, for example, the calculation represented by the above Equation (1).

However, as the combination weight W_i for the pixel value MS_i(x) of each band image MS_i, a normalized value of the image correlation Cor_i of the band image MS_i is used. The normalized value of the image correlation of each band image is a value obtained by dividing the image correlation of the band image by an average of the image correlations of all the band images. Thus, the combination weight W_i is represented by the following Equation (12):

$\begin{array}{cc}{W}_i=\frac{Co{r}_i}{\frac{1}{M}{\sum }_{i\in M}{\mathrm{Cor}}_i}.& \left(12\right)\end{array}$

Although in the above example, the similarity of an image is determined for the entire image, it is possible to determine the similarity for each pixel and determine the image correlation for each pixel.

In this case, in Equation (8) to (11), the similarity is determined by using pixels in a local region centered at the pixel of interest, and the image correlation for the pixel of interest is determined from the determined similarity.

Also in the image processing device 3b of the second embodiment, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges.

Also, since the image combiner 31b includes the correlation calculator 313 as illustrated in FIG. 6, by determining the combination weights on the basis of the image correlations between the band images used in the combination and the band image of the target wavelength band, it is possible to generate an output image with less collapse in image characteristics.

For the first embodiment, as another example of the weighting combination, there has been described a method of setting each weight W_i to 0 or 1 by using a preset noise amount threshold and determining the weighted averages. The same modification can be made to the second embodiment.

Specifically, it is possible to set each weight W_i to 0 or 1 by using a preset correlation threshold and determine the weighted averages. For example, it is possible to set the weight W_i to 1 when the image correlation is not less than the image correlation threshold and to 0 otherwise, and determine the weighted averages by using the number of band images for which the weight W_i has been set to 1, instead of M.

Third Embodiment

FIG. 8 illustrates an image processing device 3c of a third embodiment of the present invention.

The illustrated image processing device 3c is generally the same as the image processing device 3 of FIG. 1, but does not include the image combiner 31 of FIG. 1 and includes a parameter calculator 32c instead of the parameter calculator 32 of FIG. 1.

Also, in addition to the band images DIN of FIG. 1, a white image DIW is input to the image processing device 3c.

The band images DIN are the same as the band images of FIG. 1.

The white image DIW is an image having a band including all the bands of the multiple band images DIN, and is used as a reference image. The band of the white image DIW is preferably wider than a band obtained by summing all the bands of the multiple band images DIN.

The parameter calculator 32c analyzes the white image DIW, and for each pixel of the white image DIW, calculates a local variance or an edge amount as an image feature amount and calculates a filter parameter D32c from the image feature amount.

The content of the process in the parameter calculator 32c is the same as that of the process in the parameter calculator 32 of FIG. 1. It differs in determining the filter parameters by using the white image DIW as the reference image.

The filter processor 33 uses the filter parameters D32c to smooth the multiple band images DIN while preserving edges, and generates multiple output images DOUT.

The content of the process in the filter processor 33 is the same as that of the process in the filter processor 33 of FIG. 1.

In the image processing device 3c of the third embodiment, it is possible to obtain the white image having the band including all the bands of the multiple band images and use it as the reference image. Thus, even in a case in which the number of wavelength bands of the used light source is small, when a multispectral image including a band image of a wavelength band in which a sufficient signal amount is not obtained is obtained, it is possible to improve the S/N ratio of the multispectral image without destroying edges.

Fourth Embodiment

FIG. 9 illustrates an image processing device 3d of a fourth embodiment of the present invention.

The illustrated image processing device 3d is generally the same as the image processing device 3c of FIG. 8, but includes a bandpass filter group 34. Also, the multiple band images DIN are not input to the image processing device 3d, and only the white image DIW is input to the image processing device 3d.

The bandpass filter group 34 receives as an input the white image and includes multiple bandpass filters that pass different wavelength bands, and multiple band images D34 of different wavelength bands are output from the bandpass filter group 34.

In the above first to third embodiments, the multiple band images DIN are sequentially obtained by performing imaging when the original is sequentially illuminated by the multiple band light sources having different wavelength bands. On the other hand, in the fourth embodiment, the multiple band images DIN are generated by using the multiple bandpass filters that pass the different wavelength bands of the white image of a wide band.

The filter processor 33 uses the filter parameters D32c to smooth the multiple band images D34 while preserving edges, and generates multiple output images DOUT.

The fourth embodiment provides the same advantages as the first to third embodiments.

In the above first to fourth embodiments, the image processing device forms part of the image reading device. However, image processing devices of the present invention can be used for purposes other than image reading. For example, they can also be used in imaging the ground or the like from an optical satellite or aircraft.

Each of the image processing devices 3, 3b, 3c, and 3d described in the first to fourth embodiments may be partially or wholly formed by processing circuitry.

For example, the functions of the respective portions of the image processing device may be implemented by respective separate processing circuits, or the functions of the portions may be implemented by a single processing circuit.

The processing circuitry may be implemented by hardware, or by software or a programmed computer.

It is possible that a part of the functions of the respective portions of the image processing device is implemented by hardware and another part is implemented by software.

FIG. 10 illustrates an example of a configuration when all the functions of the image processing device 3, 3b, 3c, or 3d of the above respective embodiments are implemented by a computer 9 including a single processor, together with the light source 1 and image sensor 2.

In the illustrated example, the computer 9 includes a processor 91 and a memory 92.

A program for implementing the functions of the respective portions of the image processing device is stored in the memory 92.

The processor 91 uses, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, a digital signal processor (DSP), or the like.

The memory 92 uses, for example, a semiconductor memory, such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a magnetic disk, an optical disk, a magnetic optical disk, or the like.

The processor 91 implements the function of the image processing device by executing the program stored in the memory 92.

A procedure when a process in the image processing device of the first or second embodiment is performed by the computer illustrated in FIG. 10 will be described with reference to FIG. 11.

In step ST11, the multiple band images DIN are received.

In step ST12, the reference image MSr is generated by combining the multiple band images DIN. This process is performed as described for the image combiner 31 or 31b.

In step ST13, the local variances or edge amounts are calculated as the image feature amounts by analyzing the reference image MSr, and the filter parameters D32 are calculated from the image feature amounts. This process is performed as described for the parameter calculator 32.

In step ST14, smoothing the multiple band images DIN while preserving edges is performed by using the filter parameters D32. This process is performed as described for the filter processor 33.

A procedure when a process in the image processing device of the third embodiment is performed by the computer illustrated in FIG. 10 will be described with reference to FIG. 12.

In FIG. 12, processes that are the same as those in FIG. 11 are indicated by the same symbols.

In step ST11, the multiple band images DIN are received.

In step ST22, the white image DIW is received.

In step ST13c, the local variances or edge amounts are calculated as the image feature amounts by analyzing the white image DIW, and the filter parameters D32c are calculated from the image feature amounts. This process is performed as described for the parameter calculator 32c.

In step ST14, smoothing the multiple band images DIN while preserving edges is performed by using the filter parameters D32c. This process is performed as described for the filter processor 33.

A procedure when a process in the image processing device of the fourth embodiment is performed by the computer illustrated in FIG. 10 will be described with reference to FIG. 13.

In FIG. 13, processes that are the same as those in FIG. 11 or 12 are indicated by the same symbols.

In step ST22, the white image DIW is received.

In step ST13c, the local variances or edge amounts are calculated as the image feature amounts by analyzing the white image DIW, and the filter parameters D32c are calculated from the image feature amounts. This process is performed as described for the parameter calculator 32c.

In step ST33, the bandpass filtering that passes the different wavelength bands is performed on the white image DIW, so that the multiple band images D34 are generated. This process is performed as described for the bandpass filter group 34.

The process of step ST33 can be performed in parallel with the process of step ST13c.

In step ST14, smoothing the multiple band images D34 while preserving edges is performed by using the filter parameters D32c. This process is performed as described for the filter processor 33.

Although image processing devices of the present invention have been described above, the image processing methods implemented by the above image processing devices also form part of the present invention. Also, programs for causing computers to execute processes of the above image processing devices or image processing methods, and computer-readable recording media, e.g., non-transitory recording media, storing the programs also form part of the present invention.

Although embodiments of the present invention have been described, the present invention is not limited to these embodiments.

REFERENCE SIGNS LIST

• • 1 light source, 2 image sensor, 3, 3b, 3c, 3d image processing device, 31 image combiner, 32, 32c parameter calculator, 33 filter processor, 34 bandpass filter group, 311 noise amount calculator, 312, 312b weighting combiner, 313 correlation calculator.

Claims

1. An image processing device comprising: a parameter calculator to analyze a reference image of the same field of view as a plurality of band images constituting a multispectral image to calculate a local variance or an edge amount as an image feature amount, and calculate a filter parameter from the image feature amount;a filter processor to perform an edge preserving smoothing process on the band images by using the filter parameter; andan image combiner to combine the plurality of band images to generate the reference image,wherein the image combiner generates the reference image by calculating a noise amount for each of the plurality of band images and performing the combination such that a combination weight of a band image whose noise amount is smaller is greater.

2. The image processing device of claim 1, wherein the plurality of band images are obtained by performing imaging while an object is sequentially illuminated by a plurality of light sources each having a relatively narrow band.

3. The image processing device of claim 2, wherein the plurality of light sources include a light source having a visible light wavelength band, a light source having an ultraviolet wavelength band, and a light source having a near-infrared wavelength band.

4. The image processing device of claim 1, wherein the reference image is a white image having a band including all of bands of the plurality of band images.

5. An image processing device comprising: a parameter calculator to analyze a reference image of a same field of view as a plurality of band images constituting a multispectral image to calculate a local variance or an edge amount as an image feature amount, and calculate a filter parameter from the image feature amount;a filter processor to perform an edge preserving smoothing process on the band images by using the filter parameter; andan image combiner to combine the plurality of band images to generate the reference image,wherein the image combiner generates the reference image by calculating, for each of the plurality of band images, an image correlation with one of the plurality of band images of a target wavelength band and performing the combination such that a combination weight of a band image whose image correlation is greater is greater.

6. The image processing device of claim 5, wherein the image combiner calculates, for each of the plurality of band images, the image correlation on a basis of a difference in wavelength from the hand image of the target wavelength band.

7. The image processing device of claim 5, wherein the image combiner calculates, for each of the plurality of band images, as the image correlation, a similarity with the band image of the target wavelength band.

8. The image processing device of claim 7, wherein the image combiner calculates the image correlation for each pixel, determines the combination weight for each pixel on a basis of the calculated image correlations, and performs the combination.

9. The image processing device of claim 5, wherein the plurality, of band images are obtained by performing imaging while an object is sequentially illuminated by a plurality of light sources each having a relatively narrow band.

10. The image processing device of claim 9, wherein the plurality of light sources include a light source having a visible light wavelength band, a light source having an ultraviolet wavelength band, and a light source having a near-infrared wavelength band.

11. The image processing device of claim 5, wherein the reference image is a white image having a band including all of bands of the plurality of band images.

12. An image reading device comprising: a light source to illuminate an original;an image sensor to image the original; andthe image processing device of claim 1 to process an image obtained by imaging by the image sensor.

13. An image processing method comprising: combining a plurality of band images to generate a reference image;analyzing the reference image of the same field of view as the plurality of band images constituting a multispectral image to calculate a local variance or an edge amount as an image feature amount, and calculating a filter parameter from the image feature amount; andperforming an edge preserving smoothing process on the band images by using the filter parameter,wherein said combining generates the reference image by calculating a noise amount for each of the plurality of band images and performing the combination such that a combination weight of a band image whose noise amount is smaller is greater.

14. A non-transitory computer-readable recording medium storing a program for causing a computer to execute a process of the image processing method of claim 13.