1. Field of the Invention
The present invention relates to an image capturing apparatus, an image capturing system, a method for controlling an image capturing apparatus, and a storage medium.
2. Description of the Related Art
In order to examine an internal structure by creating an image of the spatial distribution of the intensity of X-rays that pass through a human body or an object, digital images of X-ray images have become widespread. In the medical field in particular, large-sized flat semiconductor image capturing apparatuses (flat panel detectors; below, “FPD”) are used to examine the inner part of a comparatively large object, such as a human body. By using an FPD, it is possible to obtain a digital medical X-ray image with a digitized X-ray intensity distribution can be acquired.
A large amount of superimposed noise on an image is given as a general characteristic of medical X-ray images. This derives from the fact that in order to minimize the X-ray dosage with the goal of reducing radiation exposure to a subject, the X-ray dosage is at a level at which the portion with a weak X-ray intensity to be formed into an image is handled as an energy quantum observed in units of several to hundreds per 1 mm2. In order to obtain the maximum amount of information from a medical X-ray image that has a lot of this kind of original noise and use it for a medical examination, it is considered to be necessary to reduce so-called system noise that occurs with an FPD and a peripheral electric circuit as much as possible.
Additionally, a wide dynamic range of the needed X-ray intensity distribution information is given as another characteristic of medical X-ray images. Ordinarily, there are cases where areas in which the X-ray transmission rate of a subject such as a human is low are several thousandths that of areas in which the X-ray transmission rate is high, and if the purpose is to simply examine an image, this can be achieved by saturating the areas with strong X-rays. However, because linearity of the X-ray intensity distribution is needed in order to use it for an arithmetic operation such as X-ray CT, a wide dynamic range is needed.
In order to achieve the two goals of system noise reduction and dynamic range expansion, it is useful to use a sensor that can read out pixel information regarding accumulated charge multiple times as the same voltage value. Specifically, by compositing multiple nondestructive read images, a composite image is created with a smaller amount of noise or an expanded dynamic range. This nondestructive read technique is disclosed in Japanese Patent Laid-Open No. 62-85585 and Japanese Patent No. 2966977.
On the other hand, there is another conventional technique that corrects a defective pixel on an FPD. Because FPDs are manufactured with a semiconductor manufacturing process, defects occur in diodes, transistors, or wiring portions due to accidental events during the manufacturing process or due to the partial intrusion of impurities, and sometimes a specific pixel will not respond to light. This is called a defective pixel, which cannot hold a pixel value obtained from a particular corresponding pixel. Usually, a pixel value that corresponds to a defective pixel is set by estimating it from surrounding normal pixel values. The most typical and mathematically stable estimated value is the average of the peripheral pixels. For example, as shown in
p11′=(p00+p01+p02+p10+p12+p20+p21+p22)÷8 (a)
In another example, as shown in
p11′=(p00+p02+p10+p12+p20+p22)÷6 (b)
Defective pixel correction is a technique for outputting a maximum-likelihood value as an average value that corresponds to an unclear defective pixel value, but there is a problem area due to it being an average value. As stated before, because medical x-ray images are generally expressed according to few energy quantum numbers, they are images with a large amount of noise. With defective pixel correction using an average, noise of a pixel value is substantially lessened, but it can become a value that is not valid as a statistical property of an X-ray image. If there is a defective pixel that is isolated singularly, it often does not become a problem, but if defective pixels are successive and become a cluster of defective pixels as shown in
As a specific effect of this, for example, if correction is performed when defective pixels form a line, the amount of noise of the corrected line will differ from the amount of noise of the peripheral pixels, and therefore a smooth linear area can be observed by an observer. For example, a case is presumed in which a linear defective pixel cluster, like the one in
Regarding this problem, for example, Japanese Patent No. 4532730 discloses operations such as adding fluctuation upon changing the number of peripheral pixels to be averaged with the goal of preserving the statistical properties. However, the method according to Japanese Patent No. 4532730 deviates from a calculation that obtains a maximum-likelihood value with high reliability, which is the primary goal of defective pixel correction, and it is not an accurate defective pixel correction (estimation).
In view of the problems stated above, the present invention provides an image capturing technique in which it is possible to correct a defective pixel so that the statistical properties of peripheral pixels and the statistical properties of the defective pixel are the same.
According to one aspect of the present invention, there is provided an image capturing apparatus comprising: an image capturing unit configured to accumulate a charge pixel-by-pixel and output an image signal corresponding to an amount of accumulated charge; a generation unit configured to acquire image signals of pixels from the image capturing unit by a set number of readouts, calculate, with respect to each pixel, image data that is an average of image signals read out from the same pixel, and generate an image having the image data as a characteristic of the pixels; and a correction unit configured to specify a position of a defective pixel of the image capturing unit with use of information indicating a position of a defective pixel, and correct the image data by generating correction data that is an average of image signals of a set certain number of pixels located in a periphery of the defective pixel.
According to the present invention, a defective pixel can be corrected so that the statistical properties of peripheral pixels and the statistical properties of the defective pixel are the same.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Exemplary embodiments of the present invention will be described in detail below with reference to the drawings. However, the constituent elements described in the embodiments are just examples, and the technical scope of the present invention is defined by the scope of the claims and not by the embodiments below.
A characteristic of this source follower amplifier is that, because the input impedance is high, a charge stored in the capacitor 103 is not consumed. Due to the output of a shift register 101 for sequential row selection being input to the gate of a transistor 105, voltage from the capacitors 103 is transmitted to a multiplexer 109 for sequential column selection.
Voltage data of row 1, selected by the shift register 101, and column 1, selected by the multiplexer 109, is sequentially output as a signal (indicated by reference numeral 106) via an amplifier 107 that holds a gain A. Here, because the source follower amplifier 104 does not consume the charge, it is possible to draw the same voltage multiple times. Because of this, even if the image signals (voltages) of pixels that configure the FPD are read out multiple times, it is possible to read out the same image signal (voltage) each readout time. This kind of readout of an image signal (voltage) is called a “nondestructive read”. If the charge accumulated by the capacitor 103 is not needed, a transistor 108 can reset a block corresponding to one pixel, and return the block to an initial state in which another charge can be accumulated.
A controller 79 that performs overall control of the image capturing apparatus outputs a New_page signal at a time immediately before a pixel charge is accumulated and resets charges accumulated by blocks of the FPD 71. Additionally, the controller 79 outputs a Read_image signal at a time when a nondestructive read is being performed. Upon receiving the Read_image signal output from the controller 79, the number of times a nondestructive read is performed is counted. A counter 74 counts the number of times a nondestructive read is performed. When the counter 74 receives the New_page signal output from the controller 79, it clears the count value to zero.
An A/D (analog/digital) converter 72 converts the signal 106, which is an output voltage (nondestructive image information) from the FPD 71, into a digital value, and outputs the digital value to an average calculation unit 73. The average calculation unit 73 has two data inputs a and b, and outputs a value (average value) shown as c based on a count number N for obtaining an average. An output value c of the average calculation unit 73 is calculated with an equation 1 written below. The average calculation unit 73 can use a floating-point operation in the calculation of equation 1.
c=b+(a−b)/N(N≧1) (1)
The output value c of the average calculation unit 73 is input to an image memory 75 and stored. When the image memory 75 receives the New_page signal output from the controller 79, all of the contents of the image memory 75 can be cleared. The pixel values stored in the image memory 75 are read out sequentially during the next nondestructive read, and are input to the average calculation unit 73 (data input b). An average value (c) of the nondestructive read image is stored in the image memory 75. An averaged image of the nondestructive read is output as a composite image from the image memory 75 to a signal line 76, and this composite image data with little system noise is obtained.
When a gain controller 88 receives the Read_image signal output from the controller 79 at the time of the nondestructive read, it outputs gain values to the FPD 71 at the same time. The gain values have been set in advance at different values such as a1, a2, and the like, and the gain controller 88 outputs these gain values to the FPD 71 via a gain signal line.
In accordance with the gain value output from the gain controller 88, the FPD 71 sets the gain A of the amplifier 107 provided in the last step of output described in
The sum-of-product average calculation unit 80 performs calculations 2 and 3 below based on a weighted average obtained by multiplying the output value a of the A/D converter 72 by reciprocals of gain values G={a1, a2, . . . }. The sum-of-product average calculation unit 80 outputs a value d, calculated with the calculation written below. The sum-of-product average calculation unit 80 can use a floating-point operation in the calculation of equations 2 and 3.
d=b(a≧s) (2)
d=b+(a/G−b)/N(a<s) (3)
When the gain controller 88 receives the Read_image signal output from the controller 79 at the time a nondestructive read is performed, the gain values that are set in advance are updated in the order a1, a2, . . . , and output. As the magnitude correlation of the gain values, it is necessary that they are a1≦a2≦a3 . . . (s1≧s2≧s3 . . . ). Ultimately, a composite image with an expanded dynamic range is output to the signal wire indicated by reference numeral 76 during the Nth nondestructive read. Here, if the magnitude correlation is a1=a2=a3 . . . (s1=s2=s3 . . . ), it will become an apparatus with the same function as in
In the present embodiment, as an example, a case is described in which image signals of pixels are acquired from an FPD by a set number of readouts (e.g., 4) and the average of four nondestructive read images is output. In this case, if σ2 is the power of system noise (variance value) of one nondestructive read image, the power of system noise of an output image averaged from four images decreases to σ2/4. Accordingly, in the case where N=1, if four nondestructive read image pixels are used in defective pixel correction, and the pixel defect correction unit 11 calculates the average (arithmetic average), the noise power of the defective pixel is equal to the noise power of the peripheral pixels.
pYX(A)=(pYX(1)+pYX(2)+pYX(3)+pYX(4))+4 (4)
Here, a case is presumed in which the area corresponding to p11(N) is a defective pixel and the peripheral pixels around it are normal pixels. The pixel defect correction unit 11 calculates an average (arithmetic average) with four peripheral pixels only when N=1, and when the calculation of equation 4 is complete, the pixel defect correction unit 11 corrects the value of p11(A) by rewriting it with the calculation result of equation 5.
p11(A)=(p00(1)+p02(1)+p20(1)+p22(1))+4 (5)
When the composite image calculation of equation 4 and the defect correction calculation of equation 5 are compared, the average peripheral pixel value and the corrected defective pixel value are both averages of four pixel values that hold similar statistical properties. Because of this, the statistical properties in defective pixel correction are the same as the statistical properties of the pixel values of the peripheral pixels and defective pixel correction is possible.
Variation
In the calculation of equation 5, in the case of a nondestructive read image when N=1, the pixel values of four pixels in the periphery of a defective pixel are used for defective pixel correction, however, the gist of the present invention is not limited to this example, and it is possible to use the pixel values of each image N. For example, let p00(1) be a pixel value in a nondestructive read image where N=1, and let p02(2) be a pixel value in a nondestructive read image where N=2. Additionally, let p20(3) be a pixel value in a nondestructive read image where N=3, and let p22(4) be a pixel value in a nondestructive read image where N=4. The pixel defect correction unit 11 calculates p11(A) from the average of four peripheral pixels using the pixel values surrounding the defective pixel with regards to an image having a different nondestructive read image number N. Then, after the calculation of equation 4 is complete, the pixel defect correction unit 11 can also rewrite the value of p11(A) with the calculation result of equation 6.
p11(A)=(p00(1)+p02(2)+p20(3)+p22(4))÷4 (6)
When the composite image calculation of equation 4 and the defect correction calculation of equation 6 are compared, the average peripheral pixel value and the defective pixel correction value are both averages of four pixels that hold similar statistical properties. For this reason, the statistical properties of defective pixel correction are the same as the statistical properties of the pixel values of the peripheral pixels and defective pixel correction is possible.
In
In the present embodiment, dynamic range expansion is performed using four nondestructive read images, and therefore, a case in which a weighted average operation is performed using the gain values G=a1, a2, a3, and a4 will be described as an example.
With regards to pixel values of reference numerals 31 to 34 {pYX(N)|Y,X=0, 1, 2; N=1 to 4}, described using
pYX(A)=(pYX(1)/a1+pYX(2)/a2+pYX(3)/a3+pYX(4)/a4)÷(1/a1+1/a2+1/a3+1/a4) (7)
Here, it is presumed that the area corresponding to p11(N) is a defective pixel, and peripheral pixels around that are normal pixels. The pixel defect correction unit 33 calculates the weighted average with four peripheral pixels only when N=1, and after the calculation of equation 7 is complete, the pixel defect correction unit 33 rewrites the value of p11(A) with the calculation results of equation 8.
p11(A)=(p00(1)/a1+p02(1)/a2+p20(1)/a3+p22(1)/a4)÷(1/a1+1/a2+1/a3+1/a4) (8)
When the composite image calculation of equation 7 and the correction calculation of the defective pixel in equation 8 are compared, the weighted average value of the peripheral pixels and the corrected value of the defective pixel are both weighted averages of four pixel values that hold similar statistical properties. Because of this, the statistical properties of the corrected values of the defective pixels are the same as the statistical properties of the pixel values of the peripheral pixels and defective pixel correction is possible.
Variation
In the calculation of equation 8, in the case of a nondestructive read image when N=1, the pixel values of four pixels in the periphery of a defective pixel are used for defective pixel correction, however, the gist of the present invention is not limited to this example, and it is possible to use the pixel values of pixels of each image N. For example, let p00(1) be a pixel value in a nondestructive read image where N=1, and let p02(2) be a pixel value in a nondestructive read image where N=2. Additionally, let p20(3) be a pixel value in a nondestructive read image where N=3, and let p22(4) be a pixel value in a nondestructive read image where N=4. The pixel defect correction unit 33 calculates p11(A) from the average of four peripheral pixels using the pixel values surrounding the defective pixel with regards to an image having a different nondestructive read number N. Then, after the calculation of equation 7 is complete, the pixel defect correction unit 33 can also rewrite the value of p11(A) with the calculation result of equation 9.
p11(A)=(p00(1)/a1+p02(2)/a2+p20(3)/a3+p22(4)/a4)÷(1/a1+1/a2+1/a3+1/a4) (9)
When the composite image calculation of equation 7 and the defective pixel calculation of equation 9 are compared, the weighted average value of the peripheral pixels and the corrected value of the defective pixel are both weighted averages of four pixel values that hold similar statistical properties. Because of this, the statistical properties of the corrected values of the defective pixels are the same as the statistical properties of the pixel values of the peripheral pixels and defective pixel correction is possible.
In the first and second embodiments described above, a case in which the number of nondestructive read images is N=4 was described as an example, but the gist of the present invention is not limited to this example, and the present invention can be applied to an arbitrary number of images N (a natural number). When there is an arbitrary number of images N, defective pixel correction according to equations 5 and 8 can be performed using the pixel values of a total of N pixels (peripheral pixels) from a nondestructive read image where N=1.
Alternatively, defective pixel correction according to equations 6 and 9 can be performed using the pixel value of one peripheral pixel from the nondestructive read image at each N. According to the present embodiment, the statistical properties of a defective pixel and a peripheral pixel are the same, and defective pixel correction is possible.
In the embodiments described above, correction of a defective pixel in a still image was given as an example, but rather than being limited to this example, the present invention is applicable to moving images also. Because a series of still images (frames) is captured when shooting a video, the methods of defect correction in the above-described embodiments can also be applied to the correction of defective frames that configure a video, with the use of, for example, a total of N normal frames.
In the embodiments described above, a case in which there is one composite image obtained with multiple nondestructive reads obtained from one accumulation of charge (reference numeral 35) is described. The gist of the present invention is not limited to this example, but rather can be applied to a case in which multiple composite images are created with multiple nondestructive reads. For example, when there are four nondestructive reads, multiple composite images can be generated with use of two or three nondestructive read images out of the four nondestructive reads. Even in the correction of defective pixels in this case, defective pixel correction according to equations 4 to 9 can be performed. According to the present embodiment, the statistical properties of the peripheral pixels and the defective pixel are the same, and defective pixel correction is possible.
The present invention can also be applied to defect correction of medical X-ray images in which X-ray dosage and the amount of noise have a clear relationship. Because X-ray dosage can be handled as a randomly generated energy quantum number, the fluctuation of X-ray dosage (i.e., a random quantum number) per unit area that arrives at a sensor, which called quantum noise, follows a Poisson distribution. That is to say, if X-ray dosage (quantum number) is expressed as Q, if it follows a Poisson distribution, the variance value σ2 is identical to the average value and can be expressed as Q. If the ratio of σ, which is the square root of the average value, and the variance value is considered to be the signal-to-noise ratio (S/N ratio), and the S/N ratio of the X-ray image acquired with an X-ray dosage of intensity Q can be expressed as Q/Q1/2=Q1/2. The X-ray dosage and amount of noise (signal-to-noise ratio (S/N ratio)) can be bound in a unique relationship.
An calculation unit 420 generates a composite image by averaging N nondestructive read images in order to improve the S/N ratio. An S/N ratio 48 of the composite image generated here reduces system noise and becomes Q/(Q+P/N)1/2. Conventionally, defective pixel correction is performed using this composite image. For example, if noise is linear as shown in
On the other hand, the nondestructive read image 47 is used for defective pixel correction in the present embodiment of this invention. The defective pixel correction according to a calculation unit 410 depends on an average of M pixels, and the S/N ratio 40 of the defective pixel cluster is an average of M pixels, and therefore, can be expressed with M1/2. Q/(Q+P)1/2.
For the sake of comparison here, the S/N ratio 48 of the composite image=Q/(Q+P/N)1/2 and the S/N ratio 40 of the defective pixel correction=M1/2. Q/(Q+P)1/2.
If the value of M is calculated with both as equals, equation 10 below is produced.
M=(Q/P+1)/(Q/P+1/N) (10)
This indicates that the number of pixels used for calculating defective pixel correction depends on the ratio (Q/P) of the X-ray dosage (variance of quantum noise) Q and the variance of system noise P, and the number of nondestructive read images N used for the calculation of the composite image. According to the result of this calculation, the number of pixels for processing of an average can be set.
In the image sensing region of the FPD, in an area with a high X-ray dosage above a predetermined reference value (an area where the ratio (Q/P) of X-ray dosage Q and the variance of system noise of the FPD P is close to zero), M=N (M, N≧1: M, N being integers). The number of images (N) for generating a composite image, and the number of pixels (M) used to calculate a defective pixel become equal. That is to say, when a total of N nondestructive read images are used for generating a composite image, the number of pixels used for defective pixel correction is N.
In the image sensing region of the FPD with an X-ray dosage that is less than the reference value, the number of pixels used to calculate defective pixels can be set according to equation 10. If the number of pixels is set according to the results of equation 10, which uses X-ray dosage Q around a defective pixel, system noise variance P of the FPD, and number of images N for generating a composite image (number of readout times), it is possible to perform defective pixel correction that has a stable statistical amount.
The image acquisition unit 54 has a similar configuration to that in
According to the present embodiment, defective pixel correction that has a stable statistical amount can be performed due to the setting of the number of peripheral pixels M to be used for defective pixel correction, and the number of nondestructive read images N to be used for composite image generation as variables in accordance with imaging conditions.
According to the embodiments described above, defective pixels can be corrected so that the statistical properties of peripheral pixels and the statistical properties of defective pixels are the same.
Aspects of the present invention can also be realized by a computer of a system or apparatus (or apparatuses such as a CPU or MPU) that reads out and executes a program recorded on a memory apparatus to perform the functions of the above-described embodiments, and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory apparatus to perform the functions of the above-described embodiments. For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (e.g., computer-readable medium).
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2012-128404, filed Jun. 5, 2012, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2012-128404 | Jun 2012 | JP | national |