INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, LEARNING APPARATUS AND LEARNING METHOD AND PROGRAM

Abstract

An information processing apparatus includes an image taking means for taking images of a subject, a class-classification means for classifying a first image outputted by the image taking means into a class according to a characteristics thereof, a storage means for storing plural coefficient memories having different image taking conditions at the time of image taking, which store prediction coefficients according to the class acquired by learning, a designation means for designating one coefficient memory from among the plural coefficient memories based on the image taking condition when the image was taken by the image taking means, and a calculation means for calculating a second image in which noise is removed from the first image by calculating a prediction coefficient of the class of the first image, which is in the designated coefficient memory.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a graph showing the relation between exposure time and the noise amount;

FIG. 2 is a graph showing the relation between luminance when taking images and an S/N ratio;

FIG. 3 is a block diagram showing a configuration example of an embodiment of an imaging apparatus to which the invention is applied;

FIG. 4 is a block diagram showing the detailed configuration example of a class-classification adaptive processing unit of FIG. 3;

FIGS. 5A and 5B are views showing tap structure examples of a prediction tap and a class tap;

FIG. 6 is a flowchart explaining the noise removal processing of the imaging apparatus of FIG. 3;

FIG. 7 is a block diagram showing a configuration example of a learning apparatus which calculates a prediction coefficient;

FIG. 8 is a block diagram showing the detailed configuration example of a learning data generating unit of FIG. 7;

FIG. 9 is a block diagram showing the detailed configuration example of a noise image generating unit of FIG. 8;

FIG. 10 is a view explaining generation of learning data;

FIGS. 11A and 11B are views explaining generation of learning data;

FIG. 12 is a view explaining generation of learning data;

FIG. 13 is a view explaining generation of learning data;

FIG. 14 is a flowchart explaining learning processing of the learning apparatus of FIG. 7;

FIG. 15 is a flowchart explaining learning data generating processing of step S31 of FIG. 14;

FIGS. 16A and 16B are views explaining the concept of a detection method detecting a defective pixel;

FIGS. 17A and 17B are views explaining the concept of a detection method detecting a defective pixel;

FIG. 18 is a block diagram showing a configuration example of a defective pixel detection system;

FIG. 19 is a block diagram showing a configuration example of a second embodiment of an imaging apparatus;

FIG. 20 is a flowchart explaining noise removal processing of the imaging apparatus of FIG. 19;

FIG. 21 is a block diagram showing a configuration example of a third embodiment of an imaging apparatus;

FIG. 22 is a block diagram explaining a configuration example of a chip in the imaging apparatus of FIG. 3;

FIG. 23 is a block diagram explaining a configuration example of a chip in the imaging apparatus of FIG. 3;

FIG. 24 is a block diagram explaining a configuration example of a chip in the imaging apparatus of FIG. 3;

FIG. 25 is a block diagram explaining a configuration example of a chip in the imaging apparatus of FIG. 3; and

FIG. 26 is a block diagram showing a configuration example of an embodiment of a computer to which the invention is applied.

Claims

1. An information processing apparatus, comprising: an image taking means for taking images of a subject;a class-classification means for classifying a first image outputted by the image taking means into a class according to a characteristic thereof;a storage means for storing plural coefficient memories having different image taking conditions at the time of image taking, which store prediction coefficients according to the class acquired by learning,a designation means for designating one coefficient memory from among the plural coefficient memories based on the image taking condition when the image was taken by the image taking means, anda calculation means for calculating a second image in which noise is removed from the first image by calculating using a prediction coefficient of the class of the first image, which is in the designated coefficient memory.

2. The information processing apparatus according to claim 1, wherein the image taking condition is luminance or exposure time at the time of image taking.

3. The information processing apparatus according to claim 1, wherein the storage means further stores coefficient memories according to the image taking condition at the time of image taking and unique information unique to the image taking means, andwherein the designated means designates one coefficient memory from among the stored plural coefficient memories based on the image taking condition and the unique information.

4. The information processing apparatus according to claim 1, wherein the storage means further stores coefficient memories according to the image taking condition and defect position information indicating a position of a defective pixel in pixels included in the image taking means,wherein the designation means designates one coefficient memory from among the stored plural coefficient memories based on the image taking condition and the defect position information andwherein the calculation means corrects the defective pixels and calculates the second image in which noise is removed from the first image.

5. The information processing apparatus according to claim 1, further comprising: a defective pixel detection means for detecting a defective pixel of the image taking means and outputting defect position information; anda storage means for storing the defect position information.

6. The information processing apparatus according to claim 5, wherein the defective pixel detection means detects the defective pixel based on whether corresponding respective pixels of two images taken at different times have the same pixel value or not.

7. The information processing apparatus according to claim 1, wherein all components of the image taking means, the class-classification means, the storage means, the designation means, and the calculation means, or a part of the above components including at least the image taking means are configured by a chip.

8. A information processing method, comprising the steps of: classifying a first image obtained by taking images of a subject into a class according to a characteristic thereof;designating one coefficient memory from among plural coefficient memories having different image taking conditions at the time of image taking, which store prediction coefficients according the class obtained by learning, based on the image taking condition when the first image was obtained; andcalculating a second image in which noise is removed from the first image by calculating using a prediction coefficient of the class of the first image, which is in the designated coefficient memory.

9. A program allowing a computer to execute the steps of: classifying a first image obtained by taking images of a subject into a class according to a characteristic thereof;designating one coefficient memory from among plural coefficient memories having different image taking conditions at the time of image taking, which store prediction coefficients according the class obtained by learning based on the image taking condition when the first image was obtained; andcalculating a second image in which noise is removed from the first image by calculating using a prediction coefficient of the class of the first image, which is in the designated coefficient memory.

10. A learning apparatus for learning a prediction coefficient used when performing noise removal processing of an taken image which is the taken image, comprising: a condition decision means for deciding an image taking condition;an image taking means for taking images of a subject under the decided image taking condition;a noise image generating means for generating noise images in which noise included in images taken by the image taking means is extracted;a teacher image generating means for generating a teacher image to be a target image after the noise removal processing;a student image generating means for generating a student image corresponding to the taken image before the noise removal processing is performed by adding the noise image to the teacher image; anda storage means for storing data of pairs of the teacher image and the student image according to different image taking conditions.

11. The learning apparatus according to claim 10, further comprising: an extraction means for extracting plural pixels used for calculating a focused pixel which is a pixel of the teacher image from the student image with respect to respective image taking conditions; anda calculation means for calculating a prediction coefficient which allows a prediction error of the focused pixel calculated by using the prediction coefficient to be statistically minimum based on the extracted plural pixels.

12. The learning apparatus according to claim 10, wherein the noise image generating means generates plural noise images by calculating the difference between an image which is an average value of plural images taken by the image taking means and respective plural images taken by the image taking means.

13. A learning method for learning a prediction coefficient used for performing noise removal processing of a taken image which is the taken image, comprising the steps of: deciding the image taking condition;taking images of a subject under the decided image taking condition;generating a noise image in which noise included in the taken image is extracted;generating a teacher image to be a target image after noise removal processing;generating a student image corresponding to the taken image before the noise removal processing is performed by adding the noise image to the teacher image; andstoring data of pairs of the teacher image and the student image according to different image taking conditions.

14. A program allowing a computer to execute processing of learning a prediction coefficient used for performing noise removal processing of a taken image which is the taken image, comprising the steps of: deciding the image taking condition;taking images of a subject under the decided image taking condition;generating a noise image in which noise included in the taken image is extracted;generating a teacher image to be a target image after noise removal processing;generating a student image corresponding to the taken image before the noise removal processing is performed by adding the noise image to the teacher image; andstoring data of pairs of the teacher image and the student image according to different image taking conditions.

15. An information processing apparatus, comprising: an image taking unit configured to take images of a subject;a class-classification unit configured to classify a first image outputted by the image taking unit into a class according to a characteristic thereof;a storage unit configured to store plural coefficient memories having different image taking conditions at the time of image taking, which store prediction coefficients according to the class acquired by learning,a designation unit configured to designate one coefficient memory from among the plural coefficient memories based on the image taking condition when the image was taken by the image taking unit, anda calculation unit configured to calculate a second image in which noise is removed from the first image by calculating using a prediction coefficient of the class of the first image, which is in the designated coefficient memory.

16. A learning apparatus for learning a prediction coefficient used when performing noise removal processing of an taken image which is the taken image, comprising: a condition decision unit configured to decide an image taking condition;an image taking unit configured to take images of a subject under the decided image taking condition;a noise image generating unit configured to generate noise images in which noise included in images taken by the image taking unit is extracted;a teacher image generating unit configured to generate a teacher image to be a target image after the noise removal processing;a student image generating unit configured to generate a student image corresponding to the taken image before the noise removal processing is performed by adding the noise image to the teacher image; anda storage unit configured to store data of pairs of the teacher image and the student image according to different image taking conditions.