Patent Application
20110304752
This application claims priority under 35 U.S.C. §119(a) to a Korean Patent Application filed in the Korean Intellectual Property Office on Jun. 11, 2010 and assigned Serial No. 10-2010-0055603, the entire disclosure of which is incorporated herein by reference.
1. Field of the Invention
The present invention relates generally to an apparatus and method for creating a lens shading compensation table, and more particularly, to an apparatus and method for creating a lens shading compensation table suitable for various photography environments.
2. Description of the Related Art
A typical camera module includes an image sensor, a band-pass filter, and a lens system.
With the trend toward miniaturization of the camera module, the diameter of a lens decreases and the Chief Ray Angle (CRA) of the lens increases. Accordingly, the brightness of the edge of an image decreases as compared to the brightness of the center of the image, which is referred to as lens shading. The lens shading deepens with an increase in the sensor resolution and also deepens as the size of an incidence region decreases for increasing the depth.
The lens shading is compensated by changing an amplification factor according to the position of a sensor. A two-dimensional (2D) grid lookup table representing amplification factors to be compensated to reduce the lens shading is called a lens shading compensation table.
For example, if a lens shading compensation table for a 5M pixel image sensor with a resolution of 2592×1944 pixels includes a 42×32 grid 2D lookup table, a grid cell of the lens shading compensation table becomes a square with a size of 64×64 pixels. The lens shading compensation table is used to perform a lens shading compensation operation by various interpolations. A smoother interpolation is used if the grid cell of the lens shading compensation table is small in size.
The lens shading compensation uses a small number of lens shading compensation tables as compared to the image resolution. Therefore, the pixels between grids are interpolated by various interpolations, such as bilinear interpolation and B-spline interpolation, to prevent the occurrence of blocks.
The band-pass filter of the camera module passes a visible ray wavelength band and interrupts the remaining unnecessary wavelength bands. However, for the wavelength transmittance of the band-pass filter, the cutoff wavelength of a long-wavelength region moves by 20 to 30 nm as the incidence angle increases. Thus, a lens shading form changes according to the spectrum of an incident light source.
Because a lens shading compensation table for each RGB channel must change according to a change in the lens shading form, a plurality of lens shading compensation tables are created to perform lens shading compensation.
Also, a memory is used to store the created lens shading compensation tables, thus increasing the device price. Moreover, lens shading compensation tables must be created for various light sources, thus reducing the mass production efficiency.
The present invention has been made to address at least the above problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the present invention provides an apparatus and method for creating a lens shading compensation table suitable for various photography conditions.
According to an aspect of the present invention, an apparatus is provided for creating a lens shading compensation table suitable for a photography environment. The apparatus includes a photography environment condition determining unit for determining photography environment conditions of an image input from a camera module, and a compensation parameter detecting unit for detecting compensation parameters corresponding to the determined photography environment conditions. The apparatus also includes a compensation table creating unit for creating a lens shading compensation table suitable for the determined photography environment conditions by applying the detected compensation parameters to a predetermined reference lens shading compensation table.
According to another aspect of the present invention, a method is provided for creating a lens shading compensation table suitable for a photography environment. Photography environment conditions of an image input from a camera module are determined. Compensation parameters corresponding to the determined photography environment conditions are detected. A lens shading compensation table suitable for the determined photography environment conditions is created by applying the detected compensation parameters to a predetermined reference lens shading compensation table.
The above and other aspects, features and advantages of the present invention will be more apparent from the following description when taken in conjunction with the accompanying drawings, in which:
Embodiments of the present invention are described in detail below with reference to the accompanying drawings. The same or similar components may be designated by the same or similar reference numerals. Detailed descriptions of constructions or processes known in the art may be omitted to avoid obscuring the subject matter of the present invention.
Referring to
The camera module 100 processes optical signals, inputted through a lens, to output a Bayer image. The Bayer image is image data outputted through a Bayer color filter array of the camera module 100.
The image signal processing unit 110 detects photography environment conditions of the inputted Bayer image and creates a lens shading compensation table suitable for the detected photography environment conditions. The photography environment conditions may be data or information.
Specifically, the photography environment condition determining unit 111 detects the photography environment conditions of the inputted Bayer image by using detailed information about a light source for image photography. The detailed information about the light source may include at least one of time information, automatic exposure information, and automatic white balance information.
The compensation parameter detecting unit 112 detects compensation parameters for lens shading compensation by using the photography environment conditions detected by the photography environment condition determining unit 111. The compensation parameter detecting unit 112 predetermines a compensation parameter table including compensation parameter values of Red/Green/Blue (RGB) channels for lens shading compensation according to a plurality of photography environment conditions. The compensation parameter detecting unit 112 detects compensation parameters of RGB channels, corresponding to the detected photography environment conditions, from the predetermined compensation parameter table.
The compensation table creating unit 113 creates a lens shading compensation table suitable for a photography environment by applying the detected compensation parameters to a reference lens shading compensation table predetermined for lens shading compensation.
The memory unit 120 stores the reference lens shading compensation table predetermined for lens shading compensation.
The compensation unit 130 compensates the inputted image by using the lens shading compensation table created by the compensation table creating unit 113.
The display unit 140 displays the image compensated by the compensation unit 130.
Embodiments of the present invention create a lens shading compensation table reflecting various photography environments, thereby making it possible to create a lens shading compensation table suitable for various photography environments, without the need to prestore a plurality of lens shading compensation tables.
Referring to
In step 210, the photography environment condition determining unit 111 determines photography environment conditions of the Bayer image by using detailed light source information of the Bayer image. The photography environment conditions may vary according to a light source difference between indoor and outdoor places, an earth incidence angle of a solar light source, a seasonal subject variation, a spectrum variation according to season and time, a spectrum variation according to weather, and a difference in indoor illumination such as fluorescent lighting and incandescent lighting.
A process for determining the various photography environment conditions is described in greater detail below with reference to
Referring to
The automatic exposure value may be calculated in various ways. For example, the camera may calculate a brightness value of a photography environment by using an exposure time, an analog gain, and a digital gain. In an embodiment of the present invention, the automatic exposure value may be calculated as “Automatic Exposure Value=Exposure Time×Analog Gain×Digital Gain”. In this case, if the photography environment is bright, the automatic exposure value is small; and if the photography environment is dark, the automatic exposure value is large.
In another embodiment of the present invention, the automatic exposure value may be calculated as “Automatic Exposure Value=1/(Exposure Time×Analog Gain×Digital Gain)”. In this case, if the photography environment is bright, the automatic exposure value is large; and if the photography environment is dark, the automatic exposure value is small.
In an embodiment of the present invention, it is assumed that the automatic exposure value is larger than the automatic exposure threshold value if the photography environment is bright. If the automatic exposure value is smaller than the automatic exposure threshold value, the photography environment condition determining unit 111 determines the photography environment to be an indoor environment. If the automatic exposure value is larger than the automatic exposure threshold value, the photography environment condition determining unit 111 determines the photography environment to be an outdoor environment.
In step 320, the photography environment condition determining unit 111 determines whether the photography date is between April and October. If the photography date is between April and October, the photography environment condition determining unit 111 proceeds to step 350. If the photography date is not between April and October, the photography environment condition determining unit 111 proceeds to step 330 of
In step 350, the photography environment condition determining unit 111 determines whether the photography time is between 10 A.M. and 4 P.M. If the photography time is between 10 A.M. and 4 P.M., the photography environment condition determining unit 111 proceeds to step 360. If the photography time is not between 10 A.M. and 4 P.M., the photography environment condition determining unit 111 determines the photography environment condition to be a third photography environment condition in step 370. The photography time is determined because the spectral distribution of light transmitted from the solar light source to the earth's surface varies as the altitude of the sun varies with time.
In step 360, the photography environment condition determining unit 111 uses automatic white balance information to determine the color temperature value of a light source. If the color temperature value of the light source is larger than a predetermined color temperature threshold value, the photography environment condition determining unit 111 determines the photography environment condition to be a first photography environment condition with a high color temperature, in step 380. If the color temperature value of the light source is smaller than the predetermined color temperature threshold value, the photography environment condition determining unit 111 determines the photography environment condition to be a second photography environment condition with a low color temperature, in step 390. If the photography environment condition is determined to be a low color temperature, the photography environment condition determining unit 111 determines that the photography weather is clear. If the photography environment condition is determined to be a high color temperature, the photography environment condition determining unit 111 determines that the photography weather is cloudy. For example, if the color temperature ranges from about 2800 K to about 9000 K and the color temperature threshold value is about 5000 K, the color temperature of a cloudy day corresponds to about 6000 K and it may be determined to be a high color temperature. Also, the color temperature of sunset corresponds to about 4000 K and it may be determined to be a low color temperature.
In step 310 of
In step 330 of
If the photography time is between 11 A.M. and 3 P.M., the photography environment condition determining unit 111 proceeds to step 340. If the photography time is not between 11A.M and 3 P.M., the photography environment condition determining unit 111 determines the photography environment condition to be a sixth photography environment condition, in step 420.
In step 340, the photography environment condition determining unit 111 determines the color temperature value of a light source. If the color temperature value of the light source is larger than a predetermined color temperature threshold value, the photography environment condition determining unit 111 determines the photography environment condition to be a fourth photography environment condition in step 430. If the color temperature value of the light source is smaller than the predetermined color temperature threshold value, the photography environment condition determining unit 111 determines the photography environment condition to be a fifth photography environment condition, in step 440.
Embodiments of the present invention detect photography environment conditions varying according to a light source difference between indoor and outdoor places, thus making it possible to create a lens shading compensation table suitable for the detected photography environment conditions.
In another embodiment of the present invention, the compensation parameter detecting unit 112 may perform steps 310, 340 and 360 of determining the color temperature by using the automatic white balance information.
Referring again to
Specifically, the compensation parameter detecting unit 112 predetermines a compensation parameter table including compensation parameters of RGB channels by using a normalized R channel gain (nrm_r) and a normalized B channel gain (nrm_b), specifically, automatic white balance information according to the photography environment conditions. The color temperature of a light source may be determined using the normalized R channel gain and the normalized B channel gain. For example, at low color temperatures corresponding to long-wavelength regions in a light source color temperature curve, the normalized R channel gain has large values and the normalized B channel gain has small values. At high color temperatures corresponding to short-wavelength regions in the light source color temperature curve, the normalized R channel gain has small values and the normalized B channel gain has large values.
The compensation parameter detecting unit 112 searches for the nrm_r and nrm_b corresponding to the photography environment conditions detected from the predetermined compensation parameter table by the photography environment condition determining unit 111.
The compensation parameter detecting unit 112 detects compensation parameters of RGB channels corresponding to the searched nrm_r and nrm_b.
For example, if the nrm_r of automatic white balance information for Bayer image photography is approximately 0.275, the compensation parameter detecting unit 112 searches the predetermined compensation parameter table for channel-by-channel compensation parameters corresponding to a channel-by-channel nrm_r value of about 0.275. If there is no nrm_r value in the compensation parameter table, the compensation parameter detecting unit 112 detects compensation parameters by linear interpolation.
The compensation parameter detecting unit 112 calculates compensation parameters for each channel corresponding to an nrm_r value of about 0.275. Herein, the R channel compensation parameter is calculated at about 0.975. Also, the G channel compensation parameter is calculated at about 0.99, and the B channel compensation parameter is calculated at about 1.025.
As illustrated in
For example, if a range of the normalized R channel gains and B channel gains corresponds to a reference numeral 500, the compensation parameter detecting unit 112 detects RGB compensation parameters corresponding to the normalized R channel gains and the normalized B channel gains, and creates a compensation parameter table.
Thereafter, the compensation parameter detecting unit 112 detects RGB compensation parameters corresponding to the detected photography environment conditions, from the created compensation parameter table.
Referring again to
Step 230 is described in greater detail below with reference to
The compensation table creating unit 113 creates a lens shading compensation table optimized for the photography environment conditions, by multiplying the RGB compensation parameter values and the RGB channel table values of the reference lens shading compensation table so that the brightness is uniform in the horizontal direction of an image. As illustrated in
The compensation unit 130 compensates the inputted image by using the lens shading compensation table created by the compensation table generating unit 113.
Referring again to
Through this process, a lens shading compensation table is created for lens shading compensation in each frame.
As described above, the lens shading compensation table creating apparatus according to the present invention detects the photography environment conditions of an image photographed by the user, detects the compensation parameters corresponding to the detected photography environment conditions, applies the detected compensation parameters to the reference lens shading compensation table values, and creates a lens shading compensation table optimized for the photography environment conditions, thus making it possible to obtain the optimum images in various environments.
While the invention has been shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2010-0055603 | Jun 2010 | KR | national |
