This application is a National Stage of International Application No. PCT/KR2014/003144, filed Apr. 11, 2014, and claims priority from Korean Patent Application No. 10-2013-0060600 filed May 28, 2013, the disclosures of which are incorporated herein in their entirety by reference.
The present invention relates to processing an image with a video quality degraded due to a haze included therein, and improving a video quality thereof.
In general, the quality of an image captured outdoors is degraded by a turbid medium such as atmospheric particles and trickles. In particular, the video quality of an image is influenced by weather, which influences the identification of the original color and shape of an object included in the image.
In particular, since haze-induced video quality degradation occurs throughout the image and is maintained relatively uniformly and continuously, various techniques are proposed to remove the haze-induced video quality degradation.
Hazy image improving methods are classified into a model-based method using an atmospheric physical model and a nonmodel-based method without using an atmospheric physical model.
Unlike the model-based method, the nonmodel-based method does not estimate an atmospheric parameter and may be simply implemented. However, the nonmodel-based method has a problem of distorting a color fidelity and a haze removing effect. Therefore, recently, it is common to improve a hazy image by using the model-based method using a piece of image.
On the other hand, since the model-based method uses an atmospheric physical model, the model-based method may effectively improve a color tone and a contrast degraded by scattered light. However, the model-based method has difficulty in accurately estimating a parameter and has a drawback of taking a long operation time.
Recently, as an example of the model-based method for improving a hazy image, there is a method of using a dark channel prior (DCP) in a piece of hazy image. However, in the case of a DCP method, since the color information feature of an RGB image is estimated on a patch basis, an operation time increases. Also, when an image is generally white or dark, a distortion occurs in the image due to the misestimation of a transmission.
Also, in the case of other model-based methods for improving a hazy image, since there is no image texture information compensation and brightness correction due to the subtraction of scattered light, there is a problem in that the restored image has a weak texture and low brightness.
An exemplary embodiment of the present invention improves the problem of the DCP method and restores/improves haze-induced image color damage and contrast degradation by using a single image. Also, the present invention provides a method for improving an image texture and solving a brightness degradation problem occurring in the conventional hazy image improving process.
The present invention provides a method for removing a haze in a single image. In the present invention, a transmission is estimated by using a dark channel prior obtained from a hazy input image. The estimated transmission includes a block artifact. In an exemplary embodiment of the present invention, in order to preserve an edge and remove the block artifact, a refined transmission value is obtained by performing weighted least square (WLS) filtering by using an estimated transmission value and a morphologically-processed input image, the image is restored based on the refined transmission value, and then multi-scale tone manipulation image processing is performed.
The present invention has an advantageous effect of solving a problem occurring in a hazy image processing process by using a dark channel prior (DCP) in a hazy image. For example, the present invention has an advantageous effect of solving a block artifact problem occurring in an image restored through a transmission obtained by estimating the color information feature of an RGB image on a patch basis.
Also, the present invention has an advantageous effect of improving the texture and brightness of a restored image by performing image texture information compensation and brightness correction due to the subtraction of scattered light in a hazy image improving process.
According to an exemplary embodiment of the present invention, a device for removing a haze in a single image includes: an image analyzing unit configured to generate a dark channel prior representing a distance-dependent haze density in an input image including a haze component; a transmission estimating unit configured to calculate an estimated transmission by estimating an amount of light reaching a photographing device based on the dark channel prior; a transmission refining unit configured to calculate a refined transmission by applying weighted least square (WLS) filtering based on the estimated transmission and morphologically-processed input image information, wherein the refined transmission represents information obtained by preserving edge information and removing a block artifact in the estimated transmission; and an image restoring unit configured to generate a restored image by removing the haze component in the input image based on the refined transmission.
For example, the device may further include an image processing unit configured to perform image processing for improvement of a local contrast of the restored image by dividing the restored image into a base layer, a medium layer, and a fine layer.
According to another exemplary embodiment of the present invention, a device for removing a haze in a single image includes: a transmission estimating unit configured to calculate a first transmission by estimating a transmission value based on a dark channel prior generated from a hazy input image; a transmission refining unit configured to calculate a second transmission for minimizing a block artifact generated in an image restoration process based on the first transmission by applying WLS filtering based on the first transmission and morphologically-processed input image information; an image restoring unit configured to generate a restored image by removing a haze component in the hazy input image based on the second transmission; and an image processing unit configured to perform multi-scale tone manipulation image processing on the restored image.
According to another exemplary embodiment of the present invention, a method for removing a haze in a single image includes: generating, by an image analyzing unit, a dark channel prior representing a distance-dependent haze density in an input image including a haze component; calculating, by a transmission estimating unit, an estimated transmission by estimating an amount of light reaching a photographing device based on the dark channel prior; calculating, by a transmission refining unit, a refined transmission by applying WLS filtering based on the estimated transmission and morphologically-processed input image information, wherein the refined transmission represents information obtained by preserving edge information and removing a block artifact in the estimated transmission; and generating, by an image restoring unit, a restored image by removing the haze component in the input image based on the refined transmission.
According to another exemplary embodiment of the present invention, a method for removing a haze in a single image includes: calculating, by a transmission estimating unit, a first transmission by estimating a transmission value based on a dark channel prior generated from a hazy input image; calculating, by a transmission refining unit, a second transmission for minimizing a block artifact generated in an image restoration process based on the first transmission by applying WLS filtering based on the first transmission and morphologically-processed input image information; generating, by an image restoring unit, a restored image by removing a haze component in the hazy input image based on the second transmission; and performing, by an image processing unit, multi-scale tone manipulation image processing on the restored image.
A method for referring to an elementary stream located in an SAF session according to the present invention for achieving the above-described object will become apparent and more readily appreciated from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In general, a Koschmider model such as Equation 1 is widely used to represent an image including a haze.
I(p)=J(p)r(p)+A(1−r(p)) [Equation 1]
In Equation 1, I(p) denotes an input image including a haze, J(p) denotes a scene radiance representing a haze-free state, A denotes a global atmospheric light source, and r(p) denotes a transmission estimated corresponding to each pixel of the input image. The transmission refers to the amount of light reaching a photographing device without dispersion.
The Koschmider model such as Equation 1 is represented for each of the R, G, and B color channels of the image. By using Equation 1, the global atmospheric light source A and the transmission r(p) are calculated from the input image I(p) including a haze, and then the scene radiance J(p) is acquired.
As the transmission r(p) decreases in Equation 1, the input image I(p) becomes a hazier image. Also, as the transmission r(p) increases, a distance to a subject decreases or a haze-induced distortion decreases.
According to an exemplary embodiment of the present invention, a dark channel prior (DCP) is used to estimate the transmission r(p) in Equation 1. However, in a conventional hazy image processing method, since only a DCP is used, there is a problem in that an artifact occurs in a restored image (see 131 in
In order to solve this problem, according to an exemplary embodiment of the present invention, a DCP is used to obtain an estimated transmission (120 in
As for the DCP, as a method for removing a haze in an image, some pixels in a block of a predetermined size around each pixel of a haze-free clean image are implemented such that at least one color channel has a very small near-zero value. This is represented as Equations 2 and 3.
In Equation 2, Jdark denotes a DCP, Jc denotes an input image of each color channel, and Ω denotes a block (or patch) around a P pixel.
When Equation 3 is used, the transmission r(p) in Equation 1 may be modified as Equation 4.
In Equation 4, μ is a constant value of about 0.95.
In the conventional art, a transmission {tilde over (r)}(p) in Equation 4 is estimated on a block-by-block basis (121, 122, 123) (120 in
As a scheme for solving this problem, a soft matting algorithm or the like is used. However, since the soft matting algorithm is complex and has a large operation amount, the soft matting algorithm has another problem of the impossibility of processing due to the lack of memory capacity when the image resolution is high.
An exemplary embodiment of the present invention provides a scheme for solving the problems of a complex operation and an artifact phenomenon occurring in the hazy image processing process.
For this purpose, according to an exemplary embodiment of the present invention, a block artifact may be removed by applying WLS filtering to the transmission {tilde over (r)}(p) obtained in Equation 4, and then brightness control image processing may be performed in a base layer image and image processing for improving a local texture may be performed in a medium layer image and a fine layer image.
Hereinafter, in an exemplary embodiment of the present invention, for prevention of a confusion between the terms, {tilde over (r)}(p) denotes an estimated transmission (see 320 in
Referring to
The image analyzing unit 210 generates a DCP representing a distance-dependent haze density in an input image including a haze component. A method of generating the DCP refers to Equations 2 and 3.
The transmission estimating unit 220 calculates an estimated transmission {tilde over (r)}(p) by applying the DCP generated by the image analyzing unit 210. A method of calculating the estimated transmission refers to Equation 4. In this case, since the estimated transmission {tilde over (r)}(P) is estimated on a block-by-block basis, a block artifact (see 121, 122, and 123 in
The transmission refining unit 230 calculates a refined transmission r(p) by using a morphologically-processed input image (330 in
For example, the transmission refining unit 230 calculates the refined transmission r(p) by removing the block artifact included in the estimated transmission by applying WLS filtering to the estimated transmission value {tilde over (r)}(p) calculated by the transmission estimating unit 220.
The transmission refining unit 230 sets an edge value of the morphologically-processed input image (330 in
Referring to
For this purpose, in the estimated transmission {tilde over (r)}(p), the edge value of a region corresponding to the edge value (e.g., 331 in
In an exemplary embodiment of the present invention, this process is referred to as a WLS filtering process, which uses Equations 5 to 7.
The image restoring unit 240 generates a restored image J(p) by removing the haze component in the input image based on the refined transmission value r(p), as represented in Equation 8.
The image processing unit 250 performs multi-scale tone manipulation image processing on the restored image J(p). As represented in Equation 10, a multi-scale image is generated by the difference between the images smoothed through a WLS filter.
In detail, for improvement of a local contrast of the restored image J(p) obtained in Equation 8, as represented in Equation 9, image processing is performed by dividing the restored image J(p) into a base layer, a medium layer, and a fine layer.
The image processing unit 250 is implemented to perform brightness control image processing in the base layer image and perform image processing for improvement of a local texture in the medium layer image and the fine layer image.
The image restoring unit 240 and the image processing unit 250 will be described later in detail with reference to
In Equation 5, p denotes a position of each pixel, rp denotes a refined transmission (340 in
An output image rp (340 in
Also,
denotes a horizontal-axis and vertical-axis first-order differential operator of the output image rp (340 in
The smoothing weight value is applied to an estimated transmission image 320 of
According to an exemplary embodiment of the present invention, a refined transmission value for minimizing an object function of Equation 5 is obtained. For this purpose, a pixel value difference between the input image (the estimated transmission, {tilde over (r)}(p), and 320 in
In Equation 6, h denotes a morphologically-processed hazy image (330 in
Also, ε denotes a constant value for preventing a denominator from being divided by 0, and may have a value of about 10−4. Also,
denotes the horizontal-axis and vertical-axis first-order differential absolute value of the morphologically-processed hazy image (330 in
r=(I+λ(DxTAxD+DyTAyDy))−1{tilde over (r)} [Equation 7]
In Equation 7, Ax, Ay denotes a matrix of the smoothing weight values wx,p(h), wy,p(h); and Dx, Dy denotes a matrix of
In this case, A may be implemented to use a value of the pixel having the greatest pixel value among the upper 0.1% in the DCP value obtained through Equation 2.
When the refined transmission r(p) is obtained through Equations 5 to 7, a haze-free restored image J(p) may be obtained as Equation 8 according to an exemplary embodiment of the present invention. In this case, A may be implemented to use a value of the pixel having the greatest pixel value among the upper 0.1% in the DCP value obtained through Equation 2. This will be described later with reference to
As the smoothing control constant λ increases in Equation 5, the output image rp (340 in
In
Herein, α is the edge control constant of the morphologically-processed hazy image. As α increases, the other region other than the region corresponding to the edge region 331 of the morphologically-processed hazy image 330 is further smoothed.
After an estimated transmission 520 of an input image 510 is obtained, a refined transmission 530 and a restored image 531 in the case of α=0.01, a refined transmission 540 and a restored image 541 in the case of α=0.1, and a refined transmission 550 and a restored image 551 in the case of α=1 are illustrated. As α increases, the other region other than the region corresponding to the edge region of the morphologically-processed hazy image is further smoothed.
When a refined transmission is obtained from a hazy input image 610, a restored image J(p) 620 is obtained based on Equation 8.
In Equation 8, r0 denotes a constant value for preventing a denominator from being divided by 0, which may be implemented to use a value of about 0.1.
In order to improve the local contrast of the restored image J(p) 620 obtained in Equation 8, the image processing unit (250 in
Vp=B+S(δ,dp1)+S(δ,dp2) [Equation 9]
In Equation 9, Vp denotes an output image, B denotes a base layer image, and S(δ,dp1) and S(δ,dp2) denote a medium layer image and a fine layer image respectively. Also, S( ) denotes a sigmoid function, and δ denotes a boosting factor and represents a reverse transmission. In an exemplary embodiment of the present invention, the reverse transmission may be implemented to use a value modified from 0 to 10. As the boosting factor increases, the texture information survives strongly in the corresponding layer.
In Equation 9, di denotes an image generated in an i-th step, which is obtained as Equation 10.
di=ui-1−ui,i−1,2, . . . ,k (K: Number of Layers) [Equation 10]
In an exemplary embodiment of the present invention, K=2 may be set.
Brightness control image processing such as gamma correction is performed in the base layer image, and texture improvement is performed in the medium layer image and the fine layer image. For this purpose, a difference value between the images smoothed through a WLS filter is used as represented in Equation 10.
In detail, according to an exemplary embodiment of the present invention, in order to improve the texture information and the local contrast of the restored image J(p), image processing is performed to enhance a band-pass signal component in the medium layer and enhance a high-pass signal component in the fine layer.
Referring to
The transmission estimating unit 220 calculates an estimated transmission based on the DCP (S720). A method of calculating the estimated transmission refers to Equation 4. The transmission represents the amount of light reaching a photographing device. The estimated transmission calculated by the transmission estimating unit 220 includes a block artifact because it is estimated on a block-by-block basis in Equation 4.
Thereafter, the transmission refining unit 230 calculates a refined transmission by applying WLS filtering based on the estimated transmission and morphologically-processed input image information (S730). According to an exemplary embodiment of the present invention, the refined transmission represents information obtained by preserving edge information and removing a block artifact in the estimated transmission. For this purpose, the transmission refining unit 230 performs WLS filtering. By using the edge information of the morphologically-processed input image in a WLS filtering process, the edge information of the morphologically-processed input image is preserved in the estimated transmission, and the other region other than the edge region of the morphologically-processed input image is smooth-processed.
Thereafter, the image restoring unit 240 generates a restored image by removing the haze component in the input image based on the refined transmission (S740). In this case, the restored image may be generated by using Equation 1 or 8.
Thereafter, the image processing unit 250 generates a final image by performing multi-scale image manipulation image processing on the restored image (S750 and S760). This refers to Equations 9 and 10.
The present invention may also be embodied as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium may be any data storage device that may store data which may be thereafter read by a computer system. Examples of the computer-readable recording medium may include ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storages. The computer-readable recording medium may also be distributed over network-coupled computer systems so that the computer-readable codes may be stored and executed in a distributed fashion.
The exemplary embodiments have been described above with reference to the drawings. The terms herein are only used to describe the present invention and are not intended to limit the scope of the present invention as defined by the following claims. Therefore, those of ordinary skill in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Thus, the scope of the present invention should be defined by the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
10-2013-0060600 | May 2013 | KR | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/KR2014/003144 | 4/11/2014 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2014/193080 | 12/4/2014 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
20030202713 | Sowa | Oct 2003 | A1 |
20100040300 | Kang et al. | Feb 2010 | A1 |
20100067574 | Knicker | Mar 2010 | A1 |
20100322478 | Nitanda | Dec 2010 | A1 |
20110188775 | Sun | Aug 2011 | A1 |
20110211758 | Joshi | Sep 2011 | A1 |
20120200669 | Lai | Aug 2012 | A1 |
20120236020 | Paris | Sep 2012 | A1 |
20130071043 | Bai | Mar 2013 | A1 |
20130089247 | Mercuriev | Apr 2013 | A1 |
20140003528 | Tourapis | Jan 2014 | A1 |
20140321540 | Leontaris | Oct 2014 | A1 |
Number | Date | Country |
---|---|---|
102750674 | Oct 2012 | CN |
10-2011-0050775 | May 2011 | KR |
10-2012-0079564 | Jul 2012 | KR |
10-2013-0015906 | Feb 2013 | KR |
2013018101 | Feb 2013 | WO |
Entry |
---|
Yu, Jing, and Qingmin Liao. “Fast single image fog removal using edge-preserving smoothing.” Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on. IEEE, 2011. |
Gibson, Kristofor, Dũng Võ, and Truong Nguyen. “An investigation in dehazing compressed images and video.” OCEANS 2010. IEEE, 2010. |
Yu, Jing et al.; “Fast Single Image Fog Removal Using Edge-Preserving Smoothing”; ICASSP; 2011; pp. 1245-1248. |
He, Kaiming et al.; “Single Image Haze Removal Using Dark Channel Prior”; IEEE Transactions on Pattern Analysis and Machine Intelligence; vol. 33; No. 12; Dec. 2011; 14 pages total. |
Search Report dated Jul. 17, 2014 issued by the International Searching Authority in counterpart International Patent Application No. PCT/KR2014/003144 (PCT/ISA/210). |
Written Opinion dated Jul. 17, 2014 issued by the International Searching Authority in counterpart International Patent Application No. PCT/KR2014/003144 (PCT/ISA/237). |
Communication dated Jul. 19, 2017, issued by the State Intellectual Property Office of P.R. China in counterpart Chinese Application No. 201480030119.1. |
Luc Vincent: “Morphological Grayscale Reconstruction in Image Analysis: Applications and Efficient Algorithms” IEEE Transactions on Image Processing, vol. 2, No. 2, Apr. 1993, pp. 176201, (26 pages total). |
Park et al: “Single Image Haze Removal With Wls-Based Edge-Preserving Smoothing Filter”, ICASSP 2013, pp. 2469-2473, (8 pages total). |
Farbman et al:“ Edge-Preserving Decompositions for Multi-Scale Tone and Detail Manipulation”, ACM Transactions on Graphics, vol. 27, No. 3, Article 67, Aug. 2008, pp. 1-10, (10 pages total). |
Number | Date | Country | |
---|---|---|---|
20160063684 A1 | Mar 2016 | US |