The present invention relates to the field of imaging. More specifically, the present invention relates to defocus estimation.
When images are acquired by a device such as a digital camera, there are often defects with the image such as blurring or defocusing. While there are several methods for attempting to correct blur, they do not sufficiently correct blur and defocusing.
A defocus estimation algorithm is described herein. The defocus estimation algorithm utilizes a single image. A Laplacian of Gaussian approximation is determined by computing a difference of Gaussian. Defocus blur is able to be estimated by computing a blur difference between two images using the difference of Gaussian.
In one aspect, a method programmed in a non-transitory memory of a device comprises acquiring an image, computing a Laplacian of a Gaussian of the image by computing a difference of Gaussian of the image and determining an estimated defocus amount of the image based on the difference of Gaussian of the image. The estimated defocus amount is determined using iterative convolution. The estimated defocus amount is determined using a lookup table. The method further comprises detecting a focus of the image. Detecting the focus of the image comprises comparing an iteration number with a predefined threshold. The image is in focus when the iteration number is smaller than the threshold. The method further comprises denoising the image. The device comprises a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, a smart phone, a portable music player, a tablet computer, a mobile device, a video player, a video disc writer/player, a high definition disc writer/player, an ultra high definition disc writer/player), a television, a home entertainment system, or a smart watch.
In another aspect, a system programmed in a memory of a device comprises an acquisition module configured for acquiring an image, a computing module configured for computing a Laplacian of a Gaussian of the image by computing a difference of Gaussian of the image and a determination module configured for determining an estimated defocus amount of the image based on the difference of Gaussian of the image. The estimated defocus amount is determined using iterative convolution. The estimated defocus amount is determined using a lookup table. The system further comprises detecting a focus of the image. Detecting the focus of the image comprises comparing an iteration number with a predefined threshold. The image is in focus when the iteration number is smaller than the threshold. The system further comprises denoising the image.
In another aspect, an apparatus comprises a non-transitory memory for storing an application, the application for: acquiring an image, computing a Laplacian of a Gaussian of the image by computing a difference of Gaussian of the image and determining an estimated defocus amount of the image based on the difference of Gaussian of the image and a processing component coupled to the memory, the processing component configured for processing the application. The estimated defocus amount is determined using iterative convolution. The estimated defocus amount is determined using a lookup table. The apparatus further comprises detecting a focus of the image. Detecting the focus of the image comprises comparing an iteration number with a predefined threshold. The image is in focus when the iteration number is smaller than the threshold. The apparatus further comprises denoising the image.
In another aspect, a camera device comprises a sensor for acquiring an image, a non-transitory memory for storing an application, the application for: computing a Laplacian of a Gaussian of the image by computing a difference of Gaussian of the image and determining an estimated defocus amount of the image based on the difference of Gaussian of the image and a processing component coupled to the memory, the processing component configured for processing the application. The estimated defocus amount is determined using iterative convolution. The estimated defocus amount is determined using a lookup table. The application is configured for detecting a focus of the lens of the image. Detecting the focus of the image comprises comparing an iteration number with a predefined threshold. The image is in focus when the iteration number is smaller than the threshold. The application is configured for denoising the image.
An approximation between a Laplacian of Gaussian (LoG) and Difference of Gaussian (DoG) to estimate defocus using a single image is described herein. The second derivative of a Gaussian function is the Laplacian of the Gaussian. A negative second derivative of the Gaussian function is shown in
Gaussian function:
A heat diffusion equation is:
By choosing σ=√{square root over (σ1σ2)}, this gives a perfect match at x=0.
In a further approximation, the DoG is:
A model for defocus blur is given by:
b=fG(x,σ2)
The goal is to estimate σ from b.
ba=bG(x,σa2)=fG(x,σ2+σa2)
Reblur: bb=bG(x,σb2)=fG(x,σ2+σb2)
σb>σa
The blur difference approximation is computed by:
Consider blur difference:
b−ba=f(G(x,σ2)−G(x,σ2+σa2))
similarly,
When σb>σa, there exists σd so that
[−G″(x,√{square root over (σ2(σ2+σa2)))}]G(x,σd2)=[−G″(x,√{square root over (σ2(σ2+σb2)))}]
where
σd2=√{square root over (σ2(σ2+σb2))}−√{square root over (σ2(σ2+σa2))}
Once σd is determined, σ is able to solved from the equation.
Implementation
To determine σd, it is equivalent to finding a Gaussian function G so that:
or, σd is found to minimize:
This is able to performed using the iterative convolution method:
where k is a Gaussian kernel with a small variance.
Denoising
Noise is able to affect the blur matching performance, particularly when the amount of blur increases. Performance is able to be improved by denoising, which is done by applying a blur filter to the captured image before iterative convolution. The same amount of denoising is applied to the calibration image and test images.
Focus Detection
Focus detection is used to determine whether the image is in focus using the single image. The amount of blur is not required for this application. Focus detection is performed by comparing the iteration number with a predefined threshold. The image is in focus when the iteration number is smaller than the threshold. Otherwise, it is out of focus. Denoising is turned off when only focus detection is required, since denoising makes it difficult to distinguish small blur and in-focus. Small σb/σa ratios are preferable for focus detection.
In some embodiments, the estimating defocus application(s) 630 include several applications and/or modules. In some embodiments, modules include one or more sub-modules as well. In some embodiments, fewer or additional modules are able to be included.
Examples of suitable computing devices include a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, a smart phone, a portable music player, a tablet computer, a mobile device, a video player, a video disc writer/player (e.g., DVD writer/player, high definition disc writer/player, ultra high definition disc writer/player), a television, a home entertainment system, smart jewelry (e.g., smart watch) or any other suitable computing device.
To utilize the estimating defocus method, a device such as a digital camera is able to be used to acquire an image. The estimating defocus method is automatically used when performing image processing. The estimating defocus method is able to be implemented automatically without user involvement.
In operation, the estimating defocus method improves image processing efficiency. By using a single image and a LoG and DoG, efficiency is improved.
Some Embodiments of Defocus Estimation from Single Image Based on Laplacian of Gaussian Approximation
The present invention has been described in terms of specific embodiments incorporating details to facilitate the understanding of principles of construction and operation of the invention. Such reference herein to specific embodiments and details thereof is not intended to limit the scope of the claims appended hereto. It will be readily apparent to one skilled in the art that other various modifications may be made in the embodiment chosen for illustration without departing from the spirit and scope of the invention as defined by the claims.
Number | Name | Date | Kind |
---|---|---|---|
4905296 | Nishihara | Feb 1990 | A |
5899999 | De Bonet | May 1999 | A |
7715018 | Gharib et al. | May 2010 | B2 |
8368964 | Xu et al. | Feb 2013 | B2 |
8867856 | Wang et al. | Oct 2014 | B2 |
20090245651 | Friedhoff | Oct 2009 | A1 |
20090268985 | Wong | Oct 2009 | A1 |
20100066874 | Ishiga | Mar 2010 | A1 |
20100080482 | Wong et al. | Apr 2010 | A1 |
20110150447 | Li | Jun 2011 | A1 |
20110249173 | Li | Oct 2011 | A1 |
20120243792 | Kostyukov | Sep 2012 | A1 |
20130182152 | Li | Jul 2013 | A1 |
20130258096 | Ali | Oct 2013 | A1 |
20130300860 | Komatsu | Nov 2013 | A1 |
20150154742 | Bocharov | Jun 2015 | A1 |
Number | Date | Country |
---|---|---|
2009156329 | Dec 2009 | WO |
Entry |
---|
Emanuela Boros et al., “UAIC:Participation in ImageCLEF2009 Robot Vision Task”, http://www.imageclef.org/. |
Mohd Ahmed et al., “Criminal Photograph Retrieval Based on Forensic Face Sketch Using Scale Invariant Feature Transform”, Mar. 26-28, 2012, http://www.icmis.net/ictbm/ictbm12/ICTBM12CD/pdf/D2017-DONE.PDF., www.evalueserve.com. |
Attached please find the International Search Report for the PCT/US2016/045841 application. |
Jing Wan et al., “An Iris Image Quality Assessment Method Based on Laplacian of Gaussian Operation”, MVA2007, LAPR Conference on Machine Vision Applications, May 16-18, 2007, Tokyo, Japan, pp. 248-251. |
Bolan Su et al., “Blurred Image Region Detection and Classification”, Proceeding of the 19th ACM International Conference on Multimedia ; 1397-1400, 2011. |
Number | Date | Country | |
---|---|---|---|
20170053187 A1 | Feb 2017 | US |