Patent Application
20100271536
1. Field of the Invention
The invention relates to autofocusing methods for camera systems, and more particularly to a blended approach of using conventional mechanical autofocus with softlens autofocus techniques to achieve best system-level autofocus in a camera module.
2. Description of the Related Art
As camera sensor pixels get smaller and smaller, the need to accurately resolve them requires more and more of the autofocus (AF) system. Efforts have been underway to improve on mechanical and softlens autofocus (AF) algorithms and methods. Softlens is a software-based auto focus approach for camera systems, popularized originally in telescopic systems and attempted more recently in cell phone camera systems. It combines the use of a (blurring) phase function that is added to the normal lens prescription and subsequent deconvolution processing via software of the captured image to achieve camera system focus with no moving (mechanical) parts. The focus point and depth of field are also variable in the deconvolution software and therefore tunable. Some of the latest efforts in this respect are listed below.
Patents and papers which relate to the present invention are:
U.S. Pat. No. 6,970,789 (Ippolito et al.) describes a method for determining a best initial focal positioning using linear and quadratic regression using a smart-focusing and double loop software autofocus. Also used are a coarse loop, fine loop and parabolic interpolation procedure. The method however relates to samples on slides under a microscope.
“Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera”, Consumer Electronics, IEEE Transactions, May 2003, Volume: 49, Issue: 2, discusses an algorithm which adopts threshold gradient and edge point count techniques and a focus value function, additionally, a relative difference ratio circuit is also proposed.
“Efficient Auto-Focus Algorithm Utilizing Discrete Difference Equation Prediction Model for Digital Still Cameras”, Chih-Ming Chena, Chin-Ming Hongb, Han-Chunc, Institute of Learning Technology National Hualien University of Educationa, Institute of Applied Electronic Technology National Taiwan Normal Universityb, Institute of Mechatronic Technology National Taiwan Normal Universityc, presents an auto-focus algorithm combining the discrete difference equation prediction model (DDEPM) and bisection search method. The algorithm uses coarse search with DDEPM and fine search with the bisection search method.
It should be noted that none of the above-cited examples of the related art provide the advantages of the below described invention.
It is an object of at least one embodiment of the present invention to provide a method for achieving best system-level autofocus in a camera module.
It is another object of the present invention to use conventional mechanical autofocus techniques (and their associated filter algorithms) in conjunction with softlens AF techniques.
It is yet another object of the present invention to have the mechanical autofocus approach get close to the best focus position (thereby relaxing its tolerances) and to then have the softlens autofocus approach take over and complete the fine tuning of the best focus.
It is still another object of the present invention to use the softlens AF approach to buy back some depth of field performance in low f-number optical systems.
It is a further object of the present invention is to thereby allow for simpler low-f-number lenses, and a higher system Signal-to-Noise Ratio (SNR) and system Modulation Transfer Function (MTF).
These and many other objects which have been achieved by a blended approach to achieving best system-level autofocus in a camera module. In this blended approach, conventional mechanical autofocus techniques (and their associated filter algorithms) are used in conjunction with softlens AF techniques. In this manner, the mechanical autofocus approach needs only get close to the best focus position (thereby relaxing its tolerances) and then the softlens autofocus approach takes over and completes the fine tuning of the best focus (thereby relaxing its capabilities requirements). Additionally, the softlens AF approach may be used to buy back some depth of field performance in low f-number optical systems (thereby allowing for simpler low-f-number lenses, higher system SNR and higher system MTF).
These and many other objects and advantages of the present invention will be readily apparent to one skilled in the art to which the invention pertains from a perusal of the claims, the appended drawing, and the following detailed description of the preferred embodiments.
As camera sensor pixels get smaller and smaller, the need to accurately resolve them requires more and more of the autofocus (AF) system. However, in modern camera systems, accurate focus may be beyond the capabilities of the AF mechanics and/or the AF filter algorithm that must decide when best focus is achieved. Furthermore, tip and tilt requirements between the lens and the sensor during camera module assembly become tighter and tighter as pixels get smaller and lens f-numbers get lower to compensate for smaller pixels.
A solution to this increasing problem is to use blended autofocusing by pairing up mechanical AF with software AF (such as the Extended Depth Of Field (EDOF) softlens AF technologies popular today). In this manner, the mechanical AF first goes as far as it can in achieving best focus and then the image is captured. Second, the softlens AF takes over during image processing and completes the focus action.
The advantages of this tandem approach are many:
With these system-level requirements relaxations, it is possible to design rational camera systems with pixels at or below 1.75 um (micron) or when resolutions are above 5 Megapixels (MP) or so. Furthermore, these camera systems can have added functionalities not available in conventional single-approach systems, such as small softlens AF processing code blocks (with attending considerable gate count reduction), compensation for assembly tip/tilt misalignment, faster (perhaps twice or more as fast) mechanical AF algorithms and depth of field tuning of low-artifact images. Tuning is possible here because the strength of the softlens AF algorithm can be varied in software after image capture. Depth of field tuning means the user gets to tweak the parameters of the softlens AF algorithm to achieve a desired look in the image. For example, this is done commonly in mechanical systems by adjusting the lens' f-number to vary the captured depth of field, or by adjusting the lens' focus position to select what part of the image is to be in focus and what part is to be out of focus. Tuning in softlens AF does this during image processing after the picture is taken instead of adjusting the lens before taking the picture.
In a first preferred embodiment of the present invention the sequence to achieve this tandem AF technique is as follows:
Where:
As an example of methodology, consider the case of a 1.4 um pixel used in conjunction with an f/2.0 imaging lens. Ideally, an f/2.0 imaging lens should provide a Full Width Half Maximum (FWHM) optical spot size in the image plane of approximately 1.5 um. However, due to aberrations, the optical spot sizes may range from FWHM of near 1.5 um in the center of the image to several microns in the image perimeter regions. As well, due to mechanical AF actuation and filter discrimination capabilities limitations, the practical resolved spot sizes may be larger than these values. If, for example, the mechanical AF has the capability to resolve the focus down to half of the imaging lens' depth of field (approximately 3 um, this refers to Step #1 above “perform best or acceptable mechanical AF”) then the focused spot sizes may also increase by another factor of 1.5 to 2 times. Overall, this results in a practical mechanical AF capability that provides best-focused spot sizes 2 to 5 times larger than ideal.
Furthermore, a typical hill climbing algorithm used with a Sobel filter may result in moderately rapid and accurate mechanical AF point selection, as in the above discussion. However, depending on how critical the mechanical AF needs to be, even these algorithms could require up to half a second or more to settle. This refers to the time-limiting factor during Step #1, that creates lag between request and capture, that the first preferred embodiment of the present invention is seeking to minimize by relaxing the mechanical AF accuracy requirements. This adds to the overall lag in the time between a capture is requested by the user and the actual capture takes place. This is undesirable.
Alternatively, if a softlens AF approach is relied upon solely, then this approach must correct for spot sizes that may range over 10 to 20 times larger than those ultimately desired to resolve to sensor's pixels. This refers to “constrained” or “simplified” in Step #3 above. A system that only has softlens AF for focus has to work very hard to succeed, and often does not. That requires a lot of processing power and time. The approach of the first preferred embodiment of the present invention relaxes this requirement on the softlens AF processing by making focus an easier problem to solve—by performing mechanical AF first, thereby range-limiting the softlens AF processing and thus requiring less of the softlens AF than would be required in a softlens-only system. This requires significant care in the application of the phase function imposed onto the lens and significant gate counts and times (seconds) for the post-capture processing. It also results in noticeable artifacts (such as color haloing and edge ringing).
However, if the softlens AF is only required to correct for spot sizes that are 2 to 5 times larger than optimal (because the mechanical AF has done most of the heavy AF lifting first), then the softlens AF process can be more successful with fewer gate counts, shorter processing times and much less noticeable artifacts. As well, because the mechanical AF does not need to achieve its highest capability in any given focus request (because the softlens AF will compensate for the mechanical AF shortcomings), the mechanical AF algorithm can proceed with greater expedience, thereby shortening the lag time during capture. Refer to Steps #1, 3 and 4.
Furthermore, because Petzval curvature exists in all lens designs (wherein the blur function is radially dependent) and because mechanical tip/tilt misalignment generally exists in camera modules, even an ideal mechanical AF process will not provide an ideal image under realistic conditions. Petzval curvature and lens-sensor tip/tilt misalignment will create blur that worsens toward the image corners.
Fortunately, with a softlens AF booster to the mechanical AF, practical artifacts such as Petzval curvature and tip/tilt blur can be corrected for. After mechanically-focused capture, the softlens AF varies its strength depending on known mechanical AF limitations across the image plane. Then, the softlens AF compensates for and levels out all net focus limitations to produce idealized images.
In a second preferred embodiment of the present invention, after a first-pass uniform depth of field softlens the autofocusing action is taken (refer to Step #3.a. below), additional location-specific processing may be required to refine the focus where tip/tilt misalignment or curvature make blur in those regions worse than in regions that do not suffer from defocus due to these effects. This results in Step #5. The sequence to achieve this is as follows:
In a third preferred embodiment of the present invention, a further sixth step is advantageous where the user can select the depth at which he wishes his image to be at best focus, as well as the overall depth of focus in his image. The sequence to achieve this is as follows:
We now describe the method of the first preferred embodiment of the present invention with reference to the block diagram of
We now describe the method of the second preferred embodiment of the present invention with reference to the block diagram of
We next describe the method of the third preferred embodiment of the present invention with reference to the block diagram of
While the invention has been particularly shown and described with reference to the preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention.
