This disclosure relates to image sharpening, and more specifically, uniform sharpening of captured images.
Digital cameras are increasingly used in day-to-day activities. Most users of digital cameras are relatively novice photographers and are not well versed with the mechanical/operational/electronic details of the cameras. The users often prefer cameras that provide the best quality of images without having to manually adjust a lot of camera settings. Additionally, users often prefer cameras that are not priced too high. To achieve lower cost, camera manufacturers use cheaper lenses that have average lens quality and thereby reducing image quality.
Reduction of image quality from lower quality lenses may be due to lens imperfections. The imperfections degrade the image quality captured on an image sensor. For example, the captured image may be out of focus or blurred images. Some digital cameras include post processing capabilities that include sharpening of captured images. The process to sharpen an image includes adjusting the image pixels to reduce the blur, thus improving the image quality without upgrading to a better performing lens. To achieve the best results, the image may be sharpened in as many possible dimensions of the camera lens. This may increase resources required for post processing the image. For example, post processing may require camera resources that include greater image processing power, more memory, and / or larger battery capacity to manage the processing needs. These additional resources may increase costs and may increase size when more compact camera forms may be desired.
The disclosed embodiments have other advantages and features which will be more readily apparent from the following detailed description of the invention and the appended claims, when taken in conjunction with the accompanying drawings, in which:
The figures and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.
Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. It is noted that wherever practicable similar or like reference numbers may be used in the figures and may indicate similar or like functionality. The figures depict embodiments of the disclosed system (or method) for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.
By way of example, the disclosed configuration may include a system that may be configured for sharpening images from a camera. The system receives an image that includes Euclidian plane components and polar components. The Euclidian plane components include dimensional components, for example, pixels in the X-coordinate, Y-coordinate or Z-coordinate. The polar components include radii and angular components, for example, pixels at a specified angle in degrees or radians or at a specific radius. A number of sub-images are determined from the image. The sub-images may be determined based on the Euclidian plane components. The system further receives a specific rotation angular component to align each sub-image.
To align the sub-image based on the specified angular component, the system determines a rotation matrix based on the specific angular component. Each pixel of the sub-image is rotated by multiplying a pixel vector of the sub-image to a vector of the rotation matrix. Further, the system receives a sagittal function and a tangential function. The sagittal function and tangential functions are determined based on a modulation transfer function (MTF) of the lens of the camera. The MTF is a performance function plot of a lens of the camera for a plurality of radii of the lens. The system determines a sharpening function based on a specific radii and the specified rotation angular component to sharpen the sagittal portion of pixels of the sub-image. Further, the system determines a sharpening function based on a specific radii and the specified rotation angular component to sharpen the tangential portion of pixels of the sub-image. The rotated and sharpened pixels of each of the sub-images are further inverse rotated based on an inverse rotation angular component. The sub-images are blended along the edge of the sub-images to sharpen any discontinuities and in the process, obtain a sharpened image.
In addition to a system, the configuration described herein also may be embodied as a method (or process) and may be embodied as a non-transitory computer readable storage medium having stored instructions (or program code) corresponding to the methods and/or functionality and are executable by one or more processors.
The unprocessed images from the camera/computing device 105 may be sent to the image sharpening module 110. The image sharpening module 110 may apply a uniform image sharpening algorithm based on polar co-ordinates on the unprocessed image, e.g. raw image. The image sharpening algorithm corrects the focus and blurs in the image, which may result in a better quality and sharpened image. The sharpened image may be further processed and sent to a display device 120 for a user to view.
The sub-image determination module 205 may be configured to receive an unprocessed image represented in the Euclidian plane that includes an X-axis and Y-axis dimension, from camera or the computing device 105. The sub-image determination module 205 may be configured to split the image in to a number of sub-images. The number is configurable, for example, for an image size of 640 pixels by 480 pixels can be split into a number of sub-images of size 40 pixels by 30 pixels.
The image 310 can be divided into a number of sub-images 320, each sub-image represented by four (X-axis, Y-axis) co-ordinates. The sub-image 320 in
Referring back to
The X/Y sharpening module 230 may apply sharpening along the X-axis or Y-axis of the rotated image based on a sagittal or a tangential function of the image. The sharpening algorithm parameters include a sagittal function S(r) or a tangential function T(r) The sagittal function S(r) and tangential function T(r) are determined based on the modulation transfer function (MTF) of the camera lens. The MTF is a measure of the transfer of modulation from the subject to the image (i.e., a measure of the image quality of the lens). The image quality changes from the center of the lens to the edge of the lens. The change in the image quality is determined by a plot of the MTF values for the each radius for tangential and sagittal orientations. Examples of the tangential and sagittal orientations are illustrated in
The sharpening may be a function based on a radius for a sagittal or tangential portion of the image. Depending on the direction of the rotation angle, the sagittal or tangential portion is sharpened on the X-axis or Y-axis. The X′ sharpening applied to a sagittal portion S(r) may be S(y) or S(x), depending on the direction of the rotation angle. Similarly, the Y′ sharpening applied to a tangential portion T(r) may be T(y) or T(x).
For example, if a sagittal portion is rotated in a clockwise direction, a sharpening function (X′) is applied along the X-axis, if a sagittal portion is rotated in a counter clockwise direction, a sharpening function (X′) is applied along the Y-axis. Similarly, if a tangential portion is rotated clockwise, a sharpening function (Y′) is applied along the Y-axis and vice versa.
Once the rotated sub-image is sharpened, the sub-image inverse rotation module 240 may rotate the sharpened sub-image to the original orientation. In one example embodiment, the inverse rotation of the sub-image is done by rotating the sub-image by the specified rotation angle in a direction opposite to the original rotation. For example, if the sub-image was rotated at an angle of 40 degrees in the clockwise direction by the sub-image rotation module 210, the sub-image inverse rotation module 240 rotates the sub-image in a counter clockwise direction by 40 degrees. In another example embodiment the sub-image can be rotated in the same direction as the original rotation but by a different rotation angle. For example, if the sub-image was rotated at an angle of 40 degrees in the clockwise direction by the sub-image rotation module 210, the sub-image inverse rotation module 240 rotates the sub-image in a clockwise direction by 320 degrees to reach the original orientation of the sub-image.
The sharpened sub-images may be blended by the sub-images blending module 250 to reduce the discontinuities between the sub-images. In one example embodiment, the blending is done by applying an average function over two or more sub-images. The averaging function reduces the step discontinuities between sub-images. In another example embodiment, the blending is done by polynomial curve fitting. The polynomial curve fitting is the process of constructing a curve (i.e., a mathematical function) that has a best fit to a series of data points. The data points in this case are the step sizes (i.e., the sub-images or portions of sub-images or portions of sub-images that overlap one or more sub-images) that the image is split up into. The polynomial curve fitting is a computationally intensive method of blending the sub-images.
In one embodiment, each pixel (e.g., X, Y as illustrated in
The pixels or set of pixels of the rotated sub-image are further sharpened by the X-Y sub-image sharpening module 230. The sharpening is applied based on the sagittal and tangential functions of the sub-image pixels. The sagittal and tangential functions of a sub-image, for each radius are retrieved 730 from the sharpening parameters database 220. The sagittal and tangential functions are based on a modular transfer function plot (MTF) of the camera lens, the MTF representing the quality of image transferred from the subject to the image. The sagittal function S(r) represents the pixels that are at longitudinal orientation of the sub-image towards the radii and the tangential function T(r) represents the pixels that are at a perpendicular orientation to the radii, i.e., tangential to the radii.
A sharpening algorithm is applied to the sagittal and tangential functions of the pixels of the sub-image. Examples of sharpening algorithms include unsharp masking, edge enhancement, etc. An exemplary form of image sharpening involves a form of contrast. This is performed by finding the average color of the pixels around each pixel in a specific radius, and then contrasting that pixel from that average color. In one embodiment, the sagittal function of the set of pixels of the sub-image is sharpened 750 along the X-axis, i.e., in a direction opposite to the orientation of the sagittal component of the pixels. Similarly, the tangential function of the set of pixels of the sub-image is sharpened 760 along the Y-axis.
The application of the sharpening further depends on the rotation direction of the sub-image. For example, if the sub-image is rotated in a counter-clockwise direction, sharpening to the sagittal function of the pixels is applied in the Y-axis direction, i.e., S(r)=S(y), if the sub-image is rotated in a clockwise direction, the sharpening is applied in the X-axis direction, i.e. S(r)=S(x). Similarly, if the sub-image is rotated in a counter-clockwise direction, sharpening to the tangential function of the pixels is applied in the X-axis direction, i.e., T(r)=T(x), if the sub-image is rotated in a clockwise direction, the sharpening is applied in the Y-axis direction, i.e., T(r)=T(y).
The sharpened pixels of the sub-images are inverse rotated 770 to their original orientation by the sub-image inverse rotation module 240. An exemplary inverse rotation matrix R−1 is illustrated in
The sharpened and inverse-rotated sub-images are blended 780 by the sub-images blending module 250. When the sub-images are stitched together, the stitched edges have discontinuities that may be visible in the image. To reduce these discontinuities a blending algorithm is applied to the each of the sub-images, or in a running window of sub-images. A running window of sub-images involves carrying forward a portion of blended sub-image to the next sub-image to ensure a smoother blend of sub-images. Exemplary blending algorithms include a point cloud mesh, polynomial webbing and other such algorithms.
The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a smartphone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions 824 (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute instructions 824 to perform any one or more of the methodologies discussed herein. For example, the computer system 800 can be used to operate the image sharpening module 110 and the processes described with respect to
Looking closer at the example computer system 800, it includes one or more processors 802 (generally processor 802) (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio-frequency integrated circuits (RFICs), or any combination of these), a main memory 804, and a static memory 806, which are configured to communicate with each other via a bus 808. The computer system 800 may further include graphics display unit 810 (e.g., a plasma display panel (PDP), a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)). The computer system 800 may also include alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing instrument), a storage unit 816, a signal generation device 818 (e.g., a speaker), and a network interface device 820, which also are configured to communicate via the bus 808.
The storage unit 816 includes a machine-readable medium 822 on which is stored instructions 824 (e.g., software) embodying any one or more of the methodologies or functions described herein. The instructions 824 (e.g., software) may also reside, completely or at least partially, within the main memory 804 or within the processor 802 (e.g., within a processor's cache memory) during execution thereof by the computer system 800, the main memory 804 and the processor 802 also constituting machine-readable media. The instructions 824 (e.g., software) may be transmitted or received over a network 826 via the network interface device 820.
While machine-readable medium 822 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions (e.g., instructions 824). The term “machine-readable medium” shall also be taken to include any medium that is capable of storing instructions (e.g., instructions 824) for execution by the machine and that cause the machine to perform any one or more of the methodologies disclosed herein. The term “machine-readable medium” includes, but not be limited to, data repositories in the form of solid-state memories, optical media, and magnetic media. It is noted that the instructions 824 can correspond to the processes of the image sharpening module 110 and its components as well as the processes of
Example benefits and advantages of the disclosed configurations may include, for example, images that are sharpened in the Euclidian and polar dimensions based on the camera lens. The sharpening algorithm provides improved image qualities, i.e., removes the blur components of an image. Hence, photos and videos are of higher quality, e.g., greater sharpness. In addition, the sharpening algorithm may be used for noise removal in other applications, for example, X-ray imaging, machine vision imaging and the like.
Throughout this specification, some embodiments have used the expression “coupled” along with its derivatives. The term “coupled” as used herein is not necessarily limited to two or more elements being in direct physical or electrical contact. Rather, the term “coupled” may also encompass two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other, or are structured to provide a thermal conduction path between the elements.
Likewise, as used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
In addition, use of the “a” or “an” are employed to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise.
Finally, as used herein any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.
Upon reading this disclosure, those of skill in the art will appreciate image sharpening processes and algorithms that sharpen an image in the Euclidian and polar dimensions of the camera lens from the principles herein. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.
This application claims the benefit of U.S. Provisional Application No. 62/259,606, filed Nov. 24, 2015, the content of which is incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
62259606 | Nov 2015 | US |