The subject application is directed generally to correction of backlit images. The application is particularly advantageous in connection with correction of images relative to backlighting characteristics particular to each acquired image.
Electronic images exist in many formats. By way of example, images may be acquired or stored in various schemes, including RAW, JPEG, GIF, TIFF or PCX, as well as many other image data types. Many image data encoding schemes define images in connection with a multidimensional color space, such as a space defined by either additive or subtractive primary colors. Such color spaces include red-green-blue (RGB); cyan, magenta, yellow (CYM), which is sometimes encoded with a blac(K) component as CMYK.
Acquired images, particularly those that result from real life images such as may be acquired by digital cameras or scans of photographs, are often captured in non-optimal situations. One such situation is presented with backlighting. A relatively bright backlighting tends to wash out or obscure objects in a forefront of such lighting. Backlighting is particularly problematic with human subjects insofar is it can result in obscured facial characteristics.
Earlier attempts at adjustment of images for backlighting artifacts included tonal remapping with piece-wise linear functions or with spatial masking. Such approaches, while somewhat effective, are frequently complex and computationally intensive, and nonetheless result in less than optimal adjustment to compensate for image backlighting.
In accordance with one embodiment of the subject application, there is provided a system and method for correction of backlit images.
Further, in accordance with one embodiment of the subject application, there is provided a system and method for correction of images relative to backlighting characteristics particular to each acquired image.
Still further, in accordance with one embodiment of the subject application, there is provided a system and method for backlit image adjustment. Image data is received which includes an image portion defined by a backlit region associated therewith and a tonal curve is isolated. At least one anchor point on an isolated tonal curve is selected in accordance with backlighting characteristics in received image data and a sectional bulging operation is applied on the isolated tonal curve in accordance with the at least one anchor point. Adjusted image data is generated in accordance with an applied sectional bulging operation and communicated to an associated data storage.
Still other advantages, aspects and features of the subject application will become readily apparent to those skilled in the art from the following description wherein there is shown and described a preferred embodiment of the subject application, simply by way of illustration of one of the best modes best suited to carry out the subject application. As it will be realized, the subject application is capable of other different embodiments and its several details are capable of modifications in various obvious aspects all without departing from the scope of the subject application. Accordingly, the drawings and descriptions will be regarded as illustrative in nature and not as restrictive.
The paten or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
The subject application is described with reference to certain figures, including:
The subject application is directed to a system and method for correction of backlit images. In particular, the subject application is directed to a system and method for correction of images relative to backlighting characteristics particular to each acquired image. More particularly, the subject application is directed to a system and method for backlit image adjustment. It will become apparent to those skilled in the art that the system and method described herein are suitably adapted to a plurality of varying electronic fields employing image adjustment, including, for example and without limitation, communications, general computing, data processing, document processing, or the like. The preferred embodiment, as depicted in
Referring now to
The system 100 also includes a document processing device 104, which is depicted in
According to one embodiment of the subject application, the document processing device 104 is suitably equipped to receive a plurality of portable storage media, including, without limitation, Firewire drive, USB drive, SD, MMC, XD, Compact Flash, Memory Stick, and the like. In the preferred embodiment of the subject application, the document processing device 104 further includes an associated user interface 106, such as a touch-screens LCD display, touch-panel, alpha-numeric keypad, or the like, via which an associated user is able to interact directly with the document processing device 104. In accordance with the preferred embodiment of the subject application, the user interface 106 is advantageously used to communicate information to the associated user and receive selections from the associated user. The skilled artisan will appreciate that the user interface 106 comprises various components, suitably adapted to present data to the associated user, as are known in the art. In accordance with one embodiment of the subject application, the user interface 106 comprises a display, suitably adapted to display one or more graphical elements, text data, images, or the like, to an associated user, receive input from the associated user, and communicate the same to a backend component, such as the controller 108, as explained in greater detail below. Preferably, the document processing device 104 is communicatively coupled to the computer network 102 via a communications link 112. As will be understood by those skilled in the art, suitable communications links include, for example and without limitation, WiMax, 802.11a, 802.11b, 802.11g, 802.11(x), Bluetooth, the public switched telephone network, a proprietary communications network, infrared, optical, or any other suitable wired or wireless data transmission communications known in the art.
In accordance with one embodiment of the subject application, the document processing device 104 further incorporates a backend component, designated as the controller 108, suitably adapted to facilitate the operations of the document processing device 104, as will be understood by those skilled in the art. Preferably, the controller 108 is embodied as hardware, software, or any suitable combination thereof, configured to control the operations of the associated document processing device 104, facilitate the display of images via the user interface 106, direct the manipulation of electronic image data, and the like. For purposes of explanation, the controller 108 is used to refer to any myriad of components associated with the document processing device 104, including hardware, software, or combinations thereof, functioning to perform, cause to be performed, control, or otherwise direct the methodologies described hereinafter. It will be understood by those skilled in the art that the methodologies described with respect to the controller 108 is capable of being performed by any general purpose computing system, known in the art, and thus the controller 108 is representative of such general computing devices and are intended as such when used hereinafter. Furthermore, the use of the controller 108 hereinafter is for the example embodiment only, and other embodiments, which will be apparent to one skilled in the art, are capable of employing the system and method for backlit image adjustment of the subject application. The functioning of the controller 108 will better be understood in conjunction with the block diagrams illustrated in
Communicatively coupled to the document processing device 104 is a data storage device 110. In accordance with the preferred embodiment of the subject application, the data storage device 110 is any mass storage device known in the art including, for example and without limitation, magnetic storage drives, a hard disk drive, optical storage devices, flash memory devices, or any suitable combination thereof. In the preferred embodiment, the data storage device 110 is suitably adapted to store document data, image data, electronic database data, or the like. It will be appreciated by those skilled in the art that while illustrated in
Illustrated in
The system 100 of
The system 100 illustrated in
Turning now to
Also included in the controller 200 is random access memory 206, suitably formed of dynamic random access memory, static random access memory, or any other suitable, addressable and writable memory system. Random access memory provides a storage area for data instructions associated with applications and data handling accomplished by processor 202.
A storage interface 208 suitably provides a mechanism for non-volatile, bulk or long term storage of data associated with the controller 200. The storage interface 208 suitably uses bulk storage, such as any suitable addressable or serial storage, such as a disk, optical, tape drive and the like as shown as 216, as well as any suitable storage medium as will be appreciated by one of ordinary skill in the art.
A network interface subsystem 210 suitably routes input and output from an associated network allowing the controller 200 to communicate to other devices. The network interface subsystem 210 suitably interfaces with one or more connections with external devices to the device 200. By way of example, illustrated is at least one network interface card 214 for data communication with fixed or wired networks, such as Ethernet, token ring, and the like, and a wireless interface 218, suitably adapted for wireless communication via means such as WiFi, WiMax, wireless modem, cellular network, or any suitable wireless communication system. It is to be appreciated however, that the network interface subsystem suitably utilizes any physical or non-physical data transfer layer or protocol layer as will be appreciated by one of ordinary skill in the art. In the illustration, the network interface 214 is interconnected for data interchange via a physical network 220, suitably comprised of a local area network, wide area network, or a combination thereof.
Data communication between the processor 202, read only memory 204, random access memory 206, storage interface 208 and the network interface subsystem 210 is suitably accomplished via a bus data transfer mechanism, such as illustrated by the bus 212.
Also in data communication with the bus 212 is a document processor interface 222. The document processor interface 222 suitably provides connection with hardware 232 to perform one or more document processing operations. Such operations include copying accomplished via copy hardware 224, scanning accomplished via scan hardware 226, printing accomplished via print hardware 228, and facsimile communication accomplished via facsimile hardware 230. It is to be appreciated that the controller 200 suitably operates any or all of the aforementioned document processing operations. Systems accomplishing more than one document processing operation are commonly referred to as multifunction peripherals or multifunction devices.
Functionality of the subject system 100 is accomplished on a suitable document processing device, such as the document processing device 104, which includes the controller 200 of
In the preferred embodiment, the engine 302 allows for printing operations, copy operations, facsimile operations and scanning operations. This functionality is frequently associated with multi-function peripherals, which have become a document processing peripheral of choice in the industry. It will be appreciated, however, that the subject controller does not have to have all such capabilities. Controllers are also advantageously employed in dedicated or more limited purposes document processing devices that perform one or more of the document processing operations listed above.
The engine 302 is suitably interfaced to a user interface panel 310, which panel allows for a user or administrator to access functionality controlled by the engine 302. Access is suitably enabled via an interface local to the controller, or remotely via a remote thin or thick client.
The engine 302 is in data communication with the print function 304, facsimile function 306, and scan function 308. These functions facilitate the actual operation of printing, facsimile transmission and reception, and document scanning for use in securing document images for copying or generating electronic versions.
A job queue 312 is suitably in data communication with the print function 304, facsimile function 306, and scan function 308. It will be appreciated that various image forms, such as bit map, page description language or vector format, and the like, are suitably relayed from the scan function 308 for subsequent handling via the job queue 312.
The job queue 312 is also in data communication with network services 314. In a preferred embodiment, job control, status data, or electronic document data is exchanged between the job queue 312 and the network services 314. Thus, suitable interface is provided for network based access to the controller function 300 via client side network services 320, which is any suitable thin or thick client. In the preferred embodiment, the web services access is suitably accomplished via a hypertext transfer protocol, file transfer protocol, uniform data diagram protocol, or any other suitable exchange mechanism. The network services 314 also advantageously supplies data interchange with client side services 320 for communication via FTP, electronic mail, TELNET, or the like. Thus, the controller function 300 facilitates output or receipt of electronic document and user information via various network access mechanisms.
The job queue 312 is also advantageously placed in data communication with an image processor 316. The image processor 316 is suitably a raster image process, page description language interpreter or any suitable mechanism for interchange of an electronic document to a format better suited for interchange with device functions such as print 304, facsimile 306 or scan 308.
Finally, the job queue 312 is in data communication with a parser 318, which parser suitably functions to receive print job language files from an external device, such as client device services 322. The client device services 322 suitably include printing, facsimile transmission, or other suitable input of an electronic document for which handling by the controller function 300 is advantageous. The parser 318 functions to interpret a received electronic document file and relay it to the job queue 312 for handling in connection with the afore-described functionality and components.
Turning now to
The read only memory 404 suitably includes firmware, such as static data or fixed instructions, such as BIOS, system functions, configuration data, and other routines used for operation of the workstation 400 via CPU 402.
The random access memory 406 provides a storage area for data and instructions associated with applications and data handling accomplished by the processor 402.
The display interface 408 receives data or instructions from other components on the bus 414, which data is specific to generating a display to facilitate a user interface. The display interface 408 suitably provides output to a display terminal 428, suitably a video display device such as a monitor, LCD, plasma, or any other suitable visual output device as will be appreciated by one of ordinary skill in the art.
The storage interface 410 suitably provides a mechanism for non-volatile, bulk or long term storage of data or instructions in the workstation 400. The storage interface 410 suitably uses a storage mechanism, such as storage 418, suitably comprised of a disk, tape, CD, DVD, or other relatively higher capacity addressable or serial storage medium.
The network interface 412 suitably communicates to at least one other network interface, shown as network interface 420, such as a network interface card, and wireless network interface 430, such as a WiFi wireless network card. It will be appreciated by one of ordinary skill in the art that a suitable network interface is comprised of both physical and protocol layers and is suitably any wired system, such as Ethernet, token ring, or any other wide area or local area network communication system, or wireless system, such as WiFi, WiMax, or any other suitable wireless network system, as will be appreciated by one of ordinary skill in the art. In the illustration, the network interface 420 is interconnected for data interchange via a physical network 432, suitably comprised of a local area network, wide area network, or a combination thereof.
An input/output interface 416 in data communication with the bus 414 is suitably connected with an input device 422, such as a keyboard or the like. The input/output interface 416 also suitably provides data output to a peripheral interface 424, such as a USB, universal serial bus output, SCSI, Firewire (IEEE 1394) output, or any other interface as may be appropriate for a selected application. Finally, the input/output interface 416 is suitably in data communication with a pointing device interface 426 for connection with devices, such as a mouse, light pen, touch screen, or the like.
In operation, image data is first received that includes an image portion defined by an associated backlit region. A tonal curve associated with the received image data is then isolated. At least one anchor point on the isolated tonal curve is then selected based upon backlighting characteristics in the received image data. A sectional bulging operation is then applied on the isolated tonal curve in accordance with the selected anchor point. Adjusted image data is then generated in accordance with the applied sectional bulging operation. The adjusted image data is then communicated to an associated data storage.
In accordance with one example embodiment of the subject application, image data is first received by the computer workstation 122, the document processing device 104, or other suitable processing device. It will be appreciated by those skilled in the art that the image data is capable of being received from operations of the document processing device 104, e.g. scanning, from an external device, e.g. camera, from a portable storage media, from a networked device (not shown), or the like. The workstation 122, controller 108 or other suitable component associated with the document processing device 104 then analyzes the received image data to determine whether the image includes a backlit portion.
A first anchor point is then selected at a lower value on the tonal curve based upon backlighting characteristics in the received image data and a second anchor point is selected at a higher value on the tonal curve based upon the backlighting characteristics.
A saturation boost is then applied to the image via first converting the adjusted image data, e.g. the input image 500, from RGB color space to HSV color space. The workstation 122, the controller 108 or other suitable component associated with the document processing device 104 then extracts the S (saturation) plane. A boost bulging factor is then applied to the extracted S component. For example, a tone reproduction curve 900 with a bulging factor of GammaS 902 is then applied as shown in
The skilled artisan will appreciate that various degrees of such tone remapping are capable of being applied through the selection of different anchor points and bulging factors. For example,
The skilled artisan will appreciate that the subject system 100 and components described above with respect to
The workstation 122 then isolates a tonal curve associated with the received image data at step 2004. Suitable examples of such a tone reproduction curve for an input image are illustrated in
At step 2010, adjusted image data is generated in accordance with the applied sectional bulging operation by the computer workstation 122. The adjusted image data is then communicated by the workstation 122 to an associated data storage (not shown) at step 2012. In accordance with one embodiment of the subject application, the controller 108 or other suitable component associated with the document processing device 104 applies the sectional bulging operation so as to generate the adjusted image data at step 2010. The skilled artisan will thus appreciate that the adjusted image data is then communicated to the associated data storage 110 at step 2012.
Referring now to
The image data is then analyzed so as to determine, at step 2104, whether the received image data contains a backlit portion. When the workstation 122 determines that the received image data does not include a backlit portion, operations with respect to
At step 2106, the computer workstation 122 isolates a tonal curve associated with the image data, as will be appreciated by those skilled in the art. Suitable examples of such tone reproduction curves are illustrated above with respect to
The workstation 122 then applies a sectional bulging operation on the tonal curve at values between the first and second selected anchor points at step 2112. Adjusted image data is then generated by the workstation 122 at step 2114 in accordance with the applied sectional bulging operation. The workstation 122 then converts the adjusted image data from RGB color space to HSV color space at step 2116. An S component of the converted image data is extracted by the workstation 122 at step 2118. The workstation 122 then applies a boost bulging factor to the S component at step 2120. At step 2122, the boosted S component is combined with the remaining planes in HSV color space by the workstation 122. The combined planes in HSV color space are then converted to RGB color space at step 2124. Thereafter, at step 2126, the adjusted image data is communicated to an associated data storage by the workstation 122, e.g. an internal hard disk drive, system memory, portable storage media, optical recording media, or the like. It will be appreciated by those skilled in the art that steps 2116-2124 correspond to the application of a saturation boost to the adjusted image data prior to the communication thereof to the associated data storage.
The foregoing description of a preferred embodiment of the subject application has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the subject application to the precise form disclosed. Obvious modifications or variations are possible in light of the above teachings. The embodiment was chosen and described to provide the best illustration of the principles of the subject application and its practical application to thereby enable one of ordinary skill in the art to use the subject application in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the subject application as determined by the appended claims when interpreted in accordance with the breadth to which they are fairly, legally and equitably entitled.