Patent Application
20050094891
The present invention is generally related to image capture devices and, more particularly, is related to a system and method for assisting in the composition of the tone of captured images.
Composition of a captured image may be difficult for some users of image capture devices. In addition to lighting, focus, focal length and field-of-view issues, the user must also consider the exposure of the image, referred to hereinafter as tone composition, so that all objects in the image have adequate discernable detail. That is, do the objects of interest and/or the background objects of the captured image have tone composition that is in an interesting and visually pleasing manner? Less than optimal tone composition in a captured image results in portions of the image being overexposed, discernable as very bright or white regions with little visible detail, and/or portions of the image being underexposed, discernable as very dark or black regions with little visible detail.
Images that exhibit a wide range of tones, referred to as high contrast images, are difficult to capture without undesirable overexposed or underexposed regions since the lighting conditions over the frame of the captured image varies over a large range. The exposure of the image is based upon determining a desirable exposure for the primary object of interest in the frame. Accordingly, when the primary object of interest is captured, details of the primary object of interest are discernible. However, when the image is captured in a high contrast image, other objects may be overexposed or underexposed because the exposure has not been optimized for the lighting conditions of these objects, resulting in less discernable detail for these objects.
Ideally, this condition would be noticed by the user, either before the image is taken, or immediately after, allowing another adjusted image to be captured. Digital cameras typically provide a display that provides a preview of the image to be captured and postview of the captured image. However, the displayed image quality and/or resolution on these displays is often inadequate for distinguishing highlight and shadow detail. Such displays may have a very limited contrast range. In addition, some viewing conditions in the field make it difficult to critically evaluate a displayed image since ambient lighting conditions interfere with image viewing. For example, even if the display was capable of displaying images with very high contrast, glare from the scene may limit the discernable contrast of the viewed image.
In the image 102, the clouds 110 are very bright, relative to the scenic mountain 104 with the deer 106 and the tree 108 in the foreground. Accordingly, when the image 102 is captured with the deer 106 and tree 108 set at a middle gray tone setting, the clouds 110 are overexposed. That is, when the image 102 is captured, the pixels of a photoelement array that are detecting light from the clouds 110 become saturated, and thereby record the clouds 110 as being very bright and white.
Similarly, in the image 112, the background area of the image where the picture 116 resides is dark, relative to the image of the woman 114. Accordingly, when the image 112 is captured with the woman 114 set at a middle gray tone setting, the background area where the picture 116 resides is underexposed. That is, when the image 112 is captured, the pixels of a photoelement array that are detecting light from picture 116 and the left-hand side of the image 112 detect relatively little light, and thereby record the picture 116 as being black or shaded. It is therefore desirable to assist a user of a digital camera in composing tone aspects of an image to be captured and/or later processed.
The present invention provides a system and method for assisting in tone composition of an image. Briefly described, one embodiment comprises receiving light information corresponding to light detected by a plurality of pixels, analyzing the light information to determine a light value for each of the plurality of pixels, determining contour information, the contour information corresponding to a plurality of groups of adjacent pixels, wherein each group of adjacent pixels is characterized by a unique predetermined range of light values, and communicating the contour information to a display.
Another embodiment comprises a photosensor comprising a plurality of pixels that detect light; a processor that receives light information from the pixels, that analyzes the light information to determine a light value for each of the plurality of pixels and that determines contour information, the contour information corresponding to a plurality of groups of adjacent pixels, wherein each group of adjacent pixels is characterized by a unique predetermined range of light values; and a display configured to display the contour information and a preview image.
The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views.
Embodiments are related to a system and method for assisting a user in the composition of the tone of captured images. An embodiment of the tone contour system 400 (
When the contour information, such as tone contour lines, are displayed concurrently with a preview image before image capture, the user understands that portions of the image, when captured, may be overexposed and/or underexposed. Tone contour lines indicate transitions between tone contour regions having similar exposure characteristics. One embodiment of the tone contour system 400 adjusts exposure for overexposed and/or underexposed pixels. In another embodiment, the user may adjust the tone composition of the image by specifying exposure changes. In yet another embodiment, the contour information is displayed concurrently with an image after image capture, such as on a personal computer (PC) or the like, so that a user may adjust the tone composition of the image at a later convenient time.
For convenience, tone characteristics of captured and preview images are described by tone values (zones) based upon the well known “zone system” which describes tone (exposure) using tone values ranging form the Roman numerals 0 through X. Overexposed objects, characterized by zone X, are viewed as completely white, with no gray shading. Zone IX shows objects having the first perceptible gray details on an otherwise white area. Underexposed objects, characterized by zone 0, are viewed as completely black, with no gray shading. Zone I shows objects having the first perceptible gray details on an otherwise black area.
Zone V is referred to as “middle gray.” Middle gray corresponds to a light reflectance from the object at approximately eighteen percent (18%) of the incident light on the object. When an image is captured, the primary object(s) of interest in the captured image (or at least a portion thereof) will be captured at an exposure that substantially corresponds to zone V.
During image capture, a portion 418 of photosensor 410, or an exposure detector 412, (
As apparent in the preview image 202, capturing an image may not result in a visibly pleasing image to some viewers if clouds 110 are overexposed (thereby showing little detail of the clouds 110). When the image is captured, in the absence of an exposure adjustment by embodiments of the tone contour system 400, the clouds 110 will be overexposed (zones X-IX) since the primary objects of interest, the deer 106 and tree 108, are exposed at a middle gray tone setting.
Once the tone contour system 400 determines contour information of preview image 202, the contour information is communicated to the user. In one embodiment, contour information is displayed concurrently with preview image 202 as one or more lines indicating borders or transitions between portions of the preview image 202 characterized by different predefined exposure values or tone values. Such lines are referred to hereinafter as “tone contour lines.” For example, in one embodiment, tone contour line 204 indicates a transition between a portion of the clouds 110 that are at zone X (as indicated by a completely white or washed out region) and a portion at zone IX (as indicated by barely discernable features of the clouds 110). Similarly, tone contour line 206 indicates a transition between the above-described portion of the clouds 110 that are at zone IX, and a portion of preview image 110 at zone VIII or lower (as indicated by the discernable features of the mountain 104).
In another embodiment, indicia are also displayed to further indicate regions of the preview image 202 that have similar tone characteristics, referred to hereinafter as “tone contour regions.” In the simplified illustrative example of the preview image 202, the letter “A” indicates a tone contour region 208 corresponding to that portion of the preview image 202, when captured, that will be overexposed at zone X. The letter “B” indicates a tone contour region 210 corresponding to that portion of the preview image 202, when captured, that will be somewhat overexposed at zone IX. Any suitable indicia may be used, including roman numerals that correspond to the zone system.
Furthermore, tone contour regions may be shaded, textured and/or colored to further enhance the discernability of a tone contour region to the viewer. In one embodiment, the tone contour region corresponding to zone X (completely white due to overexposure) is indicated by coloring the tone contour region in red or in another suitable warning color. Other tone contour regions may be similarly colored with suitable warning colors.
In one embodiment, the user actuates the tone contour system 400 such that the exposure is automatically adjusted for those pixels corresponding to the identified tone contour regions to a more desirable exposure and/or to a predetermined exposure. For example, one embodiment adjusts exposure of the tone contour region 208 (zone X) to another predefined exposure corresponding to another zone, such as, but not limited to, zone IX. Exposure of tone contour region 210 (zone IX) is adjusted to another predefined zone, such as, but not limited to, zone VIII. Exposure of tone contour regions may also be adjusted to fractional zone values. In embodiments employing other tone indicia systems, the initial exposure, initial light value, initial zone and/or initial tone indicia is changed to an adjusted exposure, adjusted light value, adjusted zone and/or adjusted tone indicia, respectively.
Some embodiments of the tone contour system 400 automatically adjust exposure. In another embodiment, the user specifies the amount of adjustment to the initial tones, exposures, zones or other tone indicia for a tone contour region, thereby enabling the user to change exposure of portions of the photosensor 410 based upon user preferences. The specification of tone adjustments may be implemented via a menu system or by other suitable controllers. It is appreciated that the adjustment exposure for tone contour regions from an initial exposure, initial light value, initial zone and/or initial tone indicia to an adjusted exposure, adjusted light value, adjusted zone and/or adjusted tone indicia, respectively, is nearly limitless.
In one embodiment, after exposure of the preview image 202 has been adjusted as described above, an adjusted preview image is displayed. Accordingly, contour information is again determined based upon the adjusted tone exposure (and/or another exposure). The adjusted contour information, in the form of adjusted tone contour regions, tone contour lines, tone contour zone indicia, and/or other suitable indicia, is displayed concurrently with an adjusted preview image to indicate to the user the tone change that has been implemented by the tone contour system 400. For example, the adjusted preview image may appear similar to image 212 of
As apparent in the preview image 302, capturing an image of the woman 114 and the picture 116 may not result in a visibly pleasing image to some viewers since the picture 116 will be underexposed when captured. Accordingly, when the image is captured, detail of picture 116 will not be discernable to a viewer of the captured image. The picture 116 will be underexposed (zones 0-I) since the primary object of interest, the woman 114, is exposed at a middle gray tone setting.
Once the tone contour system 400 determines contour information of preview image 302, the contour information is communicated to the user. In one embodiment, contour information is displayed concurrently with preview image 302 using tone contour lines indicating transitions between the tone contour regions characterized by different predefined exposure values. For example, tone contour line 304 indicates a transition between a portion of preview image 302 at zone 0 (as indicated by a completely black region) and a portion at zone I. Similarly, tone contour line 306 indicates a transition between the above-described portion of the preview image 302 that is at zone I, and a portion of preview image 302 at zone II or higher.
In another embodiment, indicia are also displayed to further indicate tone contour regions of the preview image 202 that have similar tone characteristics. In the simplified illustrative example of the preview image 202, the letter “M” indicates a tone contour region 308 corresponding to that portion of the image, if captured without an exposure adjustment by embodiments of the tone contour system 400, will be underexposed at zone 0. The letter “N” indicates a tone contour region 310 corresponding to that portion of the image that will be underexposed at zone 1. Any suitable indicia may be used, including Roman numerals that correspond to the zone system.
As noted above, tone contour regions may be shaded, textured and/or colored to further enhance the discernability of a tone contour region to the viewer. In one embodiment, the tone contour region 308 corresponding to zone 0 (completely black due to underexposure) is indicated by coloring the tone contour region in black or in another suitable warning color. Other tone contour regions, such as tone contour region 310, may be similarly colored with suitable warning colors.
In one embodiment, the user actuates the tone contour system 400 such that the exposure for those pixels corresponding to the identified tone contour regions is adjusted to a more desirable exposure and/or to a predetermined exposure. For example, one embodiment adjusts exposure of the tone contour region 308 (zone 0) to an exposure corresponding to another predefined zone, such as, but not limited to, zone I. Exposure of tone contour region 310 (zone I) is adjusted to an exposure corresponding to another predefined zone, such as, but not limited to, zone II. Exposure of tone contour regions may also be adjusted to fractional zone values.
In embodiments employing other tone indicia systems, the original exposure, zone and/or tone indicia is changed to an adjusted exposure, zone and/or tone indicia. In another embodiment, the user is able to specify the tones, zones or other tone indicia for a tone contour region, thereby enabling the user to change exposure of portions of the photosensor 410 based upon the tone contour information provided by the tone contour system 400. It is appreciated that the adjustment exposure for tone contour regions from an originally determined zone to another tone zone(s) is nearly limitless.
As described above, some embodiments determine the tone characteristics of the object of interest with a predefined portion 418 of photosensor 410. The pixels in the portion 418 detect light, and then processor 406 determines the detected light intensity. The portion 418 is located at a predefined area in the frame generally corresponding to a location where a user is expected to frame a primary object of interest into the captured image. Based upon the light detected by the portion 418, the image capture device 402 initially sets an exposure for all of the pixels of photosensor 410 such that when the pixels at the portion 418 capture light, that portion of the captured image exhibits tone that substantially corresponds to middle gray. That is, the exposure for the photosensor 406 is initially specified so that the pixels corresponding to the predefined point capture approximately 18% of the incident light reflecting from the object.
In embodiments employing exposure detector 412, the exposure detector 412 is a photosensitive device that detects light intensity at a predefined point (or small area) in the frame generally corresponding to a location where a user is expected to frame an object of interest on the captured image. The detected light intensity is used to determine contour information. In other embodiments, the exposure detector may be another multi-function photosensitive device wherein a portion is used to detect light intensity to determine contour information. The exposure device may have a sufficient number of pixels to detect the initial exposure at various points across the entire frame of the image to be captured so that contour information can be determined from light detected by the exposure detector 412.
Selected external components of digital camera 402 include lens 420, viewfinder 422, shutter button 424, controller 426, display 428 and power switch 430. For convenience, display 428 is illustrated on the top of digital camera 402. In other embodiments, display 428 is located in another suitable location on the digital camera 402.
Operation of the digital camera 402 is initiated by actuation of the power switch 430 or an equivalent device having the same functionality. Photosensor 410 is disposed in a suitable location behind lens 420 such that an image of the object of interest may be focused onto photosensor 410 for capturing. In one embodiment, display 428 displays a view of an image currently visible through the lens 420 and detected by photosensor 410, referred to herein as a preview image.
Prior to capturing an image of the object of interest, the operator of the digital camera 402 visually previews the image of the object on display 428. Or, a view corresponding to the image of the object may be viewed directly through the viewfinder 422. Embodiments of the tone contour system 400 display the determined contour information on display 428 and/or display the contour information through the viewfinder 422.
Processor 406 retrieves and executes the contour determination logic 416. Data corresponding to the light intensity detected by portion 418, or by exposure detector 412, is analyzed by processor 406 to determine the initial tone, exposure level or other suitable indicia of the primary object of interest. Exposure for all pixels of photosensor are initially set based upon the exposure specified for the primary object of interest.
In one embodiment, light information is retrieved from all of the pixels of photosensor 410. The retrieved light information is analyzed by processor 406 to determine the amount of detected light, or exposure, for the pixels when their exposure is at the initially specified exposure for the primary object of interest.
In another embodiment, light information from selected pixels of photosensor 410 is retrieved and analyzed. Pixels are selected across the photosensor 410 such that exposure across the image can be approximated. Contour information is then determined from light information provided by the selected pixels. Accordingly, processing time is reduced when light information from the selected pixels is analyzed.
In embodiments having exposure detector 412 having a sufficient number of pixels to detect the initial exposure at various points across the entire frame of the image to be captured, light information from exposure detector 412 is used to determine the initial exposure of portions of the image. Contour information is then determined for the image based upon light information provided by the pixels of the exposure detector 412. Accordingly, processing time is reduced because light information from a relatively fewer number of pixels can be quickly retrieved and analyzed.
Contour information is determined by identifying pixels that have a similar exposure, or similar tone characteristics. That is, pixels detecting similar levels of light (similar exposure) are identified. Groups of adjacent pixels having similar exposure are used to define tone contour regions.
In one embodiment, pixels having an exposure within a predefined range of exposures are identified as pixels having similar exposure. For example, fully saturated or nearly fully saturated pixels, also referred to as fully exposed or nearly fully exposed pixels, are categorized by a suitable indicia, such as, but not limited to, zone X. For example, pixels that are between 100% saturated and 95% saturated may be categorized as having an exposure corresponding to zone X (or another suitable indicia). Accordingly, groups of adjacent pixels categorized as zone X define one or more tone contour regions having a tone corresponding to zone X (or another suitable indicia).
Similarly, other pixels having another similar exposure range are identified. Highly saturated pixels may be categorized by a suitable indicia, such as, but not limited to, zone IX. For example, pixels that are between 95% saturated and 90% saturated may be categorized as having an exposure corresponding to zone IX (or another suitable indicia). Accordingly, groups of adjacent pixels being classified as zone IX define one or more tone contour regions having a tone corresponding to zone IX (or another suitable indicia). Ranges may be broadened to encompass a greater range of exposure. For example, a tone contour region may be defined to have pixels which are categorized into zones IX and VIII.
Returning to the exemplary preview image of
In the above-described example of the contour information displayed concurrently with preview image 202 using tone contour lines 204 and 206, and tone contour regions 208 and 210, exposure in one embodiment is adjusted when the value of the light information received from pixels in a particular tone contour region is adjusted. The value of the light information may be decreased by subtracting a predefined value from the received light information, and/or may be scaled by a suitable predefined value. For example, the value of the light information from pixels in contour region 208 may be scaled by 90%. Thus, light information from a pixel that has a value corresponding to 100% exposure would be adjusted to a value corresponding to a 90% exposure level. Light information from a pixel that has a value corresponding to 97% exposure would be adjusted to a value corresponding to an 87.3% exposure level. Exposure may be adjusted, for example, using the method of U.S. Pat. No. 5,999,659 to Shimazu et al., or by other known processes of adjusting exposure of images captured by digital image capture devices.
For convenience, the initial exposure is defined herein to be the exposure (level of detected light) of pixels during image preview or of the pixels at the time of image capture (before exposure adjustment). The adjusted exposure is defined herein to be the adjusted exposure of light information from pixels used during image capture after light information from the pixels is adjusted by embodiments of the tone contour system 400.
Accordingly, the initial exposure, initial light information, initial zone and/or initial tone indicia is changed to an adjusted exposure, adjusted light value, adjusted zone and/or adjusted tone indicia, respectively, by multiplying the received light information by the predefined value of 90% in the above-described simplified example. It is appreciated that the adjustment of exposure for tone contour regions is nearly limitless.
When the user has focused the image of the object of interest, and is satisfied with the exposure of the image, the operator actuates the shutter button 424 to cause digital camera 402 to capture the image. Photosensor 410 detects the object of interest through lens 420 and communicates light information corresponding to the detected image to the processor 406. Data corresponding to the captured image is stored in the captured image data region 414 of memory 408, or in a suitable detachable memory 402 (not shown) coupled to the digital camera 402.
After exposure adjustment by embodiments of the tone contour system 400, the captured image when later displayed will show additional detail in the adjusted regions of the captured image. In the simplified example of
As described above, adjacent pixels having exposure within a predefined range are grouped into a contour zone. As another simplified illustrative example, unexposed or barely exposed pixels are characterized with a suitable indicia, such as, but not limited to, zone 0. For example, pixels that are between 0% exposed and 5% exposed may be categorized as having an exposure corresponding to zone 0 (or another suitable indicia). Accordingly, groups of adjacent pixels being categorized as zone 0 define a tone contour region having a tone corresponding to zone 0 (or another suitable indicia).
Similarly, other pixels having exposure within another predefined range are identified. For example, barely exposed pixels may be characterized with a suitable indicia, such as, but not limited to, zone II. For example, pixels that are between 5% exposed and 10% exposed may be categorized as having an exposure corresponding to zone II (or another suitable indicia). Accordingly, groups of adjacent pixels being classified as zone II define a tone contour region having a tone corresponding to zone II (or another suitable indicia). Ranges may be broadened to encompass a greater range of exposure. For example, a tone contour region may be defined to have pixels which are classified into zones II and III.
Returning to the exemplary preview image of
In the above-described example of the contour information displayed concurrently with preview image 302 using tone contour lines 304 and 306, and tone contour regions 308 and 310, embodiments of the tone contour system 400 adjust the exposure of pixels classified into the tone contour regions 308 and 310. In one embodiment, actual exposure of the pixels in a particular tone contour region is increased by a predefined amount of time. In another embodiment, the value of the light information received from pixels in a particular tone contour region is increased. The value of the light information may be increased by adding a predefined value to the received light information, and/or may be scaled by a suitable predefined value. For example, the initial value of the light information from pixels in tone contour region 308 may be increased by 10%. Thus, light information from a pixel that has an initial value corresponding to 0% exposure would be adjusted to a 10% exposure. Light information from a pixel that has an initial value corresponding to 3% exposure would be adjusted to a 13% exposure.
After exposure adjustment by embodiments of the tone contour system 400, the captured image when later displayed will show additional detail in the adjusted regions of the captured image. In the simplified example of
Embodiments employing the controller 426 allow a user to selectively activate the tone contour system 400. When actuated, the tone contour system 400 displays the contour information to the user. If not actuated, the image capture device captures the image using light information at the initial exposure. In other embodiments, activation of the tone contour system 400 is implemented as a selectable feature on a menu system or the like.
Some of the above-described embodiments of the tone contour system 400 adjust the exposure of pixels categorized into the tone contour regions by adjusting the exposure value of light information received from pixels in photosensor 410. That is, the raw data received from the pixels of photosensor 410 are adjusted before image compression. Thus, a voltage level or a value corresponding to a voltage level from an individual pixel may be adjusted.
Other embodiments adjust light information at other points in the data processing pipeline. For example, but not limited to, light information from a plurality of pixels may be processed into data having eight (8) bits or sixteen (16) bits representing red, blue and green color intensities detected by that group of pixels. Accordingly, portions of the 8 bit or 16 bit data are adjusted by embodiments of the tone contour system 400.
In one embodiment, captured image data is communicated to the memory 408 from processor 406, and stored in the captured image data region 414. In another embodiment, digital image data is transferred to a detachable memory (not shown) or a remote memory (not shown). During later stages of image data processing, selected information may be saved to reduce the amount of memory used for storing captured image data. For example, if a less detailed image is acceptable, the user may instruct the digital camera 402 to store images using a smaller data file size. In one such embodiment, light information from relatively large groups of pixels are averaged together into a single light information value. In other embodiment, some light information is retained while other light information is discarded. After such processing, embodiments of the tone contour system 400 may then adjust the image data such that overexposed and/or underexposed portions of an image are adjusted.
As another example, contour information may be determined from compressed Joint Experts Photographics Group (JPEG) formatted data, and then the JPEG data may be adjusted such that overexposed and/or underexposed portions of an image are adjusted. It is appreciated that any form of data file or data format may be processed such that image data is adjusted by embodiments of the tone contour system 400 such that overexposed and/or underexposed portions of an image are adjusted.
In yet another embodiment, captured image data is transferred to another processing device such that an embodiment of the tone contour system 400 adjusts overexposed and/or underexposed portions of an image.
Tone contour system 400 illustrated in
As described above, light information from a plurality of pixels are analyzed to determine groups of pixels having similar exposure, thereby determining tone contour regions. In another embodiment, an algorithm identifies coarse exposure attributes of the preview image. The coarse exposure attributes are analyzed using a plurality of coarse grids to identify an average exposure for the grids. Grids having similar average exposure levels, or exposure levels within a predefined common range, are grouped to define tone contour regions as described above. Accordingly, this embodiment of the tone contour system 400 may more quickly determine and display contour information for a preview image.
In one embodiment, the tone contour system 400 is implemented in digital camera 402 as a selectable feature. Controller 426 is configured such that actuation of controller 426 enables or disables the operation of the tone contour system 400. Examples of controller 426 include, but are not limited to, a push-button, a toggle-switch, a multi-position sensing device configured to sense a plurality of positions, a touch sensitive device or a light sensitive device. In another embodiment, the functionality of controller 426 may be alternatively implemented as a menu displayed on display 428.
In one embodiment, viewfinder 422 is an optical based device configured to align the view of the user with the image sensed by photosensor 410. The contour information is then displayed within viewfinder 422 concurrently with a view of the preview image using a heads-up display device or other similarly suited display means. In another embodiment, viewfinder 422 includes a relatively small electronic display that resides within the viewfinder 422. The display residing in the viewfinder 422 is configured to display the image sensed by photosensor 410. Another embodiment may employ its own photosensor element that is aligned with photosensor 410.
The process begins at block 602. At block 604, light information corresponding to light detected by a plurality of pixels is received. At block 606, light information to determine a light value for each of the plurality of pixels is analyzed. At block 608, contour information corresponding to a plurality of groups of adjacent pixels, wherein each group of adjacent pixels is characterized by a unique predetermined range of light values, is determined. At block 610, the contour information is communicated to a display. The process ends at block 612.
Embodiments of the invention implemented in memory 408 (
It should be emphasized that the above-described embodiments are merely examples of implementations. Many variations and modifications may be made to the above-described embodiments. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
