Autoexposure methodology in a digital camera

Abstract

An improved method for automatic exposure control at a digital camera is described. In response to a user request for capture of a digital image, image data is captured on the camera's image sensor. The captured image data includes pixel values of brightness of pixels of the image sensor. A histogram of pixel values is generated based upon brightness of pixels of the image sensor. The histogram that is generated is then evaluated to determine whether an image is overexposed or underexposed. Coarse exposure settings for capture of a digital image are automatically generated based upon scaling the histogram of pixel values towards the midpoint of a pre-selected interval. The coarse exposure settings and image data that have been captured are evaluated to determine whether light sources are present. In the event that light sources are determined not to be present, the coarse exposure settings are adjusted based upon not scaling said histogram beyond the largest histogram value.

Description

COMPUTER PROGRAM LISTING APPENDIX

[0002] A Computer Program Listing Appendix, containing six (6) total files on compact disc, is included with this application.

Copyright Notice

[0003] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

BACKGROUND OF THE INVENTION

[0004] 1. Field of the Invention

[0005] The present invention relates generally to digital image processing and, more particularly, to improved techniques for automatic exposure control when capturing digital images.

[0006] 2. Description of the Background Art Today, digital imaging, particularly in the form of digital cameras, is a prevalent reality that affords a new way to capture photos using a solid-state image sensor instead of traditional film. A digital camera functions by recording incoming light on some sort of sensing mechanism and then processes that information (basically, through analog-to-digital conversion) to create a memory image of the target picture. A digital camera's biggest advantage is that it creates images digitally thus making it easy to transfer images between all kinds of devices and applications. For instance, one can easily insert digital images into word processing documents, send them by e-mail to friends, or post them on a Web site where anyone in the world can see them. Additionally, one can use photo-editing software to manipulate digital images to improve or alter them. For example, one can crop them, remove red-eye, change colors or contrast, and even add and delete elements. Digital cameras also provide immediate access to one's images, thus avoiding the hassle and delay of film processing. All told, digital photography is becoming increasingly popular because of the flexibility it gives the user when he or she wants to use or distribute an image.

[0007] The defining difference between digital cameras and those of the film variety is the medium used to record the image. While a conventional camera uses film, digital cameras use an array of digital image sensors. When the shutter opens, rather than exposing film, the digital camera collects light on an image sensor, a solid-state electronic device. The image sensor contains a grid of tiny photosites that convert light shining on them to electrical charges. The image sensor may be of the charged-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) varieties. Most digital cameras employ charge-coupled device (CCD) image sensors, but newer cameras are using image sensors of the complimentary metal-oxide semiconductor (CMOS) variety. Also referred to by the acronym CIS (for CMOS image sensors), this newer type of sensor is less expensive than its CCD counterpart and requires less power.

[0008] During camera operation, an image is focused through the camera lens so that it will fall on the image sensor. Depending on a given image, varying amounts of light hit each photosite, resulting in varying amounts of electrical charge at the photosites. These charges can then be measured and converted into digital information that indicates how much light hit each site which, in turn, can be used to recreate the image. When the exposure is completed, the sensor is much like a checkerboard, with different numbers of checkers (electrons) piled on each square (photosite). When the image is read off of the sensor, the stored electrons are converted to a series of analog charges which are then converted to digital values by an Analog-to-Digital (A-to-D) converter, which indicates how much light hit each site which, in turn, can be used to recreate the image.

[0009] As with conventional (i.e., non-digital) cameras, the exposure (i.e., the amount of light that reaches the image sensor) determines how light or dark the resulting photograph will be. When the shutter opens, light (reflected from the subject and focused by the lens) strikes the image sensor inside the camera. If too much light strikes the image sensor, the photograph will be overexposed. Overexposure may result in a washed out and faded looking picture. Too little light, on the other hand, produces an underexposed photograph that may be dark and lacking in details, especially in shadow areas. In conventional cameras, the amount of light that exposes the image is controlled by adjusting either the aperture (the size of the opening through which light enters the camera) or the shutter speed (the length of time light is allowed to enter). These adjustments may be made manually by the camera user or may be made by an automatic exposure mechanism of the camera which makes one or both of these adjustments for the user. Automatic exposure control (or autoexposure) is a feature desired by many camera users, particularly those that are not photography experts, as automatic exposure control enables users to take photographs without having to manually adjust the lens aperture and shutter speed (exposure time). Moreover, because autoexposure is a feature that has been available for a number of years on conventional film cameras, many users expect autoexposure to be provided as a standard feature of any digital camera they may purchase.

[0010] Automatic exposure control systems of traditional film cameras typically use light sensors and analog circuits to determine the correct exposure settings for image capture. As described above, an automatic exposure control system in a conventional camera automatically adjusts the aperture and shutter speed settings based upon the amount of light detected by a light sensor. After the system measures brightness (i.e., how light or dark the scene is), it calculates and sets the aperture and shutter speed of the camera to render this level of light as “middle gray” (i.e., the middle of a grayscale spectrum from white to black) in the photograph. If the amount of light is high, the aperture setting and/or shutter speed (exposure time) will be reduced. In darker conditions, the system will adjust the aperture setting and/or exposure time to be larger. Current digital cameras perform a similar process digitally by recording light signals on a sensor and applying an algorithm to the signal to establish the exposure settings. The sensor used may be the digital camera's main CMOS (or CCD) sensor itself or, alternatively, a separate light sensor included as part of the digital camera.

[0011] Although most current digital cameras use a separate light sensor for automatic exposure control, there are drawbacks to this approach. One drawback is cost. Use of an external light sensor adds another component to the camera that increases the overall cost of manufacture of the camera. In addition, a light sensor may also increase size of the camera and its power consumption. Both of these factors can be significant considerations, particularly for smaller cameras where size and power requirements are critical design considerations. For these reasons, an autoexposure approach that does not require a separate light sensor is preferred.

[0012] Another problem is that existing autoexposure systems typically rely on the use of a flash device for lower light conditions. However, use of a flash device also adds to the cost of manufacture of the camera as well as increasing its size and its power requirements. One would think with present-day digital technology and scale, one could create a digital camera that is extremely small and portable, particularly since a digital camera is not constrained by the physical constraints of traditional photographic film. This is not the case today, however. As it turns out, the whole process of capturing light and generating a color digital image, such as with a digital camera, is a very compute-intensive process. The process of recording an image on photographic film, in comparison, relies on straightforward chemical reactions, all without the need for computing resources. A digital image, however, entails a process of converting light into electrical signals, converting those electrical signals into digital or binary information, arranging that information into a visual representation, applying various digital filters and/or transformations, interpolating color from that representation, and so forth and so on. The process of rendering a meaningful digital picture is a compute-intensive undertaking, roughly equivalent in processing power to that required today for a desktop workstation, yet done so within the confines of a handheld portable device.

[0013] The upshot of this substantial processing requirement is that, paradoxically, digital cameras today are relatively bulky devices since they require relatively large batteries to support their processing needs. This is easily seen today in camera designs. Many digital camera designs employ either large custom lithium batteries or, alternatively four to six AA batteries. Even with these large battery arrangements, digital cameras today have relatively short battery lives, such that the digital camera user is required to change out batteries at frequent intervals. Perhaps the biggest drawback of such an approach, however, is the added bulk imparted to the camera itself with such a design. Today, most of the weight of a digital camera is attributable to its batteries. Thus, present-day digital cameras, being constrained by their battery requirements, are generally no smaller or portable than their non-digital counterparts (e.g., standard 35 mm film camera). And the smallest cameras today still remain film-based cameras, not digital ones, due in large part to the battery constraints of digital cameras.

[0014] Because of these size and power consumption considerations, an improved approach for automatic exposure control is needed that does not require a separate light sensor or a flash device for low light conditions. An autoexposure mechanism that uses the digital camera's image sensor and does not require the camera to be equipped with a flash device is required in order to enable the production of smaller, less expensive digital cameras. Avoiding the manufacturing costs and power consumption requirements of including a flash and external light sensor on a digital camera enables production of smaller, more portable cameras that are less expensive and have reduced battery requirements.

[0015] Current autoexposure systems which rely on using only the main (e.g., CMOS) image sensor of a digital camera may, however, have problems in properly capturing an image in low light conditions without using a flash. Without flash, a longer exposure (processing) time is required for darker scenes in order to avoid having an underexposed picture. This longer exposure time may result in a blurred picture as the camera or the subject may move while the image is still being processed. In low light settings the processing time required without flash may be impractical using current digital camera autoexposure methods. One possible approach to this problem is to pre-compute an exposure solution by periodically or constantly evaluating the light conditions to provide for quicker processing of a correct exposure solution. A drawback to this approach is that these continual calculations may result in considerable power consumption, thereby requiring larger batteries and/or more frequent battery changes or recharging.

[0016] Another problem with current autoexposure systems is that these systems often rely on spot metering in which only a small area in the middle of the image is evaluated. Rather than evaluating an entire image, this approach evaluates only specific spots or areas, typically a small area in the middle area of the image shown in the viewfinder. Spot metering may be problematic if, for instance, a dark object is located off center against a very light background. In this instance, the dark object may not be exposed properly because it is not located in the area the meter is emphasizing. Because only a portion of the scene is examined, other regions that are significantly lighter or darker may be underexposed when a spot metering technique is utilized. For these reasons an approach that examines the entire image is preferred as it more reliably handles a wider range of situations.

[0017] What is required is an improved automatic exposure control solution that meters all regions of an image to reliably handle a wide range of conditions, from low light, indoor scenes to outdoor, bright sun settings. This solution should not require the digital camera to be equipped with a flash or a separate light sensor, enabling its use in a wide range of devices, including small, inexpensive digital cameras in which cost, size and power consumption requirements are critical. Ideally, the solution should also enable automatic exposure control in digital cameras with a fixed lens aperture. The present invention provides a solution that fulfills these and other needs.

Glossary

[0018] The following definitions are offered for purposes of illustration, not limitation, in order to assist with understanding the discussion that follows.

[0019] CCD: CCD is short for Charge-Coupled Device, an instrument whose semiconductors are connected so that the output of one serves as the input of the next. Until recent years, CCDs were the only image sensors used in digital cameras. Each CCD consists of hundreds of thousands of cells known as photosites or photodiodes. A Charge-Coupled Device (CCD) gets its name from the way the charges on its photosites (pixels) are read after an exposure. After the exposure the charges on the first row are transferred to a place on the sensor called the read out register. From there, the signals are fed to an amplifier and then on to an analog-to-digital converter. Once the row has been read, its charges on the readout register row are deleted, the next row enters, and all of the rows above march down one row. The charges on each row are “coupled” to those on the row above so when one moves down, the next moves down to fill its old space.

[0020] CMOS: CMOS is an abbreviation of Complementary Metal Oxide Semiconductor, a widely used type of semiconductor. CMOS image sensors, like CCD image sensors, capture light on a grid of small photosites on their surfaces, however they process images differently than CCDs and are manufactured using different techniques. CMOS semiconductors use both NMOS (negative polarity) and PMOS (positive polarity) circuits. Since only one of the circuit types is on at any given time, CMOS chips require less power than chips using just one type of transistor. This makes them particularly attractive for use in battery-powered devices, such as portable computers and digital cameras. Another advantage of CMOS semiconductors is that they may be manufactured using established high-yield techniques and, therefore, are significantly less expensive to fabricate than specialist CCDs. Furthermore, whilst CCDs have the single function of registering where light falls on each of the hundreds of thousands of sampling points, CMOS can be loaded with a host of other tasks, such as analog-to-digital conversion, load signal processing, and handling white balance and camera controls.

[0021] Photosites: Photosites or photodiodes are essentially light-collecting wells that convert optical information into an electric charge. When light particles known as photons enter the silicon body of the photosite, they provide enough energy for negatively charged electrons to be released. The more light that enters the photosite, the more free electrons are available. Each photosite has an electrical contact attached to it, and when a voltage is applied to this the silicon below each photosite becomes receptive to the freed electrons and acts as a container for them. Thus, each photosite has a particular charge associated with it—the greater the charge, the brighter the intensity of the associated pixel. The photosites on an image sensor actually respond only to light, not to color. Color is typically added to an image by means of red, green and blue filters placed over each pixel.

SUMMARY OF THE INVENTION

[0022] The present invention provides an improved method for automatic exposure control at a digital camera device. The method commences when a user requests capture of a digital image at a digital camera device. In response to a user request, image data is captured on the camera's image sensor. The image data that is captured includes values of brightness of pixels of the image sensor. Next, a profile of the distribution of these values of brightness is generated. In the currently preferred embodiment, the profile is a histogram that is generated based upon distribution of values of brightness in pre-selected ranges. The profile is then evaluated and initial exposure settings for capture of a digital image are automatically generated based upon scaling the distribution of these values of brightness towards the midpoint of a pre-selected interval.

[0023] The initial exposure settings and image data that have been captured are evaluated to determine whether areas of relative brightness (e.g., overhead lights) are present in the scene being captured. In the currently preferred embodiment, determining whether areas of relative brightness are present includes evaluating whether the number of pixels with high brightness values exceeds a pre-selected maximum, as well as determining whether the initial exposure settings exceed a pre-selected threshold. The initial exposure settings are adjusted based upon whether or not areas of relative brightness are determined to be present.

BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG. 1 is a very general block diagram of a digital camera suitable for implementing the present invention.

[0025] FIG. 2A is a block diagram of a conventional digital imaging device.

[0026] FIG. 2B is a block diagram of a conventional onboard processor or computer provided for directing the operation of the digital camera and processing image data.

[0027] FIG. 3 is a high-level block diagram of an automatic exposure control system constructed in accordance with the present invention.

[0028] FIG. 4 is a diagram illustrating three sets of parameter computations made during the first or “coarse” phase of the autoexposure method of the present invention.

[0029] FIG. 5 is a flow diagram illustrating the detailed method steps of the coarse phase of the autoexposure method of the present invention.

[0030] FIG. 6 is a flowchart illustrating the detailed method steps of the fine phase of the autoexposure method of the present invention.

[0031] FIG. 7 is a graph showing how ExpGain is mapped to exposure time and sensor gain for coarse phase autoexposure.

[0032] FIG. 8 is a graph showing how ExpGain is mapped to exposure time and sensor gain for final exposure of an image.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

[0033] The following description will focus on the currently-preferred embodiment of the present invention, which is implemented in a digital camera. The present invention is not, however, limited to any one particular application or any particular environment. Instead, those skilled in the art will find that the system and methods of the present invention may be advantageously employed on a variety of different devices. Therefore, the description of the exemplary embodiment that follows is for purpose of illustration and not limitation.

[0034] I. Digital Camera-based Implementation

[0035] A. Basic Components of Digital Camera

[0036] The present invention may be implemented on a media capturing and recording system, such as a digital camera. FIG. 1 is a very general block diagram of a digital camera 100 suitable for implementing the present invention. As shown, the digital camera 100 comprises an imaging device 120, a system bus 130, and a processor or computer 140 (e.g., microprocessor-based unit). Also shown is a subject or object 150 whose image is to be captured by the digital camera 100. The general operation of these components of the digital camera 100 in capturing an image of the object 150 will now be described.

[0037] As shown, the imaging device 120 is optically coupled to the object 150 in the sense that the device may capture an optical image of the object. Optical coupling may include use of optics, for example, such a lens assembly (not shown) to focus an image of the object 150 on the imaging device 120. The imaging device 120 in turn communicates with the computer 140, for example, via the system bus 130. The computer 140 provides overall control for the imaging device 120. In operation, the computer 140 controls the imaging device 120 by, in effect, telling it what to do and when. For instance, the computer 140 provides general input/output (I/O) control that allows one to coordinate control of the imaging device 120 with other electromechanical peripherals of the digital camera 100 (e.g., flash attachment).

[0038] Once a photographer or camera user has aimed the imaging device 120 at the object 150 (with or without user-operated focusing) and, using a capture button or some other means, instructed the camera 100 to capture an image of the object 150, the computer 140 commands the imaging device 120 via the system bus 130 to capture an image representing the object 150. The imaging device 120 operates, in essence, by capturing light reflected from the object 150 and transforming that light into image data. The captured image data is transferred over the system bus 130 to the computer 140 which performs various image processing functions on the image data before storing it in its internal memory. The system bus 130 also passes various status and control signals between the imaging device 120 and the computer 140. The components and operations of the imaging device 120 and the computer 140 will now be described in greater detail.

[0039] B. Image Capture on Imaging Device

[0040] FIG. 2A is a block diagram of a conventional digital imaging device 120. As shown, the imaging device 120 comprises a lens 210 having an iris, one or more filter(s) 215, an image sensor 230 (e.g., CMOS, CCD, or the like), a focus mechanism (e.g., motors) 241, a timing circuit 242, a signal processor 251 (e.g., analog signal processor), an analog-to-digital (A/D) converter 253, and an interface 255. The operation of these components will now be described.

[0041] In operation, the imaging device 120 captures an image of the object 150 via reflected light impacting the image sensor 230 along optical path 220. The lens 210 includes optics to focus light from the object 150 along optical path 220 onto the image sensor 230. The focus mechanism 241 may be used to adjust the lens 210. The filter(s) 215 preferably include one or more color filters placed over the image sensor 230 to separate out the different color components of the light reflected by the object 150. For instance, the image sensor 230 may be covered by red, green, and blue filters, with such color filters intermingled across the image sensor in patterns (“mosaics”) designed to yield sharper images and truer colors.

[0042] While a conventional camera exposes film to capture an image, a digital camera collects light on an image sensor (e.g., image sensor 230), a solid-state electronic device. The image sensor 230 may be implemented as either a charged-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor. Both CMOS and CCD image sensors operate by capturing light on a grid of small cells known as photosites (or photodiodes) on their surfaces. The surface of an image sensor typically consists of hundreds of thousands of photosites that convert light shining on them to electrical charges. Depending upon a given image, varying amounts of light hit each photosite, resulting in varying amounts of electrical charge at the photosites. These charges can then be measured and converted into digital information. A CCD sensor appropriate for inclusion in a digital camera is available from a number of vendors, including Eastman Kodak of Rochester, N.Y., Phillips of The Netherlands, and Sony of Japan. A suitable CMOS sensor is also available from a variety of vendors. Representative vendors include STMicroelectronics (formerly VSLI Vision Ltd.) of The Netherlands, Motorola of Schaumburg, Ill., and Intel of Santa Clara, Calif.

[0043] When instructed to capture an image of the object 150, the image sensor 230 responsively generates a set of raw image data (e.g., in CCD format for a CCD implementation) representing the captured object 150. In an embodiment using a CCD sensor, for example, the raw image data that is captured on the image sensor 230 is routed through the signal processor 251, the analog-to-digital (A/D) converter 253, and the interface 255. The interface 255 has outputs for controlling the signal processor 251, the focus mechanism 241, and the timing circuit 242. From the interface 255, the image data passes over the system bus 130 to the computer 140 as previously illustrated at FIG. 1. The operations of the computer 140 in processing this image data will now be described.

[0044] C. Image Processing

[0045] A conventional onboard processor or computer 140 is provided for directing the operation of the digital camera 100 and processing image data captured on the imaging device 120. FIG. 2B is a block diagram of the processor or computer 140. As shown, the system bus 130 provides connection paths between the imaging device 120, an (optional) power management 262, a processor (CPU) 264, a random-access memory (RAM) 266, an input/output (I/O) controller 280, a non-volatile memory 282, a removable memory interface 283, and a liquid crystal display (LCD) controller 290. Removable memory 284 connects to the system bus 130 via the removable memory interface 283. Alternately, the camera 100 (and therefore the onboard computer 140) may be implemented without the removable memory 284 or the removable memory interface 283. The power management 262 communicates with the power supply 272. Also illustrated at FIG. 2B is a camera user interface 295 which is electrically connected to the LCD controller 290 and the input/output controller 280. Each of these components will now be described in more detail.

[0046] The processor (CPU) 264 typically includes a conventional processor device (e.g., microprocessor) for controlling the operation of camera 100. Implementation of the processor 264 may be accomplished in a variety of different ways. For instance, the processor 264 may be implemented as a microprocessor (e.g., MPC823 microprocessor, available from Motorola of Schaumburg, Ill.) with DSP (digital signal processing) logic blocks, memory control logic blocks, video control logic blocks, and interface logic. Alternatively, the processor 264 may be implemented as a “camera on a chip (set)” using, for instance, a Raptor II chipset (available from Conextant Systems, Inc. of Newport Beach, Calif.), a Sound Vision Clarity 2, 3, or 4 chipset (available from Sound Vision, Inc. of Wayland, Mass.), or similar chipset that integrates a processing core with image processing periphery. Processor 264 is typically capable of concurrently running multiple software routines to control the various processes of camera 100 within a multithreaded environment.

[0047] The digital camera 100 includes several memory components. The memory (RAM) 266 is a contiguous block of dynamic memory which may be selectively allocated to various storage functions. Dynamic random-access memory is available from a variety of vendors, including, for instance, Toshiba of Japan, Micron Technology of Boise, Id., Hitachi of Japan, and Samsung Electronics of South Korea. The non-volatile memory 282, which may typically comprise a conventional read-only memory or flash memory, stores a set of computer-readable program instructions to control the operation of the camera 100. The removable memory 284 serves as an additional image data storage area and may include a non-volatile device, readily removable and replaceable by a camera 100 user via the removable memory interface 283. Thus, a user who possesses several removable memories 284 may replace a full removable memory 284 with an empty removable memory 284 to effectively expand the picture-taking capacity of the camera 100. The removable memory 284 is typically implemented using a flash disk. Available vendors for flash memory include, for example, SanDisk Corporation of Sunnyvale, Calif. and Sony of Japan. Those skilled in the art will appreciate that the digital camera 100 may incorporate other memory configurations and designs that readily accommodate the image capture and processing methodology of the present invention.

[0048] The digital camera 100 also typically includes several interfaces for communication with a camera user or with other systems and devices. For example, the I/O controller 280 is an interface device allowing communications to and from the computer 140. The I/O controller 280 permits an external host computer (not shown) to connect to and communicate with the computer 140. As shown, the I/O controller 280 also interfaces with a plurality of buttons and/or dials 298, and an optional status LCD 299, which in addition to the LCD screen 296 are the hardware elements of the user interface 295 of the device. The digital camera 100 may include the user interface 295 for providing feedback to, and receiving input from, a camera user, for example. Alternatively, these elements may be provided through a host device (e.g., personal digital assistant) for a media capture device implemented as a client to a host device. For an embodiment that does not need to interact with users, such as a surveillance camera, the foregoing user interface components may not be required. The LCD controller 290 accesses the memory (RAM) 266 and transfers processed image data to the LCD screen 296 for display. Although the user interface 295 includes an LCD screen 296, an optical viewfinder or direct view display may be used in addition to or in lieu of the LCD screen to provide feedback to a camera user. Components of the user interface 295 are available from a variety of vendors. Examples include Sharp, Toshiba, and Citizen Electronics of Japan, Samsung Electronics of South Korea, and Hewlett-Packard of Palo Alto, Calif.

[0049] The power management 262 communicates with the power supply 272 and coordinates power management operations for the camera 100. The power supply 272 supplies operating power to the various components of the camera 100. In a typical configuration, power supply 272 provides operating power to a main power bus 278 and also to a secondary power bus 279. The main power bus 278 provides power to the imaging device 120, the I/O controller 280, the non-volatile memory 282, and the removable memory 284. The secondary power bus 279 provides power to the power management 262, the processor 264, and the memory (RAM) 266. The power supply 272 is connected to batteries 275 and also to auxiliary batteries 276. A camera user may also connect the power supply 272 to an external power source, as desired. During normal operation of the power supply 272, the main batteries 275 provide operating power to the power supply 272 which then provides the operating power to the camera 100 via both the main power bus 278 and the secondary power bus 279. During a power failure mode in which the main batteries 275 have failed (e.g., when their output voltage has fallen below a minimum operational voltage level), the auxiliary batteries 276 provide operating power to the power supply 276. In a typical configuration, the power supply 272 provides power from the auxiliary batteries 276 only to the secondary power bus 279 of the camera 100.

[0050] The above-described system 100 is presented for purposes of illustrating the basic hardware underlying a media capturing and recording system (e.g., digital camera) that may be employed for implementing the present invention. The present invention, however, is not limited to just digital camera devices but, instead, may be advantageously applied to a variety of devices capable of supporting and/or benefiting from the methodologies of the present invention presented in detail below.

[0051] II. Improved Method for Automatic Exposure Control in a Digital Camera

[0052] A. Overview

[0053] Automatic exposure control or “autoexposure” is the process used by a camera to adjust its internal exposure parameters to achieve correct picture exposure. Current autoexposure methods in digital cameras use one of two general approaches to measure brightness. The first approach uses light sensors to measure ambient light and accordingly adjust internal image sensor parameters to achieve correct exposure for the measured brightness. The second approach uses the camera's image sensor itself in an iterative process to achieve proper exposure of an image. Using the camera's image sensor for automatic exposure control avoids the additional cost and expense of utilizing an external light sensor to measure brightness. In either case, both types of automatic exposure control systems algorithmically determine the correct parameters for optimal “pixel exposure” of pixels of the image sensor of a digital camera.

[0054] The “pixel exposure” of pixels of the image sensor in a digital camera is influenced by several variables. First, variations in the aperture of the lens can influence exposure, as pixel exposure is directly proportional to the area of the aperture. A second variable is exposure time. The longer the exposure time, the greater the exposure value accumulated at each pixel of the image sensor. Thus, pixel exposure is also directly proportional to the exposure time. A third variable typical in digital cameras is a sensor analog gain parameter. Existing autoexposure methods typically use analog amplifiers, usually one per color channel, on the image sensor of a digital camera in order to boost the detected exposure. This sensor analog gain (or amplification) may be applied uniformly to all pixels as “pseudo” exposure control to boost the signal, although in reality analog gain does not affect the light actually accumulated at a sensor pixel. Nonetheless, sensor gain is quite useful for automatic exposure control. When an image is underexposed, amplification enables boosting of the detected exposure. This boosting of the image before quantization also enables the “background noise” in the resulting image to be reduced.

[0055] The autoexposure method of the present invention uses a coarse-to-fine strategy for exposure control. Initially, a fast, coarse search is used to roughly determine the parameters needed for a reasonable exposure. In this coarse phase, captured sensor image data is mapped into a histogram (i.e., profiled) to evaluate the distribution of pixels with high (i.e., bright or saturated) and low (i.e., dark) pixel values. Based upon evaluation of this histogram, the exposure settings are adjusted so that the captured image is not underexposed or overexposed. In the currently preferred embodiment, this includes adjusting exposure time and sensor gain in order to scale the captured pixel values towards the middle of the range represented by the histogram based upon pre-selected preferred levels. Next, a fine computation is used to detect whether or not areas of relative brightness, such as overhead lights or other light sources, are present. Based upon this evaluation, adjustments are made to the coarse exposure settings to determine the final exposure parameters. The exposure settings resulting from these coarse and fine computations may then be utilized for final exposure of an image.

[0056] The present solution has several advantages. First, because the method is fast, it suffices to execute it after the shutter button of the camera is pressed to take a picture. There is no need to continuously run the sensor (and make the associated autoexposure calculations) which would lead to significant power consumption and a drain on the battery. Second, the methodology of the present invention uses the image sensor of a digital camera and does not require a separate external light sensor or a flash attachment. Thus, the present approach is appropriate for devices having limited power resources, including small, portable cameras as well as camera modules embedded in mobile devices such as handheld computers, personal digital assistants, or the like. Furthermore, in its presently preferred embodiment, the autoexposure method of the present invention controls exposure by adjusting exposure time and sensor gain. This enables the method to be used in a digital camera having a lens with a fixed aperture, thereby avoiding the costs (both manufacturing costs and power consumption requirements) inherent in utilizing a motor to control lens aperture. However, those skilled in the art will appreciate that the methodology of the present invention may also be used in a camera having an adjustable lens aperture and/or a flash attachment.

[0057] B. Block Diagram of Automatic Exposure Control System

[0058] FIG. 3 is a high-level block diagram of an automatic exposure control system constructed in accordance with the present invention. As shown, incoming light from a scene is focused through a lens 310 onto an image sensor 320. Image data captured on the image sensor 320 is passed to an autoexposure (automatic exposure control) system 330 for evaluation and feedback. The autoexposure system 330 initially performs a fast and quick evaluation in a “coarse” phase of the present method. This coarse phase is an iterative process which enables a rough exposure solution to be generated. Based upon the results of the coarse phase, the autoexposure system 330 may adjust the exposure time 337 and/or sensor gain 333 to enable capture of an image that will not be overexposed or underexposed. The image data initially captured may then be discarded. In the “fine” phase of the method, additional computations are made by the autoexposure system 330 in an effort to optimize the exposure settings for capture of image data based upon whether or not areas of relative brightness, such as overhead light sources, are detected in a scene. Based upon the results of this fine phase of evaluation, additional adjustments may be made to exposure time 333 and sensor gain 337 for optimal exposure of the image from this scene. The coarse and fine portions of the improved autoexposure method of the present invention will now be described in greater detail.

[0059] C. Overview of Coarse Phase of Autoexposure Process

[0060] The coarse phase of the autoexposure method of the present invention is a fast and iterative process for roughly determining the exposure parameters that should be used to capture an image. In this phase, rough image data is captured and quickly evaluated to determine whether the image is overexposed or underexposed. An overexposed image with a large number of over saturated pixels is very difficult to adjust mathematically as over saturation results in loss of information about the scene being captured. On the other hand, if an image is underexposed, then the information that is available about the scene may be insufficient to generate a reasonable result. Accordingly, quickly evaluating whether an image is at one end of the spectrum (e.g., very bright) or the other (e.g., very dark) enables the correct amount of information to be gathered to generate a better final result in the fine phase of the process. By quickly evaluating the image in the coarse phase, an exposure time can be determined to enable capture of a reasonable image. This can then be fine tuned in the fine portion of the process. The computations made during the coarse phase of the present method will now be described.

[0061] D. Computations Made During Coarse Phase

[0062] FIG. 4 is a diagram illustrating the three sets of parameter computations made during each iteration of the “coarse” phase of the autoexposure method of the present invention. As shown at FIG. 4, the coarse phase of the method starts with capture of an image using the current exposure time and sensor gain (referred to hereinafter together as “ExpGain” or ExpGain) settings. ExpGain is the product of exposure time and sensor gain for an image which determines its exposure. In the coarse phase of the method, the sensor gain component of ExpGain is generally larger than in typical final exposure, as the purpose of the coarse phase is to quickly generate a rough result. A shorter exposure time is required during the coarse phase so that image capture is not delayed. A lengthy exposure time is avoided as it might result in a blurred image if, for example, the subjects of the photograph moved while the exposure settings are being calculated. The exposure time and sensor gain settings used during coarse autoexposure are described in more detail below.

[0063] As shown in box 401 at the top of FIG. 4, using raw captured sensor image data as input, a histogram is generated and certain parameters, including Bin—95 (95th percentile) and Bin_max (99th percentile) are calculated. These parameters and their calculation are described in greater detail below. The histogram that is generated is an array that stores counts of pixel values that fall into particular ranges. For example, assume that a particular digital camera has an 8-bit, VGA 640×480 sensor with a total of 307,200 pixels. In this situation, pixels having the range of legal values 0-255, may be distributed into a number of different subgroups or “bins” for purposes of mapping the values to a histogram. For example, 256 values (0-255) may be mapped into a total of 32 bins (bin 0 to bin 31). Bin 0 would count all pixels that fall in the range from 0 to 7, bin 1 would count all pixels that fall into the range 8 to 15, and so forth. The final bin (bin 31) would count all pixels in the range from 248 to 255. The histogram provides a useful way of representing the captured pixel values and evaluating whether an image is overexposed or underexposed. For example, if a large portion of the values were at the high end of the range (e.g., in bins 28, 29, 30, and 31), the image is likely to be overexposed (over saturated). Similarly, if almost all of the values are at the low end of the ranges (e.g., bins 0, 1, 2, and 3) the image is likely to be underexposed. Accordingly, the coarse phase of the autoexposure method of the present invention evaluates whether the captured sensor image is at one of these extremes so that appropriate adjustments may be made before image capture. In particular, this evaluation includes computation of certain parameters about the histogram, including cumulative percentiles Bin—95 and Bin_max.

[0064] Bin—95 is defined as the bin of the histogram for which the cumulative count of pixels equals (or just exceeds) 95 percent of the total number of pixels. Bin—95 is calculated by counting the total number of pixels in each bin, starting with bin 0. For example, if pixels are counted starting with those in bin 0, followed by those in bin 1, and so forth, then the bin for which the cumulative count of pixels just exceeds 95 percent of the total number of pixels will correspond to Bin—95. Bin_max is calculated in the same manner as bin 95, except that the cumulative percentile used for Bin_max is 99.5 percent of the total number of pixels. The number of saturated pixels is also counted as part of these calculations and the result of this count is retained as the count_saturated_pixels parameter as shown at FIG. 4.

[0065] Next, as shown in box 402 in the middle of FIG. 4, an appropriate target range in which Bin—95 should be located is determined. This target range, referred to as the target bin—95 interval is intended to ensure that the image is neither overexposed nor underexposed. The applicable target range for a particular scene is determined by taking the ExpGain (i.e., the product of exposure and sensor gain) of the captured image as input and comparing it to a threshold. One of two pre-selected ranges is used based upon whether ExpGain is greater than or less than this threshold. A lower range (Bin—95_lo_min, Bin—95_lo_max) is used for bright scenes (Low ExpGain). A higher range is used for dark scenes (High ExpGain). The appropriate legal range for the target bin—95 interval (Bin13 95_min, Bin—95_max) is generated as shown at FIG. 4.

[0066] A limit for saturated pixels is then determined based upon whether the scene has a lower or higher ExpGain product. In other words, a determination is made of how much saturation to allow. More saturation is allowed in low light scenes which have larger ExpGain products, as these are typically indoor scenes with overhead lights causing saturation in certain areas of the image. Less saturation is allowed in brighter scenes with lower ExpGain products, such as outdoor, daytime scenes. Based upon the current ExpGain product and pre-determined limits for low ExpGain(limit_sat_min, ExpGain_lsmin) and higher ExpGain (limit_sat_max, ExpGain_lsmax), the saturation limit (limit_sat) for the image is determined as shown in box 403 at FIG. 4. The application and use of the above-described parameters during iterations of the coarse phase of the autoexposure method of the present invention will now be described.

[0067] E. Detailed Method Steps of Coarse Phase of Autoexposure Method

[0068] FIG. 5 is a flow diagram illustrating the detailed method steps of the coarse phase of the autoexposure method of the present invention. As described above, the coarse phase is an iterative process to determine rough exposure settings for capture of an image. Image data may be captured by the image sensor and evaluated by the autoexposure system several times before reasonable coarse exposure settings are determined. Accordingly, at step 501, the method commences with an evaluation as to whether the number of iterations exceeds a pre-selected maximum. As the method commences after a camera user has issued a command (e.g., pressed the button on the camera) to take a picture, there is only a limited amount of time available to determine exposure settings appropriate for image capture. Accordingly, if the number of iterations exceeds the pre-selected maximum, the coarse phase of the automatic exposure method is terminated. In the event of termination as a result of exceeding the maximum number of iterations, the method may proceed to the fine exposure phase using the current exposure settings. An error message or signal may also be displayed to the user in conjunction with the capture of an image. In the currently preferred embodiment, this error message or signal may be displayed to the user as a warning that the image that is being captured may be overexposed or underexposed. In an alternative embodiment, an error message is displayed to the user in lieu of capturing an image.

[0069] At step 502, image data is captured on the image sensor using current ExpGain settings. The ExpGain settings used may be an initial setting or may result from feedback from a prior iteration. In the currently preferred embodiment, initial ExpGain settings are pre-selected settings that are automatically applied by the autoexposure system. In an alternative embodiment, ExpGain settings used for final exposure of an image are stored in cache memory. In this alternative embodiment, the initial ExpGain settings used by the autoexposure system in the coarse phase of the method are based upon the stored ExpGain settings used for the most recent picture taken by the camera. In either case, after image data has been captured, a histogram is generated and several parameters, including Bin—95 and Bin_max, are calculated as described above. A count of the number of saturated pixels is also made at step 502. The parameters computed may then be evaluated in the following steps to establish rough exposure settings for capture of the image.

[0070] A check for overexposure is made at step 503. The Bin—95 parameter computed in step 502 is compared to a maximum limitation (bin—95_max). Recall that bin—95_max is selected from a higher range or a lower range based upon the current ExpGain settings used to capture the image. If Bin—95 is less than or equal to the maximum limitation (bin—95_max), the method proceeds to step 505. Otherwise, if Bin—95 exceeds this limitation, the exposure is excessive and the method proceeds instead to step 504. At step 504, the ExpGain product is adjusted (reduced) in an effort to scale Bin—95 of the histogram towards the midpoint of the target interval. The adjusted ExpGain product is calculated by multiplying the current ExpGain product by a sum equal to the following: (bin—95_min + bin—95_max/2)/Bin 95. After adjustment of the ExpGain product, steps 501 to 503 are repeated, including capture of additional image data using the adjusted settings.

[0071] If Bin—95 is less than or equal to the maximum limitation at step 503, then the image is not overexposed and the method proceeds to step 505. At step 505 a check is made for underexposure. If Bin—95 is greater than (or equal to) the minimum limitation (bin—95_min), then the image is not underexposed and the method proceeds to step 509. However, if Bin—95 is less than the minimum (bin—95_min), the method instead proceeds to step 506 and either step 507 or 508. Step 506 checks to determine if an underexposed image contains a large number of saturated pixels. Specifically, the count of saturated pixels in the image (count_saturated_pixels) is compared to a threshold (sat_pixel_threshold). If the count of saturated pixels is greater than the threshold, then the method proceeds to step 507 to reduce ExpGain as a significant portion of the image is over saturated. In this situation, ExpGain is reduced in order to bring out the highlights in the image. As shown, ExpGain is adjusted by multiplying the current ExpGain setting by a fraction, the numerator of which is the saturated picture threshold and denominator of which is the number of saturated pixels in the image. The method then loops back to step 501 based on the adjusted ExpGain setting.

[0072] If the number of saturated pixels does not exceed the threshold at step 506, the method proceeds to step 508 to increase ExpGain. The ExpGain value is adjusted in order to move Bin—95 of the histogram to the midpoint of the preferred interval. As shown at step 508, the adjusted ExpGain product is calculated by multiplying the current ExpGain value by a sum equal to the following: (bin—95_min + bin—95_max/2)/Bin—95. Observe that this formula is the same formula that is used to decrease ExpGain when an image is overexposed. However, in this instance of underexposure, the same formula serves to increase ExpGain. In either case, the method attempts to scale the histogram (representing brightness of pixels of the image) from either end of the range (overexposed or underexposed) toward the middle of the range to enable capture of an appropriate amount of image data. After the ExpGain setting is adjusted, the method loops back stating with step 501, and image data is captured using the new setting.

[0073] If the check at step 503 verifies that an image is not overexposed, and step 505 verifies that it is not underexposed, the method proceeds to step 509. This step evaluates whether the image contains a large number of saturated pixels. The number of saturated pixels in the image (count_saturated_pixels) is compared to the saturation limit. Recall from FIG. 4 that a saturation limit is selected from a lower range or a higher range based upon whether the current ExpGain product is higher or lower. In the event that the number of saturated pixels exceeds the saturation limit, the method proceeds to step 510 to reduce ExpGain. As shown at step 510, the ExpGain setting is reduced by multiplying the current ExpGain setting by the saturation limit (limit_sat) and dividing it by the count of saturated pixels (count_saturated_pixels). The method then again loops through the prior steps commencing with step 501. On the other hand, if the image is not found to be over saturated at step 509 (i.e., if the number of saturated pixels is less than the maximum), the coarse phase terminates and the second phase of the autoexposure measure of the present invention commences.

[0074] F. Fine Phase of Autoexposure Method

[0075] As described above, the coarse (or first) phase of the method only determines an adequate exposure by attempting to adjust the exposure time and sensor gain settings (ExpGain) so that an image is not overexposed or underexposed. Any image data captured during the coarse phase is discarded and the second or fine phase of the automatic exposure control method of the present invention attempts to determine a final exposure settings that will be used to capture a particular image. In determining final exposure settings to be used to capture an image, the fine phase tries to distinguish between exposures with areas of relative brightness (e.g., overhead lights) and those without such areas of relative brightness. When light sources, such as overhead light fixtures or street lamps, are not present in a scene, it is important to preserve the highlights in the scene. However, when such light sources are not present in a scene, it is equally important to capture an image that is sufficiently bright. Accordingly, the fine phase involves several calculations to detect areas of relative brightness, which may indicate the presence of overhead lights or other light sources. Based upon these fine computations, the final exposure settings to be used for capturing and storing an image are determined.

[0076] The parameters in the following table are evaluated as part of the fine phase of automatic exposure control:

	Example
Parameter	Description	values
Bin_95	The 95th percentile bin	12
Target_95	The target bin for the 95th percentile	20
	point
Scale_95	ExpGain Scale factor to get to	20/12 =
	Target_95 =	1.667
	Target_95/Bin_95
Bin_max	99.5th percentile	15
Target_max	The target bin for the 99.5th	28
	percentile point.
Scale_max	ExpGain Scale factor to get to	28/15 =
	Target_max =	1.867
Scale	Target_max/Bin_max
	Max(Scale_95, Scale_max)	1.867
ExpGain_threshold	Threshold to assist in	50000
	determining “no
	light sources in scene” vs. “lights
	sources in scene”
ExpGain	The Exposure Gain	20000
	product value from
	coarse autoexposure
MaxTo95_threshold	If ratio of Bin_max to	2.0
	Bin_95 is larger
	than this threshold it is likely that
	the scene has light sources
Sat_threshold	If the count of saturated pixels is	10000
	larger than this number, it is likely
	that there are light sources in the
	scene.
Hist_Bins	Number of bins in the histogram	32

[0077] As shown above, the Bin—95 and Bin_max values from the coarse phase of the autoexposure method of the present invention are compared to pre-selected targets (Target—95, Target_max) and two parameters (Scale—95, Scale_max) are calculated as the ratios of the target values to the values in the current image. Examples of calculated Scale values (Scale, Scale—95, Scale_max) and other parameters for an exemplary exposure are shown in the third column of the above table to illustrate these fine computations. Based upon the computations made during this fine phase, an appropriate ratio for scaling (i.e., adjusting) the ExpGain setting established during the coarse phase is determined in the manner described below.

[0078] G. Detailed Method Steps of Fine Phase of Autoexposure Method

[0079] FIG. 6 is a flowchart illustrating the detailed method steps of the fine phase of the autoexposure method of the present invention. As described above, the fine phase adjusts the exposure settings generated during the coarse phase of the process based upon a determination about whether or not the scene has areas of relative brightness (e.g., overhead lights). Accordingly, at step 601, the fine phase commences with an evaluation as to whether or not areas of relative brightness are present in a scene. Image information and parameters computed (e.g., ExpGain, Bin—95, Bin_max, etc.) during the coarse phase are utilized during fine phase calculations. The presence of light sources (i.e., areas of brightness) is assumed if all the following conditions hold true:

(ExpGain * Scale_max)>ExpGain_threshold;   1.

(Bin_max/Bin—95)>MaxTo95_threshold; and   2.

count_saturated_pixels>Sat_threshold.   3.

[0080] As areas of relative brightness are usually present in darker scenes (which have higher ExpGain values), the first condition looks for darker scenes. For example, if the ExpGain setting is 20000 (the example value as shown in the above table), and the Scale_max value is 1.867, the product of these values (ExpGain * Scale_max) is lower than the above exemplary ExpGain_threshold value of 50000. Accordingly, in this example the first condition is not satisfied.

[0081] The second and third conditions look for the presence of areas of relative brightness in the scene. Using the same example values from the third column of the above table, Bin_max (15) divided by Bin—95 (12) is equal to 1.25. This value of 1.25 is less than the MaxTo95_threshold of 2.0 indicated above. The third condition is evaluated by comparing the count of saturated pixels (for example, a count of 5000) made during the coarse phase (count_saturated_pixels) to the saturation threshold (Sat_threshold with an example value of 10000 above). In this example, count_saturated_pixels is less than the saturation threshold. As shown at step 602, if all three of the above conditions hold true, the method proceeds to step 605 for calculation of the final exposure settings. However, if any of the conditions in step 601 are not satisfied (i.e., they do not hold true), then the method proceeds to step 603. Since none of the three conditions are satisfied in this example, the method would proceed to step 603 for adjustment of coarse exposure settings.

[0082] In step 603, the Scale value is evaluated so that the method does not scale Bin_max (i.e., the 99.5 percentile bin) beyond the largest histogram bin (e.g., bin 31 in a histogram from bin 0 to bin 31). The Scale value determined as set forth above is compared to the number of bins in the histogram divided by the Bin_max value. If Scale is greater than the number of bins divided by Bin_max (i.e., if (Scale>((Hist_Bins—1)/Bin_max)), the method proceeds to step 604 for adjustment of the Scale value. Otherwise, Scale is not adjusted and the method proceeds instead to step 605. These calculations may be illustrated by the following example, which is based upon the example values set forth in the above table. The example Scale value of 1.867 is compared to the number of bins in the histogram (Hist_Bins value of 32 minus 1) divided by the Bin_max value (15 in this example). In this case, the Scale value of 1.867 is less than 2.067 (31 divided by 15). Accordingly, the Scale value of 1.867 does not need to be adjusted as it will not scale Bin_max beyond the largest histogram value and the method may proceed to step 605.

[0083] On the other hand, if the evaluation in step 603 determines that the Scale value would shift the histogram too far towards overexposure (i.e., Scale is greater than (Hist_Bins -1)/Bin_max), the Scale value would be limited in step 604 in order to preserve highlights in an image. The following adjustment would be made to the Scale value in this situation: Scale=(Hist_Bins-1)/Bin_max. For instance, if the Bin_max value were 20, the Scale value would be equal to 1.55 (31 divided by 20). This adjustment serves to reduce the final ExpGain value that will be used for final exposure of the image.

[0084] After the appropriate Scale value has been determined through the above steps, the final exposure settings may be computed. At step 605, the final ExpGain value is determined by multiplying ExpGain by Scale. As shown, the final ExpGain setting is equal to the coarse ExpGain value multiplied by the Scale parameter (i.e., ExpGain=ExpGain * Scale). As described in the above table, Scale is equal to the greater of Scale—95 or Scale_max. In this example, the Scale_max setting of 1.867 (as shown in column 3 of the above table opposite Scale) is used to determine the final ExpGain value. At step 606, the adjusted ExpGain value determined as a result of this fine phase may then be used to generate final exposure time and sensor gain settings for capture of an image on the digital camera's image sensor. The mapping of the final ExpGain value to particular exposure time and sensor gain settings for taking a final exposure is described in greater detail below.

[0085] H. Mapping ExpGain to Exposure Time and Sensor Gain

[0086] During the coarse phase of the autoexposure method of the present invention, speed of processing is preferred over image quality of the captured image as the captured image data is discarded at the end of the coarse phase in any event. In order to speed up processing during the coarse phase, high sensor gains are used. FIG. 7 is a graph illustrating the manner in which ExpGain is mapped to particular exposure time (Exp) and sensor gain (Gain) settings during the coarse phase. As shown, a piecewise constant curve is mapped that is defined by arrays E [N], that represents exposure time (Exp) and G [N], that represents sensor gain (Gain). The curve is defined so that it is monotonically increasing. For a given ExpGain value, NO is determined so that E [NO] *G [NO] just exceeds ExpGain as follows:

Gain=G[NO];

Exp=ExpGain/Exp;

[0087] As previously described, the ExpGain value used during the coarse phase may be an initial setting (either pre-selected or based upon the most recent picture taken) or may result from feedback received during a prior iteration.

[0088] FIG. 8 illustrates the mapping of the final ExpGain value to the exposure and gain settings to be used for final exposure of an image. As shown at FIG. 8, a piecewise constant curve that is defined by arrays Ef [N], that represents exposure time (Exp) and Gf [N] that represents sensor gain (Gain) is used for final exposure. The curve is defined so that it is monotonically increasing. For a given ExpGain, NO is determined so that Ef [NO] *Gf [NO] just exceeds ExpGain as follows:

If (ExpGain > Ef [NO − 1] * Gf [NO] )
{
Gain = Gf [NO] ;
Exp = ExpGain/Gain;
}
else
{
Exp = Ef [NO − 1] ;
Gain = ExpGain/Exp;
}

[0089] This formula used during final exposure represents a finer computation than for the coarse phase, but the extra processing step is justified because this calculation determines the final exposure time and gain settings before the picture destined for compression and storage is captured.

[0090] I. Alternative Embodiment

[0091] As previously noted, in the preferred embodiment the digital camera has only an optical viewfinder and does not utilize a Direct View Display. However, the present invention is also useful in a camera which includes a Direct View Display enabling a user to view the scene being captured. A Direct View Display enables a digital camera device to continuously capture preview images (frames), thereby providing an opportunity to revise exposure control settings continuously (per frame). In this context, the autoexposure method of the present invention can be used to jumpstart or bootstrap the display by determining correct exposure settings for the first preview frame. Similarly, the present autoexposure method may also be used to start the automatic exposure control method on a digital camera having a Direct View Display.

[0092] While the invention is described in some detail with specific reference to a single-preferred embodiment and certain alternatives, there is no intent to limit the invention to that particular embodiment or those specific alternatives. For instance, those skilled in the art will appreciate that modifications may be made to the preferred embodiment without departing from the teachings of the present invention.

Claims

1. A method for automatic exposure control at a digital camera device, the method comprising: in response to user input requesting capture of a digital image at a digital camera device, capturing image data on an image sensor, said image data including values of brightness of pixels of said image sensor; profiling distribution of said values of brightness; generating at least one initial exposure setting for capture of said digital image, said at least one initial exposure setting based upon scaling distribution of said values of brightness towards the midpoint of a pre-selected interval; evaluating said image data to determine whether areas of relative brightness are present; and generating at least one final exposure setting by scaling said at least one initial exposure setting based upon whether areas of relative brightness are determined to be present.

2. The method of claim 1, further comprising: capturing a digital image using said at least one final exposure setting.

3. The method of claim 1, wherein said step of profiling distribution of said values of brightness includes generating a histogram of said values of brightness

4. The method of claim 3, wherein said histogram is generated based upon counts of said values of brightness in pre-selected ranges.

5. The method of claim 3, wherein said step of scaling said initial exposure setting includes not scaling said histogram beyond the largest histogram value in the event areas of relative brightness are determined not be present.

6. The method of claim 1, wherein said digital camera device has a fixed lens aperture.

7. The method of claim 1, wherein said digital camera device does not include a flash device.

8. The method of claim 1, wherein said digital camera does not include an external light sensor.

9. The method of claim 1, wherein said step of capturing image data includes capturing image data using pre-selected exposure settings.

10. The method of claim 9, wherein said pre-selected exposure settings comprise the exposure settings used for capture of a prior image on said digital camera device.

11. The method of claim 1, wherein said at least one final exposure setting includes at least one of exposure time and sensor gain.

12. The method of claim 1, further comprising: counting saturated pixels; and reducing at least one of exposure time and sensor gain in the event said count of saturated pixels exceeds a pre-selected maximum.

13. The method of claim 1, wherein step of evaluating said image data to determine whether areas of relative brightness are present includes determining whether light sources are present.

14. The method of claim 1, wherein step of generating said at least one final exposure setting includes using piecewise constant curves.

15. The method of claim 1, wherein step of capturing image data on an image sensor includes using relatively high sensor gain values.

16. The method of claim 1, said step of generating at least one final exposure setting includes using relatively low sensor gain values.

17. The method of claim 1, wherein said step of determining whether areas of relative brightness are present includes determining whether a count of pixels with high brightness values exceeds a pre-selected maximum.

18. The method of claim 1, wherein said step of determining whether areas of relative brightness are present includes determining whether said at least one initial exposure setting exceeds an established threshold.

19. The method of claim 1, further comprising: using said at least one final exposure setting to generate an initial image on a direct view display of a digital camera.

20. A method for automatic exposure control for a digital camera device, the method comprising: receiving user input requesting capture of a digital image at a digital camera device; in response to said user input, capturing image data including luminosity values of pixels of an image sensor of said digital camera device; generating a profile of said luminosity values; determining an initial exposure solution for exposure of a digital image based upon scaling said profile of said luminosity values towards the mid-point of a pre-selected interval; counting pixels with high luminosity values to determine whether areas of relatively high luminosity values are present; and generating a final exposure solution by scaling said initial exposure solution based upon whether areas of relatively high luminosity values are determined to be present.

21. The method of claim 20, further comprising: capturing a digital image using said final exposure solution.

22. The method of claim 20, wherein said step of generating a profile of said luminosity values includes generating a histogram of said luminosity values.

23. The method of claim 20, wherein said step of generating a final exposure solution includes using piecewise constant curves.

24. The method of claim 20, wherein said step of capturing image data on pixels of an image sensor includes using relatively high sensor gain values.

25. The method of claim 20, wherein said step of scaling said initial exposure solution includes reducing at least one of exposure time and sensor gain in the event areas of relatively high luminosity values are determined not to be present.

26. The method of claim 20, wherein said digital camera device has a fixed lens aperture.

27. The method of claim 20, wherein said digital camera device does not include a flash device.

28. The method of claim 20, wherein said digital camera does not include an external light sensor.

29. The method of claim 20, wherein said step of capturing image data includes capturing image data using pre-selected exposure settings.

30. The method of claim 20, wherein said step of generating a profile of said luminosity values includes mapping said luminosity values into a histogram.

31. The method of claim 30, wherein said step of mapping luminosity values into a histogram includes mapping at least 256 luminosity values into at least 8 pre-selected ranges.

32. The method of claim 20, further comprising: counting saturated pixels in said image data; and if said count of saturated pixels exceeds a pre-selected maximum, adjusting said initial exposure solution to avoid over saturation.

33. The method of claim 20, further comprising: using said final exposure solution to generate an initial image on a direct view display of a digital camera.

34. The method of claim 20, further comprising: using said final exposure solution to start a direct view display automatic exposure control process on a digital camera having a direct view display.

35. A method for automatic exposure control for a digital camera, the method comprising: generating an initial exposure solution for capture of an image on a digital camera by iteratively performing the substeps of: capturing values of brightness of pixels on an image sensor of a digital camera using an initial exposure solution; determining whether distribution of said values of brightness is within a pre-selected range; if distribution of said values of brightness is determined not to be within said pre-selected range, adjusting said initial exposure solution by scaling distribution of said values of brightness towards the mid-point of said pre-selected range; if distribution of said values of brightness is determined to be within said pre-selected range, generating a final exposure solution by performing the substeps of: evaluating said values of brightness to determine whether areas of relatively high brightness values are present; and scaling said initial exposure solution based upon whether areas of relatively high brightness values are determined to be present.

36. The method of claim 35, wherein said substep of capturing values of brightness using an initial exposure solution includes using relatively high sensor gains to decrease exposure time.

37. The method of claim 35, wherein said step of generating an initial exposure solution continues until distribution of said values of brightness is determined to be within said pre-selected range.

38. The method of claim 35, wherein said substep of determining whether distribution of said values of brightness is within a pre-selected range includes evaluating whether an image is overexposed.

39. The method of claim 35, wherein said substep of determining whether distribution of said values of brightness is within a pre-selected range includes evaluating whether an image is underexposed..

40. The method of claim 35, wherein said initial exposure solution comprises a product of exposure time and sensor gain.

41. The method of claim 35, wherein said step of generating an initial exposure solution further comprises the substeps of: counting pixels with high brightness values; and if said count of pixels with high brightness values exceeds a pre-selected maximum, scaling said initial exposure solution to avoid over saturation.

42. The method of claim 35, wherein said final exposure solution comprises specific exposure time and sensor gain values.

43. The method of claim 35, wherein said final exposure solution is generated using piecewise constant curves.

44. The method of claim 35, wherein said step of generating a final exposure solution includes using lower sensor gain values to improve image quality.

45. The method of claim 35, wherein said digital camera device has a fixed lens aperture.

46. The method of claim 35, wherein said digital camera device does not include a flash device.

47. The method of claim 35, wherein said digital camera does not include an external light sensor.

48. The method of claim 35, wherein said substep of evaluating said values of brightness to determine whether areas of relatively high brightness values are present includes determining whether a count of pixels with high brightness values exceeds a pre-selected maximum.

49. The method of claim 35, wherein said substep of evaluating said values of brightness to determine whether areas of relatively high brightness values are present includes determining whether a count of pixels with high brightness values exceeds a pre-selected maximum.

50. The method of claim 35, further comprising: mapping said final exposure solution to specific exposure time and sensor gain settings.

51. The method of claim 50, wherein said step of mapping said fine exposure settings to exposure time and sensor gain values includes mapping using piecewise constant curves.

52. The method of claim 50, further comprising: capturing an image on said digital camera using said exposure time and sensor gain settings.

53. The method of claim 35, further comprising: using said final exposure solution to generate an initial image on a direct view display of a digital camera.

54. The method of claim 35, further comprising: using said final exposure solution for starting an automatic exposure method on a digital camera having a direct view display.