Patent Application
20100045821
This invention pertains to a digital camera, and in particular, to a device and method for triggering a digital camera to save images.
Digital cameras have become increasingly popular in the past few years and are replacing traditional film-based cameras for many users.
However, until now, digital cameras typically have exhibited some disadvantages compared to traditional film-based cameras. One shortcoming of digital cameras has been the difficulty they present for a user who is trying to capture a transient image. In particular, there is normally a delay between when a user depresses a shutter control and when an image is captured by the camera. This can make it very difficulty for a user to depress the shutter at the precise instant that is necessary to have the camera capture a desired ephemeral image.
Accordingly, it would be desirable to provide a digital camera which facilitates the capture of desired transient images.
In one aspect of the invention, a digital camera comprises: an image sensor; a buffer memory adapted to receive images from the digital image sensor and to store the images therein; an image processor adapted to receive selected images and to process the selected images, including applying a video compression algorithm to the selected images; an image storage device adapted to store the processed, selected images; and an image selector adapted to select the selected images among the stored images in the buffer memory based on at least one image selection criterion, and to cause unselected stored images to be discarded from the buffer memory.
In another aspect of the invention, a method of storing images comprises: sensing images; processing selected images, including applying a video compression algorithm to the selected images; storing the processed, selected images in a storage device; selecting the selected images among the sensed images based on at least one image selection criterion; and causing unselected images to be discarded.
In yet another aspect of the invention, a digital camera comprises: an image sensor adapted to sense images; an image processor adapted to receive selected images and to process the selected images, including applying a video compression algorithm to the selected images; an image storage device adapted to store the processed, selected images; and an image selector adapted to select the selected images among the sensed images based on at least one image selection criterion, and to cause unselected images to be discarded.
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided as teaching examples of the invention.
Digital camera 100 includes shutter 110, an image sensor 120, a buffer memory 130, an image processor 140, and an image storage device (e.g., memory) 150.
Shutter 110 controls a time period during which digital camera 100 senses an image. Normally, the shutter is controlled by a user depressing a shutter button on the housing of digital camera 100.
Image sensor 120 senses an image from light provided to the camera when shutter 100 is opened. Beneficially, image sensor 120 includes a plurality of pixels which respond to light to produce image data representing the sensed image. Image sensor 110 may be a charge-coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor, or other convenient sensor.
When image sensor 120 captures an image, it outputs the corresponding image data to buffer memory 130. In a typical arrangement, buffer memory 130 comprises a relatively high speed semiconductor memory device. In one embodiment, buffer memory 130 comprises synchronous dynamic random access memory (SDRAM). Typically, buffer memory 130 has sufficient capacity to store in a range of 2-10 uncompressed images.
Image data stored buffer memory 130 is subsequently processed by image processor 140 in the background while additional images may be captured by image sensor 120 and stored in buffer memory 130. In general, image processor 140 compresses the images stored in buffer memory 130, for example, by applying joint picture experts group (JPEG) compression. In the compression process, image processor 140 may compress the size of the image data for each image by a factor of 10, 20 or more, before the compressed image is stored on image storage device 150.
Beneficially, image storage device 150 comprises nonvolatile memory. In one embodiment, image storage device 150 comprises flash memory. Normally, image storage device 150 is a removable memory device that typically has a capacity or anywhere from tens of megabytes up to several gigabyte. Image storage device 150 is generally compliant with one of various standards for removable memory devices such as smart disk, secure digital, memory stick, XD, etc.
The most time consuming process in digital camera 100 is the compression process performed by image processor 140. Ultimately, it is this compression process which limits the speed at which digital camera can continue to capture new images.
As explained above, one challenge faced by digital cameras is the difficulty it presents for a user who is trying to capture a transient image.
To address this challenge, many digital cameras have a “fast shooting mode” that allows a user to take a series of pictures in quick succession. In the fast shooting mode, images from image sensor 120 are stored in buffer memory 130 very rapidly, while the image compression process of image processor 140 lags far behind. Images can continue to be captured at the rapid rate of image sensor 120 and buffer memory 130 only until all of the available space in buffer memory 130 is exhausted. At that point, the speed of the fast shooting mode drops to the much slower image compression rate which governs how quickly capacity in buffer memory 130 is freed up to store additional images. Buffer memory 130 normally has only sufficient space for 2-10 uncompressed images. So it can be seen that very quickly, the fast shooting mode is reduced to the relatively slow image compression speed of image processor 140. Typically, this happens after only a few seconds in the fast shooting mode.
Accordingly, there is a limitation to how effective a fast shooting mode can be in assisting a human photographer in capturing a fleeting event at the precise instant when it occurs.
Digital camera 200 includes shutter 210, an image sensor 220, a buffer memory 230, an image selector 235, an image processor 240, an image storage device (e.g., memory) 250, a light sensor 260, a microphone 270, and a user input 280.
Typically, all of the components of digital camera 200 shown in
Many of the functional blocks in digital camera 200 of
Beneficially, digital camera 200 includes image selector 235. Image selector allows digital camera 200 to discard images which are determined not to be likely to be of particular interest without compressing them or storing them in image storage device 250. For example, when digital camera 200 is operated in a fast shooting mode, image selector 240 can discard redundant images before compression, and thereby allow digital camera 200 to capture only those images likely to be desired by a user, and to do so for an extended period of time. This enhances the likelihood that digital camera 200 will successfully capture ephemeral images that are more likely to be of interest to the user.
In particular, image selector 235 employs one or more image selection criteria for determining which images in buffer memory 230 are to be processed by image processor 240, and which images are to be discarded without processing (e.g., compression) by image processor 240.
The decision which frames to store and which to discard can be made based on external sensor data, or properties of the images themselves, or combinations of both. Beneficially, the selection criteria are chosen to correlate events with the open shutter time during which an image was taken.
In one embodiment, light sensor 260, typically used for iris or flash control, is sampled to determine if there was a change in overall lighting level during the shutter period. The sampling period for light sensor 260 may optionally include time before and/or after the shutter period. This approach effectuates a crude motion sensor which tags the picture in question with a value related to the lighting level. Image selector 235 may compare the change in light intensity against a fixed threshold, or against a change in light intensity for other images, as a selection criterion for determining which images are processed by image processor 240 and stored in image storage 250, and which images are discarded.
Alternatively, an overall intensity of the light sensed by light sensor 250 may be used as a selection criterion to discard images that are too dark or too bright to be visible and therefore, are unlikely to be of interest. In that case, image selector 235 may compare the light intensity to one or more thresholds, and select images for processing and storage, or to be discarded, depending upon the relationship between the light intensity when the image was captured, and the threshold(s).
In another embodiment, a sound level detected by microphone 270 may be processed in a similar manner to the overall light level. For example, image selector 235 uses sudden changes in sound volume during the exposure time as a selection criterion to indicate moments of interest and corresponding images to be processed and stored.
In a further embodiment, a user may stop a continuous shooting mode (without storage) by operating a user control (e.g., a shutter button) to capture a number of images before and after the user depresses the shutter button. In other words, in such an embodiment, user input 280 is a user-operable shutter control that allows a user to indicate in a continuous shooting mode a time instant at which the user desires to capture an image. In that case, the image selection criterion can be the difference between the time instant indicated by the user and the time instant when a particular image is sensed by the digital image sensor. Image detector 235 can compare this time difference to a predetermined threshold to determine whether or not the image was sensed close enough in time to the user-indicated instant. Thus, the “right” instant for capturing an image is indicated by the user, with a margin for error.
In still another embodiment, images stored in buffer memory 230 are compared against each other, for instance, to determine motion between the frames. To save time, this process could be limited to partial pictures (e.g. a center point of an image) or a grid of sampling points. In particular, one or more characteristics of a first image are compared against one or more characteristics of a second image. In that case, when the amount of change between the characteristic(s) of the images is determined to be greater than some threshold, then one or both of the images may be scheduled for compression and storage. This embodiment effectuates a more sophisticated motion sensing, where the camera can trap sudden unexpected motion—common, for instance in wildlife photography.
In yet another embodiment, the buffered images in buffer memory 230 are analyzed individually. Such analysis may include space and/or frequency domain analysis to detect one or more characteristics, such as brightness, hue, sharpness, etc. Image selector 235 can compare the detected characteristic(s) of an image against a threshold as a selection criterion for determining whether the image is to be processed by image processor 240 and stored in image storage 250, or to be discarded.
Due to its nature, image processor 240 is very well suited to frequency domain analysis, as this is a significant part of the JPEG compression algorithm. In that case, beneficially, image selector 235 may be combined with image processor 240. Again, in some cases the analysis may be applied only to parts of an image.
Image selector 300 includes an environmental sensor(s) 310, an image tagger 320, a threshold 330, and a comparator 340. Environmental sensor(s) 310 may comprise light sensor 260, microphone 270, etc., and/or user input 280. Environmental data provided by environmental sensor(s) may include a light intensity, a change in light intensity, a sound volume, a change in sound volume, a time at which an image in captured with respect to a time selected by a user, etc.
Image tagger 320 tags each image stored in buffer memory 230 with one or more tag values corresponding to the environmental data for that image.
Comparator 340 compares the tag value(s) 325 against one or more threshold(s) 330 and outputs one or more signals 345 which indicate whether to process and store the corresponding image, or to discard it.
Image selector 400 includes an image analyzer 420, a threshold 430, and a comparator 440. Image analyzer may perform space and/or frequency domain analysis of an image to detect one or more characteristics of the image, such as brightness, hue, sharpness, etc., and/or a change in any of these characteristics between a first image and a second image. Image analyzer 420 outputs one or more characteristics of an image and/or differences between one or more characteristics of a first image and a second image.
Comparator 440 compares the image characteristic(s), and/or differences in characteristics between a first image and a second image, against one or more threshold(s) 430 and determines whether to process and store the corresponding image, or to discard it.
In some embodiments, image selector 240 can be conceptualized as a triggering device that triggers images for processing and storage by image camera 200.
Many variations of digital camera 200 are possible within the scope of the principles discussed above. For example, in some embodiments, one or all of light sensor 260, microphone 270, and/or user input 280 may be omitted, and/or may not be connected to image selector 235.
When external events are used for the selection criteria, in some embodiments it is not necessary to actually store the image in buffer memory before making the decision whether to process it or drop it.
The functional blocks in
While preferred embodiments are disclosed herein, many variations are possible which remain within the concept and scope of the invention. Such variations would become clear to one of ordinary skill in the art after inspection of the specification, drawings and claims herein. The invention therefore is not to be restricted except within the spirit and scope of the appended claims.
