The present invention relates generally to digital cameras, and more particularly to a method and apparatus for providing an instant review mode in digital cameras.
Most digital cameras today are similar in size to and behave like conventional point-and-shoot cameras. However, most digital cameras store digital images in an internal flash memory or on external memory cards and are equipped with a liquid-crystal display (LCD) screen on the back of the camera. Through the use of the LCD, most digital cameras operate in two modes, live and play. In live mode, the LCD is used as a viewfinder in which the user may view an object or scene before taking a picture. In play mode, the LCD is used as a playback screen for allowing the user to review previously captured images either individually or in arrays of four, nine, or sixteen images.
When a user wishes to capture an image in live mode, the user sends a capture command, usually by pressing a capture or shutter button. When the image is captured, the image is frozen on the LCD for a relatively long time while the image is processed and stored in memory. Typically, the image is partially obscured by a “wait” message during processing. Once the camera has finished processing and storing the image, the camera reverts back to the live mode for capturing further images. If the user desires to view the image just captured, the user must change to play mode.
Although a user can capture images in live mode, a user cannot obtain an unobstructed view an image that has just been captured without changing from live mode to play mode. The user also lacks control over the image frozen on the LCD during processing. Similarly, there is no control over the mode which the camera returns to after processing. Instead, the camera automatically returns to a live view, forcing the user to switch to play mode to view or perform operations on the last image captured.
Accordingly, what is needed is an improved method and system for accessing the last image captured by a digital camera. The present invention addresses such a need.
The present invention provides a method and system for providing instant review of a last image captured in an image capture device. The image capture device includes a viewfinder for displaying a live image and each image of a plurality of previously captured images. The method and system comprise allowing a user to select instant review of the last image captured by the digital camera. The method and system further comprise determining the status and location of the last image and providing the last image to the viewfinder for display. Therefore, the image capture device is capable of displaying the last image substantially immediately after the last image has been captured. In another aspect, the image capture device contains an image processing system. In this aspect, the method and system comprise allowing a user to access a play mode and a review mode while the last image captured undergoes processing by the image processing system.
According to the system and method disclosed herein, the present invention allows a user to view and perform operations on an image very shortly after the image is captured, thereby increasing overall system performance.
The present invention relates to an improvement in digital cameras. The following description is presented to enable one of ordinary skill in the art to make and use the invention and is provided in the context of a patent application and its requirements. Various modifications to the preferred embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. Thus, the present invention is not intended to be limited to the embodiment shown, but is to be accorded the widest scope consistent with the principles and features described herein.
Many conventional digital cameras have two modes: live mode and play mode. In live mode, images are actually captured and the liquid-crystal display (LCD) screen is used as a viewfinder. In play mode, the LCD screen on the back of the camera can display previously captured images.
In order to switch between modes and control the digital camera in a given mode, a user interface is typically employed. The user interface typically includes a number of buttons or switches for setting the camera into one of the two modes and for navigating between images in play mode. For example, a digital camera may include two navigation buttons labeled “−” and “+”, a mode button, a display button, a zoom button, a delete button, and a capture or shutter button.
In play mode, the digital camera may begin with a default screen displaying a full-sized individual image that was previously captured. Other images stored in the camera may be displayed in a backward or forward sequence by pressing the “−” and “+” navigation buttons, respectively. During play mode, the user can also cause four images to be displayed in a 2×2 array, nine images to be displayed in a 3×3 array, or sixteen images to be displayed in a 4×4 array. The user can then “page” through screens of image arrays or move from image to image in the arrays. The user may also have the full-sized image displayed of a chosen image and can delete the image.
One disadvantage of conventional digital cameras is that when a user wishes to capture an image, the user must first switch from play mode to live mode. In live mode, the LCD acts as a viewfinder. The user captures the image by pressing the capture or shutter button, which typically controls the shutter. The image is then frozen on the LCD while the image is processed and stored in memory. Processing generally includes compression of the image prior to storage in memory. Therefore, the image is typically frozen on the LCD screen for a relatively long time, often up to a few seconds. The image frozen on the LCD screen is also partially obscured by a “wait” message during processing. In addition, the image displayed on the screen during processing may actually be the image just prior to the image being processed.
Once the camera has finished processing the image, the camera automatically removes the frozen image and reverts back to the live mode for capturing further images. If the user desires to view the image just captured, the user must change from live mode to play mode. Only then can the user view the last images, as well as all previously captured images. Once in play mode, the user can also perform operations on the image, such as deleting the image.
Although a digital camera allows a user to review previously captured images and capture new images, one of ordinary skill in the art will readily recognize that a user cannot obtain an unobstructed view of an image that has just been captured in live mode. Although an image is frozen on the screen after capture, the image is partially obscured by the “wait” message. If a user desires an unobstructed view or to perform operations on the last image captured, the user must switch to play mode. There is no control over the image frozen on the LCD during processing. One of ordinary skill in the art will also realize that the user is unable to control the mode to which the camera returns to after processing. Instead, the camera automatically returns to a live view.
The present invention provides a method and system for providing an instant review mode which allows instant review of the last image captured in a digital camera. Thus, although the present invention will be described in terms of instant review of an image, the method and system provide rapid review of the image. Although the present invention will be described in terms of a digital camera having at least two modes, live view and instant review, one of ordinary skill in the art will realize that the present invention is not limited to a digital camera having two modes. In a one embodiment, the digital camera employing the method and system supports additional modes such as review mode and play mode, described below. Finally, although the present invention is described in terms of a digital camera, one of ordinary skill in the art will readily realize that the method and system are fully applicable to any image capture device.
Referring now to
Referring now to
In operation, imaging device 114 captures an image of object 112 via reflected light impacting image sensor 224 along optical path 236. Image sensor 224, which is typically a charged coupled device (CCD), responsively generates a set of raw image data in CCD format representing the captured image 112. The raw image data is then routed through ASP 228, A/D converter 230 and interface 232. Interface 232 has outputs for controlling ASP 228, motors 234 and timing generator 226. From interface 232, the raw image data passes over system bus 116 to computer 118.
Referring now to
Power manager 342 communicates via line 366 with power supply 356 and coordinates power management operations for camera 110. CPU 344 typically includes a conventional processor device for controlling the operation of camera 110. In the preferred embodiment, CPU 344 is capable of concurrently running multiple software routines to control the various processes of camera 110 within a multithreaded environment. DRAM 346 is a contiguous block of dynamic memory which may be selectively allocated to various storage functions. LCD controller 390 accesses DRAM 346 and transfers processed image data to LCD screen 402 for display.
I/O 348 is an interface device allowing communications to and from computer 118. For example, I/O 348 permits an external host computer (not shown) to connect to and communicate with computer 118. I/O 348 also interfaces with a plurality of buttons and/or dials 404, and an optional status LCD 406, which in addition to the LCD screen 402, are the hardware elements of the camera=s user interface 408.
Non-volatile memory 350, which may typically comprise a conventional read-only memory or flash memory, stores a set of computer-readable program instructions to control the operation of camera 110. Removable memory 354 serves as an additional image data storage area and is preferably a non-volatile device, readily removable and replaceable by a camera 110 user via buffers/connector 352. Thus, a user who possesses several removable memories 354 may replace a full removable memory 354 with an empty removable memory 354 to effectively expand the picture-taking capacity of camera 110. In the preferred embodiment of the present invention, removable memory 354 is typically implemented using a flash disk.
Power supply 356 supplies operating power to the various components of camera 110. In the preferred embodiment, power supply 356 provides operating power to a main power bus 362 and also to a secondary power bus 364. The main power bus 362 provides power to imaging device 114, I/O 348, non-volatile memory 350 and removable memory 354. The secondary power bus 364 provides power to power manager 342, CPU 344 and DRAM 346.
Power supply 356 is connected to main batteries 358 and also to backup batteries 360. In the preferred embodiment, a camera 110 user may also connect power supply 356 to an external power source. During normal operation of power supply 356, the main batteries 358 provide operating power to power supply 356 which then provides the operating power to camera 110 via both main power bus 362 and secondary power bus 364. During a power failure mode in which the main batteries 358 have failed (when their output voltage has fallen below a minimum operational voltage level) the backup batteries 360 provide operating power to power supply 356 which then provides the operating power only to the secondary power bus 364 of camera 110.
Referring now to
RAM disk 532 is a memory area used for storing raw and compressed image data and typically is organized in a Asectored@ format similar to that of conventional hard disk drives. In the preferred embodiment, RAM disk 532 uses a well-known and standardized file system to permit external host computer systems, via I/O 348, to readily recognize and access the data stored on RAM disk 532. System area 534 typically stores data regarding system errors (for example, why a system shutdown occurred) for use by CPU 344 upon a restart of computer 118.
Working memory 530 includes various stacks, data structures and variables used by CPU 344 while executing the software routines used within computer 118. Working memory 530 also includes several input buffers 538 for temporarily storing sets of raw image data received from imaging device 114, and a frame buffer 536 for storing data for display on the LCD screen 402. In a preferred embodiment, each input buffer 538 and the frame buffer 536 are split into two separate buffers, called ping-pong buffers (shown by the dashed lines), to improve the display speed of the digital camera and to prevent the tearing of the image in the display 402. Referring now to
Referring now to
The header 602 includes information identifying the particular image file and the image represented by the image data 604. The image data 604 is the actual data comprising the full-sized captured image in compressed form, preferably in JPEG format. Although the user can typically choose the resolution mode in which images are captured, once an image is processed and compressed, the compressed image data 604 is the high-resolution representation of the image compared to the thumbnail 606 and screennail 608. If the image is captured at a resolution of 640×480 pixels, for example, then the compressed image data 604 is typically fifty-to-sixty kilobytes in size.
The thumbnail image 606 is a small, uncompressed low-resolution version of the image. In a preferred embodiment, the pixel size of thumbnail image 606 is less than the display size of the LCD screen 402 (e.g., 80×60 pixels), and has a storage size of approximately ten kilobytes.
The screennail image 608 is a medium-resolution version of the image and in a preferred embodiment is also compressed, although compressing the screennail 608 is optional. Unlike the thumbnail image 606, the screennail image 608 is display-sized and fills the visible area of the LCD screen 402 when displayed. In one embodiment, the screennail image 608 is optional. The screennail image 608 may be optional because the image sensor 224 is small enough, for example 640×480 pixels in size, that the image data 602 can be very rapidly provided to the LCD screen 402. Similarly, the screennail image 608 may be optional where the hardware for the camera 110 is capable of providing the image data 604 rapidly enough. In one embodiment in which the screennail image 608 is optional, the data for the screennail image 608 remains in memory available for display until the image file 600 is completed. In a preferred embodiment, the pixel size of a compressed screennail image 608 is preferably 288×216 and requires approximately fifteen kilobytes to store.
The image tag field 610 includes information, preferably in the form of tags, regarding the image represented by the image data 604. Media type tags, for instance, indicate all the media types associated with the image, such as whether the image is a single image or a panorama image, for example. In certain operating modes, the media type tags are used to select the type of icon that is displayed in the LCD 402 along side the thumbnail image 606. Besides media tags, the image tag field 610 may also include other types of tags for storing additional information regarding the image and/or the camera 110 itself. For example, a tag could be used to indicate the settings of the camera 110 at the time the image was captured, or indicate the identity of the camera manufacturer, for instance. The information in these tags may be accessed through the buttons on the camera interface 400. The additional information may then be displayed either as text in the LCD 402.
The enhanced image file 600 of the present invention is created for each image as the user takes pictures while the camera is in capture mode. The enhanced image file 600 is then used to accelerate the user interface of the digital camera in the review and play mode as follows. When the camera is placed into review mode, the thumbnail images 606 contained in the image files 600 are directly displayed on the LCD 402 as representations of captured images, which eliminates the need to process and decompress the compressed image data 604. And when the camera is placed into play mode, the screennail image 608 contained in the image file 600 is first decompressed and displayed on the LCD 402 and then optionally updated with the higher-resolution compressed image data 604 as the image data 604 is being decompressed. This feature enables the digital camera to quickly display a full-sized version of the captured image in the LCD 402 without the delay incurred by first decompressing the higher-resolution JPEG image and resizing it to fit on the LCD 402. Whether or not to decompress and display the compressed image data 604 depends on the resolution of the display and the resolution of the screennail images 608.
Referring now to
Once the user presses the shutter button 418 to capture an image, the raw image data is captured by the image device 114 at a resolution set by the user and the raw image data is stored into an appropriate number of input buffers 538.
The raw image data is then used to generate an enhanced image file 600 for the captured image including the compressed image data 604, the thumbnail 606, and the screennail 608, as shown in
When generating the thumbnail and screennail images 606 and 608, the present invention takes advantage of the fact that the YCC data in the frame buffers 536 has already been processed by the live view generation process 612 and stored at the reduced resolution of the LCD screen 402. Since the thumbnail and screennail images 606 and 608 are also intended to be lower-resolution representations of the captured image, the previously processed YCC data in the frame buffers 536 is used to generate the thumbnail 606 and screennail 608 directly, rather than using the raw image data stored in the input buffers 538.
To generate the screennail image 608, the YCC data in the frame buffers 536 is converted from YCC 222 format into YCC 422 format and compressed by a conversion and compression process 614. To generate the thumbnail image 606, the YCC data in the frame buffers 536 is converted from the YCC 222 format into YCC 422 format and then resized by a conversion and resizing process 616. During the conversion and resizing process 616, the thumbnail image 606 may be resized by averaging in which a block of pixel values from the YCC 422 data are averaged to represent one pixel value of the thumbnail image 606, and/or by sub-sampling the YCC 422 data in which only a certain number pixels in a block are used to represent one pixel in the thumbnail image 606.
Referring now to
In another preferred embodiment, the thumbnail image 606 and the screennail 608 may be generated using a two-stage live view generator 612. In the first stage, the live view generator 612 provides images to the frame buffer 536 for display as described above. When the user captures an image, the raw image data from the imaging device is compressed due to higher quality before being stored in the input buffers 538, and the live view generator 612 switches to the second stage. In this stage, the live view generator 612 decompresses the compressed raw image data and processes the data into both YCC 222 data and YCC 422 data. The live view generator 612 may then transfer the YCC 422 data to the frame buffer 536 for display, and generate the thumbnail image 606 and the screennail 608 from the YCC 422 data.
The background spooling process 618 preferably includes RAM spoolers 1 and 2 (620), removable memory spoolers 1 and 2 (624), and an image processing and compression process (IPC) 622. Processes 620, 622 and 624 are preferably implemented as background processes on CPU 344 and may therefore run in parallel with other processes. As used herein, a spooler is a process that transfers data from one process or device to a second process or device. The primary purpose of the background spooling process 618 is to move data out of the input buffers 538 as fast as possible in order to free the input buffers 538 to capture another image. After the data is moved, the data is processed in the background. This allows the next image to be captured before the previous image is processed and compressed, which increases the capture rate of the digital camera.
In operation, after the user has captured an image, control of the raw image data in the input buffers 538 is transferred to RAM spooler 1 (620) if the RAM disk 532 is not full. If the RAM spooler 1 (620) obtains control of the raw image data, then the RAM spooler 1 (620) transfers the raw image data to the RAM disk 532.
Alternatively, if the RAM disk 532 is full, then control of the raw image data is transferred to the IPC 622 where the data is processed and compressed to generate the compressed image data 604 (
After the compressed image data 604 is generated, the IPC 622 may provide the compressed image data 604 to the RAM spooler 2 (620). The compressed image data 604 is then combined with the thumbnail 606 and the screennail 608 to generate the enhanced image data file (
Because image data is moved between different components during background spooling, the computer 118 has a mechanism for keeping track of and accessing the image data. For example, in review mode, the digital camera 110 may be required to display the thumbnail image 606 of a particular image on the LCD screen 402. Similarly, in play mode, the screennail image 608 of a particular image may be required for display on the LCD screen 402. Processing of the image may, however, be incomplete when the digital camera 110 is required to display the image on the LCD screen 402. As a result, the computer 118 must be capable of accessing the image at any point during processing.
If the thumbnail image 606 and screennail image 608 are completed, then the image data requested is accessed. If a thumbnail image 606 was requested, for example for use in review mode, the thumbnail image 606 is retrieved and resized via step 720. If the screennail 608 was requested, for example for use in play mode, then via step 722, the screennail 608 is retrieved, decompressed, and resized. Either the thumbnail image 606 or the screennail 608 are then displayed on the LCD screen 402. The last image may also be retrieved once processing of the image is complete. If the full image is required for display, the image is decompressed, resized, and displayed via step 724. As a result, the digital camera 110 can provide a high resolution update of the screennail 608 displayed on the LCD screen 402.
If processing of the image is not complete, then the thumbnail image 606 and screennail 608 are generated via 714. If the thumbnail image 606 was requested, then the thumbnail image 606 is resized and displayed via step 716. Similarly, if the screennail is requested, the screennail 608 is decompressed and displayed via step 718. Consequently, the user can view the image even though processing is incomplete.
To further illustrate the method and system in accordance with the present invention, refer now to
The top view 430 also depicts a slide/lock switch 421 for accessing an instant review mode in accordance with the method and system. The switch 421 is shown as having three positions: live 422, instant review 423, and lock 424. When in the live position 422, the digital camera 110 will allow capture of an image by depressing shutter button 404. Because of the spooling of data, described above, image capture appears extremely fast to a user. Thus, when the user depresses the shutter button 404, there is a brief response indicating the image has been captured, such as a flicker, a sound, or a very brief freezing of the image on the LCD screen 402. The camera 110 then quickly returns to live view. Typically, the user need not wait for the image to be processed before the digital camera 110 reverts to live mode. The image is, therefore, not frozen for a relatively long time on the LCD 402, and no “wait” message is required.
When the user pushes the switch 421 into the instant review position 423, the digital camera is placed into instant review mode and the user is given the opportunity to view the last image captured. The camera 110 remains in instant review mode until the user returns the switch 421 to the live position 422. In a preferred embodiment, the switch 421 is a spring loaded switch for which the default position is the live position 422. Thus, the camera 100 remains in instant review mode only as long as the user holds the switch 421 in the instant review position 423. When the user releases the switch 421, the digital camera 110 reverts to live mode. Once the camera 110 returns to live mode, the user can then capture another image. By holding and releasing the switch 421 between the instant review position 423 and the live position 422, the user can toggle between the live view and instant review of the last image captured. While in instant review mode, the user may perform certain operations on the last image captured in addition to viewing the last image captured. For example, the user will be able to add sound to the last image captured through the use of the sound button 414. When the user captures a second image, the second image will be placed on the LCD screen 402 until the user either exits instant review mode by releasing the switch 421 or presses the shutter button 404, as discussed more fully below.
To add sound to an image, a user presses the sound button 414 to begin recording. To finish recording, the user presses the sound button 414 again. In a preferred embodiment, the sound clip can be recorded either before or after the image to be annotated is captured. In addition, a user can add a recorded sound clip to any previously captured image, not just the last image captured.
If the image processing is complete, then the screennail image 608 for the last image is retrieved, decompressed, and displayed via step 816. If the last image was very recently captured and the processing is not complete, then the screennail image 608 is generated via step 818. The screennail image 608 is then decompressed and displayed via step 820. In one embodiment, the thumbnail image 606 and screennail image 608 are created prior to spooling of the image data. As a result, steps 818 and 820 are not required in such an embodiment.
The user can also lock the digital camera 110 into instant review mode by placing the switch 421 in the lock position 424. When the digital camera 110 is first locked in instant review mode, the LCD screen 402 maintains a live view if there is no previously captured image. Otherwise, the last image captured will be placed on the LCD screen 402. The last image captured remains on the LCD screen 402 until the user either exits instant review mode by releasing the switch 421 or by pressing a shutter button 404. A user captures an image by depressing the shutter button 404. In one embodiment, the shutter button 404 has two positions: a first position, where the button is slightly depressed, and a second position where the button 404 is further depressed. Depressing the shutter button 404 to the first position causes a live image to appear on the LCD screen 402, allowing the LCD screen 402 to be used as a viewfinder. Depressing the shutter button 404 to the second position causes an image to be captured.
When the user pushes the shutter button 404 to the second position, the camera 110 provides a response indicating the image has been captured. Because of the spooling of the image data, the image is not frozen on the LCD screen 402 during image processing. Instead, the camera 110 provides a brief response indicating the image has been captured. Instant review mode then causes the image to be placed on the LCD screen 402. In this case, instant review mode is provided using a method similar to the method 800 of
In order to return to a live view, the user releases the shutter button then presses the shutter button 404 into the first position. As a result, the last image is removed from the LCD screen 402 and replaced with a live view. In one embodiment, if the user then releases the shutter button, the last image is not placed on the screen again. However, in another embodiment, when the user releases the shutter button 404 from the first position, the last image is again displayed on the LCD screen 402. If available, the image data from the frame buffer 536 is used to display the image. If, however, the data was removed from the frame buffer, then the screennail image 608 from the last image is used. Thus, steps 814 through 280 of the method 800 would employed.
In one embodiment, if the digital camera 110 retains this image on the LCD screen 402 for a long enough interval, the digital camera 110 will update the screennail image 608 using the full resolution image. In such a case, the digital camera 110 would also retrieve, decompress and display the corresponding compressed image data 602. While in instant review mode, the user can perform operations on the image being reviewed, such as annotating the image with sound, as in the review and play modes.
In order to be able to capture another image, the user again depresses the shutter button 404 to the second position. When the user captures the second image, the second image will be placed on the LCD screen 402 until the user again releases and presses the shutter button 404 into the first position. In order to exit instant review mode, the user simply moves the switch 421 from the lock position 424 to the live position 422.
Although the instant review mode is described as being accessed through a switch 421, nothing prevents instant review mode from being accessed in another fashion. In another embodiment, instant review mode may be accessed through software, for example by choosing an appropriate icon displayed on the LCD 402. In such an embodiment, however, switching between live mode and instant review mode may be made more difficult by the numbers of commands through which a user may need to scroll to access the instant review mode icon. In addition, in an embodiment in which the user accesses instant review mode through software, once a user leaves instant review mode through the software command, the last image may not be recalled onto the screen by switching back to instant review mode. Although the embodiment discussed above retains the last image on the LCD screen 402 until the shutter button 404′ is released and pressed into a first position, nothing prevents the removal of the last image for another reason. For example, in an alternate embodiment, the last image may be retained on the LCD screen for a given time interval. Finally, although reference is made to an instant review mode, the mode provides rapid review of the last image captured.
A method and system have been disclosed for providing an instant review mode in a digital camera. Although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations to the embodiments and those variations would be within the spirit and scope of the present invention. Accordingly, many modifications may be made by one of ordinary skill in the art without departing from the spirit and scope of the appended claims.
Continuing Application (prior application not abandoned): Continuation of prior application Ser. No. 09/584,657, filed May 30, 2000, now U.S. Pat. No. 6,512,548 which is a Continuation of application Ser. No. 08/890,896, filed Jul. 10, 1997, now U.S. Pat. No. 6,137,534.
Number | Name | Date | Kind |
---|---|---|---|
4456931 | Toyoda et al. | Jun 1984 | A |
4827347 | Bell | May 1989 | A |
5043816 | Nakano et al. | Aug 1991 | A |
5070406 | Kinoshita | Dec 1991 | A |
5164831 | Kuchta et al. | Nov 1992 | A |
5497193 | Mitsuhashi et al. | Mar 1996 | A |
5576759 | Kawamura et al. | Nov 1996 | A |
5633678 | Parulski et al. | May 1997 | A |
5724579 | Suzuki | Mar 1998 | A |
5815201 | Hashimoto et al. | Sep 1998 | A |
5845166 | Fellegara et al. | Dec 1998 | A |
5949950 | Kubo | Sep 1999 | A |
6137534 | Anderson | Oct 2000 | A |
Number | Date | Country | |
---|---|---|---|
20020186307 A1 | Dec 2002 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 09584657 | May 2000 | US |
Child | 10154496 | US | |
Parent | 08890896 | Jul 1997 | US |
Child | 09584657 | US |