Patent Application
20060182437
Image mosaicing involves stitching together multiple separate sets of image information to create a composite still image. In particular, image mosaicing involves stitching together frames of digital image information that are captured by digital still cameras or digital video cameras. Image mosaicing is often utilized to generate a single panoramic image from a series of individual images.
The process of generating a high-quality image mosaic requires large digital storage and processing capacity. In order to provide the storage and processing capacity needed to create a high-quality image mosaic, a still or video digital camera can be connected to a support computer equipped with an image mosaicing application. Image information captured by the digital camera is uploaded directly from the digital camera to the support computer and stitched together by the mosaicing application into an image mosaic. The image mosaic generated by the support computer can be viewed through the support computer, printed, edited, and/or permanently stored. A stand-alone digital camera could be used to capture a sequence of images and to generate a high-quality image mosaic, however, additional processing power and storage capacity would be required to generate a high-quality image mosaic.
In general terms, a camera is disclosed herein, along with related methodology allowing a stitched panoramic image to be generated from a series of individual image frames onboard the camera and displayed on the camera display in a “real-time” environment. This allows a user to preview the completed panorama on the camera without the need to first upload the image frames onto a support computer. In this manner, the user is able to identify potential problems with the panorama and, if necessary, reacquire the individual image frames. In order to reduce the amount of onboard processor power and/or memory required, stitching may be performed on relatively lower resolution image frame files within the camera. When the preview indicates that image data for the panorama has been successfully acquired, then the full resolution image data may be uploaded to a support computer for processing.
For purposes of the description presented herein, the term “stitched image” is synonymous with the terms “panoramic image”, “panorama” and “mosaic”. The term “stitching” is synonymous with the term “mosaicing”.
With reference to
It is noted that the features discussed above are listed only for exemplary purposes. In practice, the camera could include additional and/or different features depending on the desired functionality of the camera, as will readily be appreciated by one skilled in the art.
The camera 10 may include the ability to acquire panoramic image data in the form of a plurality of overlapping individual images. The individual image data may then be uploaded to the support computer 100,
With reference again to
Since the process of generating a high-quality image mosaic requires large digital storage and processing capacity, this process is typically performed on a support computer (e.g., the support computer 100,
As can be appreciated, it is typically necessary to wait until the camera 10 is connected to the support computer 100 and the final composite mosaic view generated by the support computer 100 before any problems with the image capture become apparent. Examples of such problems include failure to capture all of the desired image data, failure to obtain adequate overlap between images and failure to obtain vertical alignment between the images. If a problem is not discovered until after the camera 10 is reconnected to the support computer 100, then it will generally be necessary to return to the scene where the images were obtained and to reacquire the images. Accordingly, it would be desirable, during the capture of image information that is to be utilized to create an image mosaic to view at least an approximate image mosaic in real-time (e.g., on the display 62 of the camera 10) in order to determine if the target scene has been properly captured. However, real-time image mosaic building is an expensive operation, requiring a relatively high level of onboard memory and processing power.
The camera 10 disclosed herein, however, addresses this problem by providing the ability to preview the stitched composite image on the camera itself. In order to reduce the memory and processing power required, the stitching operation is performed on a lower resolution image stored within the camera. The stitching operation may, for example, be performed on a lower resolution image sometimes referred to in the industry as a “screennail” image, as will now be described in further detail. The stitching process itself may be identical to any known stitching algorithm conventionally used on a personal computer or other support computer.
For each image acquired by the camera 10, a file may be created and stored within the camera memory 76. In addition to the raw image data for each image (which may or may not be compressed), the image file may also contain other information such as a header identifying and the image represented by the image data. Further, a “screennail” image may be stored within the file. The screennail image is a reduced resolution version of the image and may optionally be compressed. The screennail image may be display-sized such that it fills the visible area of the display 62 when displayed. Accordingly, the screennail image is much smaller than the raw image data discussed above. A typical raw image may, for example, be about 2592 by 1936 pixels whereas a typical screennail image may, for example, be about 320 by 240 pixels (this exemplary screennail resolution is sometimes referred to as “QVGA” resolution). The screennail image data may be generated from the raw image data at the time that the image file is created in a conventional manner. The generation and use of screennail images and image files is well known in conjunction with digital cameras and is discussed, for example, in U.S. Pat. No. 5,933,137 of Anderson, which is hereby incorporated by reference for all that is disclosed therein.
As discussed above, performing the stitching operation onboard the camera 10 allows a user to review a panorama immediately after capture to review for potential problems. If problems are detected, then the images making up the panorama may immediately be reacquired by the camera and reviewed again. Performing the stitching operation on the screennail image data (rather than the much larger raw image data) allows the stitching to be performed onboard the camera without the need for prohibitively costly higher onboard processor capability and/or memory capacity. If the stitched panorama previewed on the camera indicates no problems with the acquired panorama images, then the images may be saved in the camera memory for later uploading to a support computer. After uploading the data to the support computer, stitching may be carried out on the full-size image data in a conventional manner.
To perform an onboard preview of a panorama, a user of the camera 10 first acquires the individual panorama image frames (e.g., the image frames 140, 150,160,
After playback mode is entered, the user may then bring up the playback menu by pressing the “Menu/OK” button 66. The playback menu items are then displayed on the display 62 (and may, for example, be displayed as an overlay on the currently displayed image). The menu items may be navigated and selected using the scroll button 64 and Menu/Ok button 66 in a conventional manner. One of the playback menu items is a “Preview Panorama” option. When this option is selected by the user, the stitching/mosaicing operation previously discussed is performed on the individual image frames 140, 150, 160 to generate the panoramic image 180. The time required to generate the panoramic image 180 may, for example, be about ½ second per individual frame being stitched; although this, of course, depends upon hardware performance.
The panoramic image is then displayed on the display 62 for review by the user. Initially, the center portion of the panorama will be displayed on the display 62 and the panorama sequence will appear cropped by the right and left boundaries of the display (since the stitched panoramic preview image 180 (
It is noted that the user interface methodology discussed above is exemplary of only one of any number of possible scenarios. As can be appreciated, different menu options and/or pathways could readily be employed to achieve the desired result.
While illustrative and presently preferred embodiments have been described in detail herein, it is to be understood that the inventive concepts may be otherwise variously embodied and employed and that the appended claims are intended to be construed to include such variations except insofar as limited by the prior art.
