Patent Application
20070273750
The above and other objects, advantages and features of the present invention will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:
The invention will be now described herein with reference to illustrative embodiments. Those skilled in the art will recognize that many alternative embodiments can be accomplished using the teachings of the present invention and that the invention is not limited to the embodiments illustrated for explanatory purposes.
Embodiments of the present invention are described below in detail with reference to the accompanying drawings. Precise positional control on a transportable mobile device such as a camera phone, a digital camera, or a digital video camera is difficult, for instance. The following embodiment enables formation of a composite image by notifying a user of a photographed area even in such portable device the position of which cannot be precisely controlled, to thereby aid in photography upon forming the composite image.
The above camera phone 100 compresses an image 200 taken with the camera 110 by the image compressing unit 120 and stores the compressed image in the auxiliary storage 150. In addition, the taken image stored in the auxiliary storage 150 is decompressed and decoded with the image decompressing unit 130 and then displayed on the display 180. The image compressing unit 120 and the image decompressing unit 130 are software which are driven by the CPU 160 reading and executing programs stored in the memory 170 or the auxiliary storage 150.
In addition, if sounds as well as images are recorded, the images are displayed on the display 180 and at the same time, the sounds are output from the speaker 140. In addition, the speaker 140 can additionally output button sounds or alert sounds. In addition, the display 180 and the speaker 140 of this embodiment function as a photographing condition notifying unit for notifying a user of a current photographing condition during or after photography as described below.
The keyboard 190 is an input unit via which a user enters instructions. For example, a command to start photography, a command to end photography, a delete command, a save command, an edit command, or the like can be input. In response to the user's instructions, the CPU 160 controls each block, reads necessary programs from the memory 170, and executes various operations based on programs.
Here, the camera phone 100 of this embodiment includes a photographing condition analyzing unit 10 for analyzing current photographing conditions for aiding a user in photography. The photographing condition analyzing unit 10 is a software that is driven by the CPU 160 reading and executing programs stored in the memory 170 or the auxiliary storage 150.
The photographing condition analyzing unit 10 is a processing unit for aiding a user in obtaining images necessary for generating a composite image through, for example, mosaicing processing or super-resolution processing. As described in detail below, this unit helps a user obtain necessary images during or after photography or sends an error notification to aid the user in obtaining a composite image.
For example, if mosaicing processing and super-resolution processing are carried out in combination, high-definition digital image data can be obtained even for a poster, print, or other such area larger than the angle of field of the camera, or an area whose image becomes indistinct if the entire area is photographed. A post processing unit (not shown) executing the mosaicing processing or the super-resolution processing is realized by the CPU 160 based on a captured image. The following description is made on the assumption that mosaicing and super-resolution processings are carried out on images of flat and rectangular areas for ease of explanation. In addition, the mosaicing and/or the super-resolution processing is referred to as “post processing”. Further, a rectangular area subjected to the post processing is referred to as a target area. Incidentally, an area to be photographed, that is, an area subjected to mosaicing and super-resolution processings may be, of course, a non-rectangular area or a non-flat area such as a landscape image.
Here, the mosaicing processing and super-resolution processing are described in brief. A mosaicing processing technique of combining plural partial images captured with a small camera to compose the images is combined with a super-resolution processing technique of generating a high-definition image based on a superimposed image of moving pictures, making it possible to read an A4-sized text with a camera of a camera phone or the like, for example, in place of a scanner. The mosaicing processing generates a wide-field image (mosaic image) of a subject that is flat or seemingly almost flat like a long-distance view, which exceeds the original angle of view of the camera. If the entire subject image cannot be taken by the camera, the subject is partially photographed plural times in different camera positions and orientations. The captured images are combined to generate the whole subject image.
In addition, the super-resolution processing combines plural images obtained by photographing a subject with the angle changed a little to assume/reconstruct data on details of a subject to generate a high-definition image beyond the intrinsic performance of the camera. In a super-resolution technique as disclosed in Japanese Unexamined Patent Application Publication No. 11-234501, a part of a subject is photographed while the camera position is changed and movements in moving pictures are analyzed to estimate camera movements such as a three-dimensional position of the camera or image-taking direction upon capturing each frame image on real time. Based on the estimated result, the mosaicing processing is carried out. Thus, a mosaic image can be taken while a camera is held in hand and freely moved without using a special camera scanning mechanism or positional sensor. Further, high image quality equivalent to a quality of an image read with a scanner is realized through super-resolution processing based on high-definition camera movement estimation.
Incidentally, to obtain correct results of the post processing, it is necessary to photograph the whole area of the target area with a proper amount. That is, if a captured image amount is too small, post-processing cannot be executed or even if executed, high-quality images cannot be obtained. However, in manual operations, a moving speed of the camera 110 should be determined based on use's guess. In this case, if a user is not accustomed to use a camera, there is a fear that a moving speed is set too high, and information necessary for the processing cannot be obtained. As a result, a desired super-resolution effect cannot be expected.
On the other hand, if the moving speed of the camera 110 is too low, a photographing period is elongated and image information is captured more than necessary. As a result, it takes much time to capture moving pictures. In addition, a larger data capacity is required of the auxiliary storage 150 to store the thus-captured images because of the long photographing period. Furthermore, mosaicing processing and super-resolution processing should be carried out on more information than necessary, resulting in a problem that processing time thereof is long, and the total processing time is increased.
To that end, the photographing device of this embodiment is provided with a photography condition analyzing unit 10 to aid a user to set a camera-phone moving speed optimum for photographing a target area to obtain a proper amount of images during and after photography. The user is thereby supported to capture the optimum amount of images for post-processing. Further, if there is little probability that a correct result is obtained due to very high moving speed of the camera phone, information that encourages a user to photograph the area again may be provided.
Next, the photographing result analysis executed by the photographing condition analyzing unit 10 is described in more detail. The following description is made of an example where the camera 110 captures moving pictures to subject the moving pictures to the post processing to obtain a composite image. Incidentally, this embodiment describes moving pictures by way of example, but a composite image may be generated based on plural still images.
Here, the image compressing unit 120 executes, for example, well-known image compression such as MPEG to compress a captured image. In this case, the image compressing unit 120 divides the entire photography area of the camera 110 into several macro blocks to execute processing for each block.
The speed detecting unit 11 of this embodiment receives the displacements 240 and 250 from a motion detecting unit 121. Then, the unit calculates a moving speed of the camera 110 based on the motion information. That is, an X-directional displacement 240, a Y-directional displacement 250, and Δ time 230 are obtained upon capturing moving images of a target area. Then, the moving speed of the camera 110 is calculated and compared with a proper camera moving speed based on these information. If the moving speed in question is not within the proper moving speed range, an error message is displayed on a display 180 or a speaker 140 gives warning beeps to inform a user that the speed is abnormal.
In general, the camera phone includes an image compressing unit 120 or similar image compressing unit to obtain motion information. In this way, motion information is obtained with the in-built image compressing unit 120 to calculate movement and in turn a moving speed. Hence, it is unnecessary to provide a motion information detecting unit or the like as an additional function. Further, as for a photographing speed for the optimum amount of moving images for post-processing, a speed range (upper limit and lower limit) is preset by experiment. Incidentally, a proper speed may be set or reset through plural user operations.
Further, the photographed area map creating unit 11 obtains the above motion information to thereby obtain information about the areas that have been captured at present time during photography or information about the whole captured areas after photography.
After the completion of photographing, the photography locus is displayed to create a map representing a photographed area as shown in
According to this embodiment, if the moving speed of the camera 110 is too high to obtain the optimum amount of images for post-processing, that is, information amount would be insufficient, warning beep or message is given to inform a user of this situation. Further, if the moving speed is too low to obtain the optimum amount of images for post processing, that is, post-processing amount and time would increase due to an excessive amount of information, and a large storage capacity would be necessary for saving captured moving images, warning beep or message is given to inform a user of this situation. As a result, a proper moving speed is visually and acoustically informed to the user to thereby capture moving images with the camera moving speed appropriate for post processing such as super-resolution processing without relying on user's memory or hunches.
That is, in general, a user does not need to rely on memory or follow one's hunches to control the moving speed of a handy camera phone. Thus, if the camera moving speed is too high to obtain data enough for processing, or if information is obtained more than necessary due to very low camera moving speeds. In contrast, if the camera moving speed is outside the preset speed range, a user is informed of this situation to thereby move a camera at a proper moving speed to reduce failures during photography.
Further, in this embodiment, the proper camera moving speed is-preset, the range can be changed. That is, although a larger image amount requires longer processing time, a high-quality composite image can be obtained. Hence, the upper and lower limits of the proper camera moving speed may be set variable to allow a user to select a quality of a composite image obtained through post processing before capturing moving images to set the upper and lower limits of the proper camera moving speed in accordance with the quality level.
Further, in this embodiment, whether or not the camera moving speed is proper is informed to a user with warning beeps from the speaker or warning messages displayed on the display. Instead, it can be informed to the user through vibrations. Further, if a user determines that the moving speed is too high to obtain images for post processing, photographing may be automatically suspended.
As described above, the proper camera moving speed is informed to a user to thereby obtain a proper amount of moving images and prevent the post-processing time from increasing. Further, it is possible to obtain a composite image of proper size through post-processing, and set an appropriate data storage area of the auxiliary storage 150 for storing the composite image.
If the moving speed is too fast to obtain sufficient information, a user is urged to recapture images by warning beeps or the like. As a result, a user can cancel post-processing if it is difficult to generate a desired composite image even through post-processing. It is possible to eliminate unnecessary processing and save processing time and power consumption.
Further, if the moving speed is low, a photographing period for a desired photography area is increased more than necessary, and in addition, images are captured beyond necessity. As a result, post-processing is elongated. The longer photographing period and post-processing period lead to larger power consumption. This undesirably reduces battery life especially in a small portable device such as a cell phone. In contrast, according to this embodiment, even if the moving speed is below the lower limit of the proper moving speed, the user is urged to recapture images by warning beeps or the like to adjust a captured image amount into an appropriate one, thereby preventing a battery from exhausting.
Incidentally, the photographed area map creating unit 11 may inform a user of at least one of a photographed area shape, overlap of photographed areas, and the locus of a camera that is photographing or has photographed a target area, during or after photography. Since the whole photographed area is displayed during or after photographing of the target area, proper control can be executed without relying on user's memory or hunches.
Further, it is possible to provide a mask image generating unit for displaying a photographed area in outline on all or a part of the display screen in the miniature form during photography as well as after photography (hereinafter referred to as “mask image”). Thus, a proper amount of moving images can be obtained for post-processing during photography.
Incidentally, although the above embodiment describes a method of executing mosaicing processing and super-resolution processing after capturing moving images. That is, moving images are captured independently, and after the completion of capturing moving images, mosaicing processing and super-resolution processing are executed. According to this method, a user needs to waits for a while until the post-processing is completed. If a high processor speed is secured, mosaicing processing and super-resolution processing may be executed while moving images are captured. If the moving images are captured in parallel to post-processing, a user can obtain a result of mosaicing processing and super-resolution processing substantially at the completion of capturing moving images or more quickly than a general post processing does. In this case as well, similar to the above embodiment, a photographed area may be displayed or abnormal moving speed is informed during photography to thereby provide a user with support information for obtaining a proper composite image.
Further, in the above embodiment, a photographed area is displayed or a moving speed is detected based on motion information in the image compressing unit 120, but the photographing device may capture and store moving images in an uncompressed form. In this case, a motion detecting unit may be separately provided to display a photographed area or detect a moving speed. Further, in this embodiment, a photographed area or moving speed is informed so as to obtain an appropriate overlap throughout the entire target area because the obtained image is subjected to post-processing such as mosaicing processing and super-resolution processing. However, the other image synthesizing processing can be carried out. In this case, a photographed area or camera moving speed may be informed so as to assist a user to photograph an area necessary for the image synthesizing processing.
Further, the above embodiment describes the hardware components. However, the invention is not limited thereto, and processing of each block can be also performed by a CPU (Central Processing Unit) executing a computer program. In this case, the computer program can be recoded on a recording medium and provided or transferred through the Internet or such other transfer media.
It is apparent that the present invention is not limited to the above embodiment that may be modified and changed without departing from the scope and spirit of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2006-135782 | May 2006 | JP | national |
