1. Field of the Invention
The present invention relates to an imaging apparatus and a movement controlling method.
2. Description of the Related Art
In order to obtain a left-eye image (an image which is viewed by a viewer with a left eye) and a right-eye image (an image which is viewed by the viewer with a right eye) for displaying a stereoscopic image using a digital still camera with a single imaging apparatus instead of a digital still camera for stereoscopic imaging, the camera is deviated in a horizontal direction by a parallax amount between the left-eye image and the right-eye image, and performs imaging twice. A stereoscopic imaging apparatus which extracts images with parallax from images imaged in advance is known (JP2009-3609A). A stereoscopic imaging apparatus in which controls a positional difference between a plurality of photographing positions based on the depth of a subject is also known (JP2003-140279A).
However, according to the invention disclosed in JP2009-3609A, there are few images having an appropriate parallax amount. Moreover, according to the invention disclosed in JP2003-140279A, while a plurality of photographing positions are controlled, the control will becomes cumbersome and complicated.
Therefore, an object of the present invention is to obtain image data for a stereoscopic image more easily.
An imaging apparatus (stereoscopic imaging apparatus) according to an aspect of the present invention includes an imaging unit which continuously images a subject in an imaging range and continuously outputs imaged image data, a first recording control unit which, if a recording instruction is given, records image data imaged at the timing, at which the recording instruction is given, in a recording medium as image data representing a first subject image, an object detection unit which detects all objects satisfying a predetermined condition from subject images for object detection among subject images represented by image data continuously output from the imaging unit, a first distance information calculation unit which calculates distance information between an object closest to the imaging apparatus and an object farthest from the object detection unit among a plurality of objects, a parallax amount decision unit which decides a parallax amount based on the distance information calculated by the first distance information calculation unit, and a second recording control unit which, when the imaging apparatus is deviated in a horizontal direction to make the amount of deviation between a subject image represented by image data continuously output from the imaging unit and the first subject image in the horizontal direction equal to the parallax amount decided by the parallax amount decision unit (including not only when both are perfectly equal but also when it is considered that both are substantially equal), records image data imaged at this timing as image data representing a second subject image in the recording medium in association with image data representing the first subject image.
Another aspect of the present invention provides a movement controlling method for an imaging apparatus. That is, in this method, an imaging unit continuously images a subject in an imaging range and continuously outputs imaged image data, if a recording instruction is given, a first recording control unit records image data imaged at the timing, at which the recording instruction is given, in a recording medium as image data representing a first subject image, an object detection unit detects all objects satisfying a predetermined condition from subject images for object detection among subject images represented by image data continuously output from the imaging unit, a first distance information calculation unit calculates distance information between an object closest to the imaging apparatus and an object farthest from the object detection unit among a plurality of objects, a parallax amount decision unit decides a parallax amount based on the distance information calculated by the first distance information calculation unit, and when the imaging apparatus is deviated in a horizontal direction to make the amount of deviation between a subject image represented by image data continuously output from the imaging unit and the first subject image in the horizontal direction equal to the parallax amount decided by the parallax amount decision unit, a second recording control unit records image data imaged at this timing as image data representing a second subject image in the recording medium in association with image data representing the first subject image.
According to the aspects of the present invention, the subject in the imaging range is continuously imaged. If the recording instruction is given, image data imaged at this timing is recorded in the recording medium (including a recording medium which is removable from the imaging apparatus, and a recording medium which is embedded in the imaging apparatus) as image data representing the first subject image. All objects (for example, a face of a character or an object having a spatial frequency equal to or greater than a predetermined threshold value) satisfying a predetermined condition are detected from any subject image for object detection among the subject images obtained by continuously imaging the subject. The distance information between the object closest to the imaging apparatus and the object farthest from the imaging apparatus among a plurality of detected objects is calculated. parallax amount (a parallax amount for allowing the first subject image to be viewed as a stereoscopic image) is decided based on the calculated distance information. If the imaging apparatus is moved by the user, and the parallax amount between the imaged subject image and the first subject image becomes equal to the decided parallax amount, image data imaged at the timing at which the parallax amount becomes equal is recorded in the recording medium as image data representing the second subject image in association with image data representing the first subject image. The stereoscopic image is obtained using the first subject image and the second subject image.
The imaging apparatus may further include a second distance information calculation unit which measures distance information from the imaging apparatus to each of a plurality of objects in the imaging range. In this case, for example, the first distance information calculation unit calculates the distance information between the closest object and the farthest object from the distance information to the closest object and the distance information to the farthest object calculated by the second distance information calculation unit.
The imaging unit may include an imaging element and a focus lens. In this case, the imaging apparatus further includes an AF evaluation value calculation unit which calculates an AF evaluation value representing the degree of focusing at each movement position from image data imaged at each movement position while moving the focus lens. The second distance information calculation unit measures the distance to each of the plurality of objects based on the position of the focus lens when the AF evaluation value calculated by the AF evaluation value calculation unit becomes equal to or greater than a threshold value. The focus lens freely moves the front side of the imaging element, that is, the subject side with respect to the imaging element.
For example, the parallax amount decision unit decides the parallax amount defined in advance when the second distance information calculation unit is able to measure only the distance to one object among a plurality of objects detected by the object detection unit.
The parallax amount decision unit decides the parallax amount based on the size of a display screen on which a stereoscopic image is displayed and the distance information calculated by the first distance information calculation unit.
The imaging apparatus may further include a setting unit which sets the size of a display screen on which a stereoscopic image is displayed. In this case, the parallax amount decision unit decides the parallax amount based on the size of the display screen set by the setting unit and the distance information calculated by the first distance information calculation unit. For example, the second recording control unit repeats processing for, when the imaging apparatus is deviated in the horizontal direction to make the amount of deviation between the subject image represented by image data continuously output from the imaging unit and the first subject image in the horizontal direction equal to any parallax amount of a plurality of parallax amounts decided by the parallax amount decision unit, recording image data imaged at this timing as image data representing the second subject image in the recording medium in association with image data representing the first subject image for the plurality of parallax amounts.
The imaging apparatus may further include a reading unit which reads image data representing the first subject image stored in the recording medium and image data representing the second subject image recorded in the recording medium from the recording medium in response to a stereoscopic reproduction instruction, and a display control unit which performs control such that a display device displays a first subject image represented by image data representing the first subject image and a second subject image represented by image data representing the second subject image read by the reading unit with deviation in the horizontal direction by the parallax amount decided by the parallax amount decision unit.
The imaging apparatus may further include an object type decision unit which decides the type of an object in the subject images for object detection. In this case, for example, the object type decision unit detects an object of a type defined in advance among the types of objects decided by the object type decision unit. For example, the object detection unit detects an object of a type excluding a type defined in advance among the types of objects decided by the object type decision unit.
The imaging apparatus may further include a distance calculation unit which calculates the distance to an object whose type is decided by the object decision unit. In this case, for example, the object detection unit detects an object excluding an object, whose distance calculated by the distance calculation unit is equal to or smaller than a first threshold value, and an object, whose distance is equal to or greater than a second threshold value greater than the first threshold value, among the objects of the types decided by the object type decision unit.
Preferably, the imaging apparatus further includes a display device which displays the first subject image on a display screen, and a touch panel which is formed in the display screen. In this case, for example, the object detection unit detects an object displayed at a position where the touch panel is touched.
In order to display a stereoscopic image, a left-eye image which is viewed by a viewer with a left eye and a right-eye image which is viewed by the viewer with a right eye are required. In a digital still camera for imaging a stereoscopic image, two imaging apparatuses are provided, the left-eye image is imaged using one imaging apparatus, and the right-eye image is imaged using the other imaging apparatus. In this example, a left-eye image and a right-eye image for displaying a stereoscopic image are obtained using a digital still camera with a single imaging apparatus, instead of the digital still camera for imaging a stereoscopic image with two imaging apparatuses.
There are a tree subject OB1, a character subject OB2, and an automobile subject OB3 in front of the digital still camera 1. The tree subject OB1 is closest to the digital still camera 1, and the character subject OB2 is second closest to the digital still camera 1. The automobile subject OB3 is farthest from the digital still camera 1.
First, the digital still camera 1 is positioned at a reference position PL1 and the subjects OB1, OB2, and OB3 are imaged, and image data representing the subject images of the subjects OB1, OB2, and OB3 is recorded. The subject images imaged at the reference position PL1 becomes left-eye images (may become right-eye images).
As described below, for example, a parallax amount dl suitable for displaying a stereoscopic image on a 3-inch display screen and a parallax amount d2 suitable for displaying a stereoscopic image on a 32-inch display screen are calculated.
The user moves the digital still camera 1 in a right direction while continuously (periodically) imaging the subjects OB1, OB2, and OB3. While the digital still camera 1 is moving in the right direction, the subjects OB1, OB2, and OB3 are imaged. When the digital still camera 1 is at a position PRI, if the parallax of the imaged subject images becomes the calculated parallax amount d1, the imaged subject images become right-eye images which are displayed on the 3-inch display screen, and are recorded as image data representing the right-eye images. When the user moves the digital still camera 1 in the right direction, and the digital still camera 1 is at a position PR2, if the parallax of the imaged subject images becomes the calculated parallax amount d2, the imaged subject images become right-eye images which are displayed on the 32-inch display screen, and are recorded as image data representing the right-eye images.
The display screen size setting image is used to set the size of a display screen on which a stereoscopic image is displayed. Image data representing the left-eye images and image data representing the right-eye images having a parallax amount corresponding to the size of a display screen set using the display screen size setting image are recorded.
A setting mode is set by a mode setting button in the digital still camera 1. If a display screen size setting mode in the setting mode is set, the display screen size setting image is displayed on a display screen 2 formed in the rear surface of the digital still camera 1.
In the display screen size setting image, display screen size input regions 3, 4, and 5 are formed. The size of a display screen is input to the input regions 3, 4, and 5 using buttons in the digital still camera 1.
If the stereoscopic imaging mode is executed, the subjects are imaged continuously (periodically), and the imaged subject images are displayed on the display screen in the rear surface of the digital still camera 1 as a motion image (through image). The user decides a camera angle while viewing a motion image being displayed on the display screen.
If a two-step stroke-type shutter release button is pressed (Step 11), the distance to a subject is calculated (Step 12). As the distance to the subject, while the distance to the character subject OB2 substantially at the center of the imaging range is calculated, the distance to another subject OB1 or OB3 in another portion of the imaging range may be calculated.
The distance to the subject can be calculated using the displacement of a focus lens.
The subjects are imaged while moving the focus lens from a NEAR position (or a home position) to a FAR position. Of image data obtained by imaging the subjects, the high-frequency component (AF evaluation value) of image data in the central portion of the imaging range is extracted. The distance to the subject OB2 in the central portion of the imaging range can be calculated from the displacement of the focus lens at a focus lens position PO when the AF evaluation value becomes a maximum value AF0.
Returning to
Then, a size variable i is reset to 1 (Step 15).
A necessary parallax amount is decided for each display screen size set in the display screen size setting (Step 16).
The relationship between a necessary parallax amount and the distance to a subject is defined in advance for each display screen size of which a stereoscopic image is displayed. The example shown in
In the table, the display screen size is 32-inch. A necessary parallax amount is set for every distance to a subject. The table is defined for every display screen size.
As described above, if the distance to a subject and the display screen size are decided, the necessary parallax amount is determined.
When the display screen size is set to 3-inch and 32-inch, the necessary parallax amount when the display screen size is 3-inch becomes d1 and the necessary parallax amount when the display screen size is 32-inch becomes d2 in accordance with the distance to a subject.
Returning to
If the necessary parallax amount between a left-eye image and a right-eye image necessary for displaying a stereoscopic image on all display screens of the set display screen size (3-inch and 32-inch) (YES in Step 17), timing starts (Step 19).
A message which requests the user to horizontally move the digital still camera 1 is displayed on the display screen, and the user moves the digital still camera 1 in the horizontal direction (the right direction, or when a reference image is a right-eye image, the left direction) according to the display (Step 20).
The image of the subjects are continued while the digital still camera 1 is moving, and so-called through images are continuously obtained. The amount of deviation between a first subject image and a through image is calculated (Step 21). The moving of the digital still camera 1 (Step 20) and the calculation of the amount of deviation between the first subject image and the through image are repeated (Step 21) until the calculated amount of deviation becomes equal to the necessary parallax amount.
If the calculated amount of deviation becomes equal to the necessary parallax amount (Step 22), image data representing a subject image (a second subject image or a right-eye image) imaged when the amount of deviation becomes equal to the necessary parallax amount is recorded in the memory card (Step 23). An image having an optimum parallax amount can be recorded without awareness of the user. Since an image according to a display screen size is recorded, it is possible to prevent an excessive increase in the parallax amount when a stereoscopic image is displayed on a large display screen. It is also possible to prevent imaging failure.
If subject images having all calculated necessary parallax amounts are not recorded (NO in Step 24), the processing from Step 20 is repeated unless the time limit elapses (NO in Step 25). If image data which represents subject images having all calculated necessary parallax amounts is recorded in the memory card, the processing in the stereoscopic imaging mode ends. As described above, when the set display screen size is 3-inch and 32-inch, the right-eye image for 3-inch having the parallax amount d1 and the right-eye image for 32-inch having the parallax amount d2 are obtained, the processing in the stereoscopic imaging mode ends,
In the foregoing example, the parallax amount for stereoscopically displaying a single specific subject in the imaging range is calculated, and a single right-eye image is generated for each display screen size. In contrast, in this modification, a parallax amount for stereoscopically displaying each of a plurality of subjects in the imaging range is calculated. A single right-eye image is generated for each subject and for each display screen size. As shown in
A subject variable j for representing the number of principal subjects in the imaging range is introduced. In the case of the subjects OB1, OB2, and OB3, the subject variable j becomes 1 to 3. The number of principal subjects may be input by the user, and as described below, the number of peak values (maximum values) of the AF evaluation value equal to or greater than a predetermined threshold value may be used. The subjects are divided into a foreground subject (a subject close to the digital still camera 1) OB1, a middle distance subject (a subject neither close to nor far from the digital still camera 1) OB2, and a background subject (a subject far from the digital still camera 1) OB3 in accordance with the distance from the digital still camera 1 to the subject. For each of the subjects OB1, OB2, and OB3, a necessary parallax amount appropriate for a display screen size is calculated. The calculated necessary parallax amount is stored in the table shown in
If the shutter release button is half-pressed (YES in Step 11), the distance to each of a plurality of principal subjects in the imaging range is calculated (Step 12A).
If the focus lens moves from the NEAR position to the FAR position during imaging, the high-frequency component is extracted from image data representing images in the entire imaging range, the graph having the relationship shown in
Returning to
The subject variable j and the size variable i are reset to 1 (Steps 26 and 15).
Then, the necessary parallax amount is calculated (Step 16). Initially, since the subject variable j is 1 and the size variable i is 1, for the foreground subject OB1, the necessary parallax amount appropriate for the display screen size of 3-inch is calculated (Step 16). From the graph having the relationship shown in
If the size variable i becomes the number of types of the display screen size (2 since the display screen size is 3-inch and 32-inch) (YES in Step 17), it is confirmed whether or not the subject variable j becomes the number of subjects (Step 27). If the subject variable j does not become the number of subjects (NO in Step 27), the subject variable j increments (Step 28). Accordingly, for the next subject, processing for calculating the necessary parallax amount for each display screen size is performed.
In this way, all of the necessary parallax amounts for the display screens for principal subjects in the imaging range are calculated. The calculated necessary parallax amounts are stored in the table shown in
In this modification, when there are a plurality of principal subjects in the imaging range, a representative distance to a subject is calculated, and a necessary parallax amount is calculated from the calculated representative distance.
As described above, if the shutter release button is pressed in the first step (YES in Step 11), the distances to a plurality of principal subjects in the imaging range are calculated (Step 12A).
As described above, if the distances to the principal subjects are measured, a table showing the distances is generated and stored in the digital still camera 1. For example, the distance to the foreground foreground subject OB1 is 1 m, the distance to the middle distance subject OB2 is 1.5 m, and the distance to the background subject OB3 is 3 m.
Returning to
Then, a representative distance representing the distance to the representative image is calculated (Step 28). As the representative distance, the average distance of the distances to a plurality of principal subjects in the imaging range, the distance to a subject closest to the digital still camera 1, or the like is considered. When the average distance is used as the representative distance, the parallax amount of the foreground subject increases. Meanwhile, when the representative distance is the closest distance, it is possible to prevent an increase in the parallax amount. Since the imaged subject images are displayed on the display screen in the rear surface of the digital still camera 1, a desired subject image may be selected from among the displayed subject image, and the distance to the selected subject image may be used as the representative image.
A plurality of subject images OB1, OB2, and OB3 (represented by the same reference numerals as the subjects) are displayed on the display screen 2. The user designates a representative image from among the subject images OB1, OB2, and OB3 with a finger F.
The display screen 2 is provided over the entire rear surface of the digital still camera 1. A plurality of subject images OB1, OB2, and OB3 are displayed on the display screen 2. A move button 6 is provided in the lower portion on the right side of the display screen 2. A decide button 7 is provided above the move button 6. A wide button 8 and a tele button 9 are provided above the decide button 7. A cursor 10 is displayed on the display screen 2. The cursor 10 moves on the images displayed on the display screen 2 in accordance with operation of the move button 6 by the finger F of the user. The cursor 10 is operated by the move button 6 so as to be located on a desired subject image. If the cursor 10 is positioned on a desired subject image, the user presses the decide button 7 with the finger F. When this happens, a subject image on which the cursor 10 is positioned becomes the representative image.
The distance to the representative image selected in this way is known from the position of the focus lens with the peak value of the AF evaluation value which is the high-frequency component obtained by extracting image data representing a representative image portion touched with the finger F or a representative image portion designated by the cursor 10 among image data obtained by repeating imaging while moving the position of the focus lens as described above in the same manner as shown in
If the representative distance to the representative image is calculated, as described above, the necessary parallax amount corresponding to the representative distance is calculated, and image data representing a subject image when the necessary parallax amount is reached are recorded in the memory card. Since image data which represents the subject image having the parallax amount corresponding to the representative distance is recorded in the memory card, there is no case where image data is recorded more wastefully than necessary.
In this modification, the necessary parallax amount calculated in the above-described manner is equal to or smaller than an allowable parallax amount value. When the parallax amount is large, while the viewer of the stereoscopic image feels a sense of discomfort, since the upper limit of the necessary parallax amount is restricted, it is possible to prevent the viewer of the stereoscopic image from feeling a sense of discomfort.
The photographing position of the left-eye image is represented by X1, and the photographing position of the right-eye image is represented by X2. It is assumed that there are a first subject OB11 comparatively close to the photographing positions X1 and X2, and a second subject OB12 comparatively far from the photographing positions X11 and X12. The first subject OB11 and the second subject OB12 are imaged at the photographing position X1, and the left-eye image is obtained. The first subject OB11 and the second subject OB12 are imaged at the photographing position X2, and the right-eye image is obtained.
Referring to
Referring to
The left-eye image 30L and the right-eye image 30R are superimposed such that the first subject image 31L in the left-eye image 30L shown in
A graph G1 which represents a parallax amount for allowing the subject to be viewed stereoscopically is defined to correspond to the distance to the subject. For example, if the distance to the first subject OB11 is 0.3 m, the parallax amount becomes 40 pixels. When the distance to the second subject OB12 farther than the first subject OB11 is 1.5 m, it is understood from a graph G2 that the allowable parallax amount value of the subject image of the second subject OB12 is 25 pixels. If the parallax amount of the subject image of the first subject OB11 is 40 pixels, the parallax amount of the second subject OB12 exceeds 25 pixels as the allowable parallax amount value. For this reason, in this example, the parallax amount of the first subject OB11 is set to the allowable parallax amount value 25 of the second subject OB12.
The necessary parallax amount of the subject is calculated from the distance to the subject using the graph G1 shown in
When there is a principal subject farther than the subject with the calculated necessary parallax amount (YES in Step 42), the allowable parallax value of the farthest subject from among the principal subjects farther than the subject with the calculated necessary parallax amount is calculated using the graph G2 (Step 43).
If the calculated necessary parallax amount exceeds the allowable parallax value (YES in Step 44), as described above, the allowable parallax value of the farthest subject becomes the necessary parallax amount (Step 45).
Where there are no principal subjects farther than the subject with the calculated necessary parallax amount (NO in Step 42), the processing of Steps 43 to 45 is skipped. If the necessary parallax amount does not exceed the allowable parallax value of the farthest principal subject (NO in Step 44), the processing of Step 45 is skipped. Of course, the same processing may be performed on principal subjects closer to the subject with the calculated necessary parallax amount.
It is possible to prevent an increase in parallax when displaying the stereoscopic image.
A file includes a header recording region 51 and a data recording region 52.
The header recording region 51 stores information for managing the file.
In the data recording region 52, image data representing a plurality of images, or the like is recorded.
A plurality of recording regions 71 to 78 are formed in the data recording region 52. The first recording region 71 and the second recording region 72 are the regions for the left-eye image. The third recording region 73 to the eighth recording region 78 are the regions for the right-eye image. If there are a large number of right-eye images which are represented by right-eye image data stored in the file, the number of recording regions may of course further increase.
In the first recording region 71, an SOI region 61 where SOI (Start Of Image) data representing the start of image data is stored, an auxiliary information region 62 where auxiliary information, such as an image number and image information representing whether or not image data to be successively recorded is a right-eye image or a left-eye image is stored, a region 63 where image data is recorded, and an EOI region 64 where EOI (End Of Image) data representing the end of image data is stored are formed. In the region 63 where image data of the first recording region 71 is recorded, image data representing the left-eye image is recorded. In the region 63 where image data of the second recording region 72 is recorded, image data which represents a thumbnail image of the left-eye image represented by left-eye image data recorded in the first recording region 71 is recorded.
Image data representing the left-eye image or the right-eye image obtained through imaging is recorded in the odd-numbered recording regions among the first recording region 71 to the eighth recording region 78, and image data representing the thumbnail image of the left-eye image or the right-eye image obtained through imaging is recorded in the even-numbered recording regions.
The third recording region 73 to the eighth recording region 78 are the same as the first recording region 71 and the second recording region 72 except that image data of the right-eye image is recorded. Of course, in regard to the right-eye image, data representing the display screen size and the position of a principal subject (foreground, middle distance, background, or the like) may be recorded in the auxiliary information in addition to the image number and the right-eye image.
Image data representing the left-eye image and image data representing a plurality of right-eye images obtained in the above-described manner are stored in the file and recorded in the memory card.
The overall operation of the digital still camera is controlled by a CPU 80, The digital still camera is provided with an operation device 81 which includes various buttons including a mode setting button which is used to set a mode, such as a stereoscopic imaging mode for parallax image generation, an imaging mode in which normal two-dimensional imaging is performed, a two-dimensional reproduction mode in which two-dimensional reproduction is performed, a stereoscopic reproduction mode in which a stereoscopic image is displayed, or a setting mode, a two-step stroke-type shutter release button, and the like. An operation signal which is output from the operating device 81 is input to the CPU 80.
The digital still camera includes a single imaging element (a CCD, a CMOS, or the like) 88 which images a subject and outputs an analog video signal representing the subject. A focus lens 84, an aperture stop 85, an infrared cut filter 86, and an optical low-pass filter 87 are provided in front of the imaging element 88. The lens position of the focus lens 84 is controlled by a lens driving device 89. The aperture amount of the aperture stop 85 is controlled by an aperture stop driving device 90. The imaging element 88 is controlled by an imaging element driving device 91.
If the stereoscopic imaging mode is set, a subject is imaged periodically by the imaging element 88. A video signal representing a subject image is output periodically from the imaging element 88. The video signal output from the imaging element 88 is subjected to predetermined analog signal processing in an analog signal processing device 92, and is converted to digital image data in an analog/digital conversion device 96. Digital image data is input to a digital signal processing device 96. In the digital signal processing device 96, predetermined digital signal processing is performed on digital image data. Digital image data output from the digital signal processing device is given to a display device 102 through a display control device 101. An image obtained through imaging is displayed on the display screen of the display device 102 as a motion image (through image display).
If the shutter release button is pressed in the first step, as described above, the subject is imaged while the focus lens 84 is moving. In a subject distance acquisition device 103, a high-frequency component is extracted from image data obtained through imaging, and the distance to the subject is calculated from the peak value or the like of the high-frequency component and the displacement of the focus lens. Image data is input to an integration device 98, and photometry of the subject is conducted. The aperture amount of the aperture stop 85 and the shutter speed (electronic shutter) of the imaging element 88 are decided based on the obtained photometric value.
If the shutter release button is pressed in the second step, image data imaged at the second timing represents the left-eye image. Image data which represents the left-eye image is given to and temporarily stored in a main memory 95 under the control of a memory control device 94. Image data is read from the main memory 95 and compressed in a compression/expansion processing device 97. Compressed image data is recorded in a memory card 100 by a memory control device 99.
Data representing the distance to the principal subject acquired in the subject distance acquisition device 103 (or the distance to one subject at the center of the imaging range) is input to a necessary parallax amount calculation device 105. In the necessary parallax amount calculation device 105, as described above, the necessary parallax amount is calculated. Data representing the distance to the principal subject is also given to a representative distance calculation device 104. The distance to a representative subject is calculated by the representative distance calculation device 104. Of course, as described above, when a representative subject is selected from among a plurality of principal subjects displayed on the display screen of the display device 102, the distance to the selected subject is calculated as the representative distance.
If image data representing the left-eye image is recorded in the memory card 100, the digital still camera itself is moved in the horizontal direction (right direction) by the user. The subject is continuously imaged while the camera is moving, and the subject images are continuously obtained. Image data obtained through continuous imaging is input to a through image parallax amount calculation device 106. In the through image parallax amount calculation device 106, it is confirmed whether or not the input subject image becomes the calculated necessary parallax amount. If the input subject image becomes the necessary parallax amount, image data representing the input subject image is recorded in the memory card 100 as right-eye image data. As described above, image data representing the right-eye image is recorded in the memory card 100 so as to have the parallax amount according to the display screen size.
The digital still camera also includes a light emitting device 82 and a light receiving device 83.
If the stereoscopic reproduction mode is set, left-eye image data recorded in the memory card 100 and, when right-eye image data corresponding to the display screen size of the display device 102 is recorded, right-eye image data are read. Read left-eye image data and right-eye image data are expanded in the compression/expansion processing device 97. Expanded left-eye image data and right-eye image data are given to the display device 102, and a stereoscopic image is displayed. When right-eye image data corresponding to the display screen size of the display device 102 is not recorded in the memory card 100, right-eye image data recorded in the memory card 100 may be read, and the parallax between the left-eye image and the right-eye image may be adjusted so as to become the parallax amount appropriate for the display screen size of the display device 102.
In this example, the necessary parallax amount is decided based on the distance (inter-object distance, distance information) between an object (an object closest to the digital still camera 1 among objects whose AF evaluation value is equal to or greater than a threshold value, called the closest object) closest to the digital still camera (stereoscopic imaging apparatus) 1 among a plurality of objects in the imaging range and an object (an object farthest from the digital still camera 1 among the objects whose AF evaluation value is equal to or greater than the threshold value, called the farthest object) farthest from the digital still camera 1.
There are a first object OB10, a second object OB20, and a third object OB30 in front of the digital still camera 1. The first object OB10 is closest to the digital still camera 1, the second object OB20 is second closest to the digital still camera 1, and the third object OB30 is farthest from the digital still camera 1. The first object OB10 is the closest object, and the third object OB30 is the farthest object.
When the first object OB10 is at a position indicated by reference numeral L11, and the third object OB30 is at a position indicated by reference numeral L31, the inter-object distance between the closest object and the farthest object becomes a comparatively short distance L1. Meanwhile, if the first object OB10 is at a position indicated by reference numeral L12 closer to the digital still camera 1 than reference numeral L11, and the third object OB30 is at a position indicated by reference numeral L32 farther from the digital still camera 1 than reference numeral L31, the inter-object distance between the closest object and the farthest object becomes a comparatively long distance L2.
When the inter-object distance is short, even if the left-eye image and the right-eye image are obtained in the above-described manner, the relative parallax between a principal subject (the second object OB20, and it is assumed that the principal subject is between the closest object and the farthest object) and the closest object or the farthest object decreases. For this reason, in this example, when the inter-object distance is short, the necessary parallax amount between the right-eye image and the left-eye image for forming the stereoscopic image increases. Conversely, when the inter-object distance is long, the relative parallax between the principal subject and the closest object or the farthest object increases. For this reason, in this example, when the inter-object distance is long, the necessary parallax amount between the right-eye image and the left-eye image for forming the stereoscopic image decreases.
As in the above description, first, the digital still camera 1A is positioned at the reference position PL11 and the objects OB10, OB20, and OB30 in the imaging range are continuously imaged. The objects OB10, OB20, and OB30 are detected from a subject image for object detection which is one subject image among the continuously imaged subject images. If a recording instruction is given, image data which represents the subject images of the objects OB10, OB20, and OB30 imaged at the timing at which the recording instruction is given is recorded. The subject images obtained at the reference position PL11 through imaging become left-eye images (may become right-eye images).
As described below, for example, a parallax amount d11 appropriate for displaying a stereoscopic image on a display screen of predetermined size is decided in accordance with the inter-object distance.
As in the above-described example, the user moves the digital still camera 1A in the right direction while continuously (periodically) imaging the objects OB10, OB20, and OB30. While the digital still camera 1 is moving in the right direction, the subjects OB10, OB20, and OB30 are imaged. When the digital still camera 1 is at a position PR11, if the parallax between the subject images obtained through imaging becomes the parallax amount d11 decided as described below, the subject images obtained through imaging become right-eye images (second subject images) which are displayed on a display screen of predetermined size, and are recorded as image data representing the right-eye images. A parallax amount appropriate for a display screen of different size is decided based on the inter-object distance, and if subject images having the decided parallax amount are imaged, image data representing the imaged subject images is recorded. Of course, a parallax amount may be decided based on the inter-object distance regardless of the size of the display screen. A setting unit which sets the size of the display screen for displaying the stereoscopic image may be provided in the digital still camera 1. In this case, a parallax amount is decided from the size of the display screen set by the setting unit and the inter-object distance. Of course, a table which represents the relationship between the size of the display screen, the inter-object distance, and the parallax amount may be defined in advance, and a parallax amount may be decided using the table.
The relationship between a necessary parallax amount and an inter-object distance is defined in advance for each display screen size of which the stereoscopic image is displayed. The example of
In the table, the display screen size is 3-inch. A necessary parallax amount is defined for each inter-object distance. The table is defined for each display screen size.
As described above, if the inter-object distance which is the distance between the closest object and the farthest object, and the display screen size are decided, the necessary parallax amount is decided. Of course, as described above, the necessary parallax amount is decided depending on only the inter-object distance without taking into consideration the display screen size.
If the camera angle is decided while imaging of a plurality of objects is continuously repeated, the shutter release button is half-pressed (Step 11). When this happens, all objects which satisfy a predetermined condition are detected from a subject image imaged at the timing at which the shutter release button is half-pressed (a subject image for object detection, and a subject image is not limited to a subject image imaged at the timing at which the shutter release button is half-pressed, and may be one subject matter among subject images to be continuously imaged) (Step 29). The inter-object distance which represents the distance between the closest object and the farthest object from among the detected objects is calculated (Step 12A).
The inter-object distance can be calculated as follows.
As described with reference to
If the shutter release button is full-pressed (recording instruction) (YES in Step 13), a subject image imaged at the timing at which the shutter release button is full-pressed becomes a first subject image and is recorded in the memory card (Step 14). As described above, the size variable i is reset to 1 (Step 15), and the necessary parallax amount is decided from the table (see
While the size variable i is incremented (Step 18) until the size variable i becomes equal to the number of types of the display screen size (Step 17), the necessary parallax amount corresponding to the size of the display screen and the inter-object distance is decided.
If the necessary parallax amount is decided, as described above (see
Although in the above-described example, a case where the inter-object distance which is the distance between the closest object and the farthest object can be calculated has been described, when only one object is detected in the imaging range, it is not possible to calculate the inter-object distance. In this case, a necessary parallax amount decided in advance (preferably, a necessary parallax amount defined in advance corresponding to the display screen) is decided.
Since
Image data representing the left-eye image is stored in the image data recording region 63 of the first recording region 71. Image data representing the thumbnail image of the left-eye image is stored in the image data recording region 63 of the second recording region 72.
As described above, image data which represents the right-eye image having the necessary parallax amount corresponding to the inter-object distance and the display screen size is recorded in the third recording region 73, the fifth recording region 75, and the seventh recording region 77. Thumbnail image data is recorded in the fourth recording region 74, the sixth recording region 76, and the eighth recording region 78.
In this way, image data which represents the left-eye image and the right-eye images for a plurality of frames is stored in a single file, and the file is recorded in the memory card.
Referring to
Referring to
The left-eye image 30L and the right-eye image 30R are superimposed such that the left-eye image 140L shown in
The digital still camera shown in
If the stereoscopic reproduction mode is set, as described above, image data which represents each of the left-eye image and the right-eye image recorded in the memory card 100 and corresponds to the size of the display screen of the display device 102 is read. Read image data is given to the display control device 101, such that the stereoscopic image is displayed on the display screen of the display device 102.
In the second example, as in the above-described first example, a parallax amount may be decided based on the size of the display screen on which a stereoscopic image is displayed and the distance information between the closest object and the farthest object. As shown in
As described above, if the shutter release button is half-pressed, a subject is imaged, and image data representing a subject image (subject image for object detection) is obtained.
In the image 160 for object detection, a road image 162 is in front, and an automobile image 161 is on the road image 162. A character image 163 is substantially at the center, and tree images 164 and 165 are on the left and right side of the character image 163. A cloud image 166 is on the upper left side of the image 160 for object detection. An upper portion in the image 160 for object detection is a sky image 167.
Referring to
If the subject image 160 for object detection is divided into a plurality of regions, the type of object which is represented by each divided region is decided with reference to a learning database (Step 154). The learning database stores the feature amount, such as the color, contrast, or brightness of an object, or the position when imaging, and the type of object in association with each other, and is stored in advance in the main memory 95 as described above. From the feature amount of each divided region, the type of object which is represented by the region can be decided.
As described above, the subject image 160 for object detection is divided into a plurality of regions 171 to 177. The region 171 represents an automobile as the type of object. Similarly, the region 172 represents a road, the region 173 represents a person, the regions 174 and 175 represents trees, the region 176 represents a cloud, and the region 177 represents a sky as the types of objects.
If the type of object is decided as described above (Step 181), it is determined whether or not the decided type of object corresponds to an object of a type defined in advance (Step 182). If the decided type of object corresponds to an object of a type defined in advance (YES in Step 182), the corresponding object is detected as an object. The closest object and the farthest object are decided from among the detected objects as described above, and the inter-object distance between the closest object and the farthest object is obtained as described above.
When deciding the parallax amount based on the inter-object distance between the closest object and the farthest object, there are objects which will be viewed stereoscopically and objects which will not be viewed stereoscopically. For example, objects which will be viewed stereoscopically are characters, automobiles, trees, buildings, and the like, and objects which will not be viewed stereoscopically are sky, roads, sea, and the like. An object which will be viewed stereoscopically or will not be viewed stereoscopically can be freely decided. For example, it may be presumed that sky, roads, sea, and the like will be viewed stereoscopically, and characters, automobiles, trees, buildings, and the like will not be viewed stereoscopically.
In this example, the type (for example, character, automobile, tree, building) of object which will be viewed stereoscopically is defined in advance, and it is determined whether or not the decided type corresponds to an object which will be viewed stereoscopically. It is possible to prevent the calculation of the inter-object distance between the closest object and the farthest object from the objects which will not be viewed stereoscopically. That is, it is possible to prevent the decision of the parallax amount such that an object which will not be viewed stereoscopically is viewed more stereoscopically.
In the processing procedure shown in
The type of object is decided as described above (Step 181). When this happens, it is determined whether or not the object of the decided type is an object (for example, road, sky, cloud, sea, or the like) of a type to be excluded defined in advance (Step 184). If the object of the decided type is not an object to be excluded (NO in Step 184), the object of the decided type is detected as an object (Step 183). If the object of the decided type is an object to be excluded (YES in Step 184), the object of the decided type is not detected as an object.
As described above, if the types of objects are decided (Step 181), the distance to each of the objects of the decided types is calculated (Step 191). As described above, the distance to the object can be calculated by extracting a high-frequency component from image data obtained by imaging the subject while moving the focus lens 84 (AF evaluation value), and using a graph showing the relationship between the AF evaluation value and the lens position of the focus lens 84. As shown in
In
In
As described above, the present invention is not limited to the distance to the automobile or the character, and the distance to a different object may be calculated in the same manner.
Returning to
As described above, if the subject image 160 for object detection is obtained through imaging, the subject image 160 for object detection is displayed on the display screen 2 (Step 201). A touch panel is formed in the surface of the display screen 2, and a desired object from the displayed subject image 160 for object detection is touched by the user (Step 202).
Referring to
As described above, the type of the object decided in the object decision processing may be displaced near the corresponding object of the subject image 160 for object detection as shown in
Number | Date | Country | Kind |
---|---|---|---|
2010-187316 | Aug 2010 | JP | national |
2011-020549 | Feb 2011 | JP | national |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2011/063799 | Jun 2011 | US |
Child | 13765430 | US |