Patent Application
20130222376
1. Technical Field
The technical fields relates to a stereo image display device for displaying a stereo image.
2. Related Art
Conventionally, an apparatus for preparing a left-eye image and a right-eye image which have parallax as a stereo image, and projecting them on left and right eyes independently through shutter type eyeglasses to provide stereoscopic vision is present. With this apparatus, a user adjusts parallax of stereoscopic video signals while viewing a two-dimensional video obtained by synthesizing a left-eye image and a right-eye image in a multiplexed manner (for example, see Japanese Patent Application Laid-Open No. 2005-110120).
In another method, when pixels of which visual load for users is large due to large parallax increase in an image, the pixels of which visual load is large are colored to be displayed, and parallax is adjusted while these pixels are being viewed (for example, see Paper on Feasibility Study about Environmental Development of Next-Generation Stereoscopic Contents Production, The Mechanical Social Systems Foundation, March 2009, pages 41 to 43).
However, in the conventional technique, information about parallax obtained based on a left-eye image and a right-eye image can be adjusted on an entire image, but cannot be adjusted on a certain partial region.
The present disclosure provides a stereo image display device for while displaying an image based on a left-eye image and a right-eye image, being capable of adjusting parallax information on any region pointed within the image.
A stereo image display device according to the present disclosure includes an image input unit operable to obtain image data representing a stereo image, a display unit operable to display an image based on the image data obtained by the image input unit, an operation unit operable to receive pointing position on an image displayed on the display unit, an information calculator operable to obtain parallax information on a portion corresponding to the position pointed by the operation unit in the image data obtained by the image input unit, and a controller operable to control the display unit to display the parallax information obtained by the information calculator as well as the image. The display unit displays the parallax information in a format representing a magnitude and a direction of parallax. The operation unit receives a command for changing the magnitude of parallax represented by the parallax information. The controller changes a magnitude of parallax on the portion corresponding to the pointed position in the image data based on the changing command received by the operation unit.
With this configuration, the stereo image display device can point a position within the image based on image data representing a stereo image. Parallax information about a portion corresponding to the pointed position can be displayed together with an image displayed on a display unit. As a result, a user checks the image based on the image data representing the stereo image and simultaneously can check the parallax information on the pointed portion.
Further, the stereo image display device can change a magnitude of the parallax of the image data obtained by the image input unit according to a change in a size of the parallax information. As a result, the user can check the parallax information displayed on the display unit and adjust the parallax of the stereo image only through changing of the size of the parallax information.
The stereo image display device of the present disclosure can display the parallax information on the pointed position as well as at least the image based on the image data representing the stereo image. The parallax information on the pointed position can be easily adjusted. For this reason, the present disclosure can provide the stereo image display device that can be user-friendly for users.
Additional benefits and advantages of the disclosed embodiments will be apparent from the specification and Figures. The benefits and/or advantages may be individually provided by the various embodiments and features of the specification and drawings disclosure, and need not all be provided in order to obtain one or more of the same.
An embodiment is described below with reference to the drawings.
The imaging unit 1 has a first optical system 210, a second optical system 220, and a camera controller 230. The first optical system 210 is arranged on a first viewpoint position, and has a first lens group 211, a first imager 212, and a first A/D converter 213. The second optical system 220 is arranged on a second viewpoint position, and has a second lens group 221, a second imager 222, and a second A/D converter 223.
The first lens group 211 is composed of a plurality of optical lenses. The first lens group 211 collects light incident to the first imager 212.
The first imager 212 is composed of an imaging device, and captures light incident through the first lens group 211. Concretely, the first imager 212 converts an input light signal into an analog signal (electric signal), and outputs the analog signal to the first A/D converter 213.
The first A/D converter 213 converts the analog signal output from the first imager 212 into a digital signal. The first A/D converter 213 outputs the converted digital signal as a first image to the parallax information calculator 2 and the signal processor 3.
The second lens group 221 is composed of a plurality of optical lenses. The second lens group 221 collects light incident to the second imager 222.
The second imager 222 is composed of an imaging device, and captures light indent through the second lens group 221. Concretely, the second imager 222 converts an input light signal into an analog signal (electric signal), and outputs the analog signal to the second A/D converter 223.
The second A/D converter 223 converts an analog signal output from the second imager 222 into a digital signal. The second A/D converter 223 outputs the converted digital signal as a second image to the parallax information calculator 2 and the signal processor 3.
In the above configuration, the first optical system 210 and the second optical system 220 are separate from each other. However, the first optical system 210 and the second optical system 220 may be formed into the same (single) device. In other words, the first optical system 210 and the second optical system 220 may have any configuration as long as they can obtain a first image on the first viewpoint position and a second image on the second viewpoint position.
The camera controller 230 controls the respective units of the imaging unit 1 to perform operations corresponding to shooting parameters, such as a focal distance and an aperture value under control of the controller 9.
The parallax information calculator 2 calculates information about parallax of a stereo image composed of a first image and a second image (hereinafter, parallax information) based on image data composing the input first image and second image. The parallax information calculator 2 outputs the calculated parallax information to the GUI generator 7. For example, the parallax information calculator 2 divides the first image and the second image into a plurality of regions, and calculates the parallax information for each of the divided regions. The divided regions may be of any size, for example, 16×16 pixels. The parallax information calculator 2 may use any method such as block matching method for calculating the parallax information.
The parallax information means a value indicating a shift amount (hereinafter, “a magnitude of parallax”) in a horizontal direction of a horizontal position of an object on a second image with respect to a horizontal position of the object on a first image when the object is commonly captured on both the first image and second image and a position of the object on the second image is different from a position of the object on the first image. For example, the parallax information may be a pixel value as the shift amount in the horizontal direction. As to display of the parallax information, a magnitude of parallax may be displayed in a numerical value, or may be displayed in a vector as described later. In this case, the parallax information is a concept including the magnitude (amount) of parallax and an orientation (direction) of parallax.
The first image and the second image may be images generated by the imaging unit 1 or images read from the recording medium 11. That is to say, images to be input into the parallax information calculator 2 are not limited to the first image and the second image generated by the imaging unit 1.
Concretely, the parallax information calculator 2 calculates parallax information corresponding to a position pointed by a user through the operation unit 6 based on a signal from the controller 9.
When the user points a position 1501 through the operation unit 6, the parallax information calculator 2 calculates parallax information on a region 1502 including the position 1501. In this case, an average value of parallax information (a magnitude of parallax, a direction of parallax) is calculated from image data included in the region 1502. The parallax information calculator 2 may calculate parallax information on the position 1501 instead of parallax information on the region 1502. In short, the region 1502 may be composed of a plurality of pixels or one pixel.
Further, the parallax information calculator 2 outputs to the controller 9 maximum parallax information in the parallax information on the entire region of the image. The maximum parallax information means parallax information in which a magnitude of parallax composing the parallax information is maximum in a plurality of pieces of parallax information. Further, the maximum parallax information means at least one piece of parallax information about an object viewed as being popped out most (the closest to the user) and parallax information about an object viewed as being retreated most when the user views a first image and a second image as a stereoscopic video. In short, the parallax information about the object viewed as being popped out most may be the maximum parallax information. Further, the parallax information about the object viewed as being retreated most (farthest from the user) may be the maximum parallax information. Further, both parallax information about the object viewed as being popped out most and the parallax information about the object viewed as being retreated most may be the maximum parallax information.
The signal processor 3 executes various processes on the first image and the second image generated by the imaging unit 1. The signal processor 3 executes a process on image data composing either or both of the first image and the second image, generates a review image as the image data to be displayed on the display unit 5, and generates a video signal to be recorded. For example, the signal processor 3 executes various video processes such as gamma correction, white balance correction and flaw correction on the first image and second image. The signal processor 3 outputs the generated review image to the display processor 4. A review image generated by the signal processor 3 may be a two-dimensional image or a three-dimensional image.
The signal processor 3 executes a compressing process on the processed first image and second image according to a compressing format that meet the JPEG standards. Compressed signals obtained by compressing the first image and the second image are related to each other and are recorded in the recording medium 11 via the recording processor 10. For example, the compressed signals are recorded in an MPO file format upon recording of the respective compressed signals. When a video signal to be compressed is a moving image, a moving image compressing standard such as H.264/AVC is employed. Further, the image in the MPO file format and the JPEG image or the MPEG moving image may be simultaneously recorded. The compressing format and the file format that are applied to the recording of the first image and the second image may be any formats as long as they are suitable for stereo images.
The signal processor 3 executes the signal processes on the first image and the second image based on a signal input from the controller 9, and adjusts parallax. Concretely, the signal processes executed by the signal processor 3 are realized by, for example, a trimming process. The signal processes executed by the signal processor 3 are not limited to the above method, and may be any methods as long as parallax can be electronically adjusted.
The signal processor 3 can be realized by DSP or a microcomputer. Resolution of a review image may be set equal to image resolution of the display unit 5, or may be set equal to resolution of image data which is compressed and generated by the compressing format conforming to the JPEG standards.
The display processor 4 superimposes a GUI image input from the GUI generator 7a on a review image input from the signal processor 3. The display processor 4 outputs a video signal obtained by the superimposing to the display unit 5.
The display unit 5 displays a video signal input from the display processor 4.
The operation unit 6 includes a touch panel and receives a touching operation from the user. When receiving the touching operation from the user, the operation unit 6 converts the operation into an electric signal to output it to the input unit 8. The user can point any position in an image displayed on the display unit 5 through the touching operation of the operation unit 6. The operation unit 6 is not limited to the touch panel, and may be composed of an operation member that can input information about up, down, right and left directions. The operation unit 6 may be a joy stick that can input information about any direction. That is to say, as the operation unit 6, any device may be used as long as it receives user's operations.
The GUI generator 7 generates a GUI (Graphical User Interface) image based on a signal input from the controller 9. For example, the GUI generator 7 generates and displays a GUI image relating to parallax information on a position pointed by the user through the operation unit 6 within a video displayed on the display unit 5. The GUI generator 7 generates the GUI image including the parallax information corresponding to the position pointed by the user through the operation unit 6 within the video displayed on the display unit 5. The GUI generator 7 may generate a GUI image including parallax information in a vector format as a display element. The vector format is a display format that clarifies a parallax magnitude (an amount of parallax) and a parallax direction (an orientation) including the parallax information, and, for example, a display format expressed by an arrow. That is to say, the parallax direction is expressed by an orientation of the arrow, and the magnitude of parallax is expressed by a length of the arrow. When the parallax information is displayed in the vector format, the format is not limited to the arrow and parallax information may be displayed in another format for clarifying its direction and its magnitude.
The input unit 8 receives an electric signal from the operation unit 6, and outputs a signal based on the received electric signal to the controller 9.
When an electric signal on the same position on an image displayed on the display unit 5 is sequentially input from the operation unit 6 through a user's operation of the operation unit 6, the input unit 8 outputs a signal for instructing parallax adjustment to the controller 9.
Further, when an electric signal for specifying a dragging operation is input from the operation unit 6 through a user's operation of the operation unit 6, the input unit 8 outputs a signal for instructing the adjustment of parallax according to the dragging operation to the controller 9.
The controller 9 entirely controls the stereo camera 200.
The recording processor 10 records the first image and the second image input from the signal processor 3 into the recording medium 11.
A display screen displayed on the display unit 5 based on a video signal generated by the display processor 4 in this embodiment is described below with reference to the drawings.
The user can instruct operations in the up-down, right and left directions and determination via the operation member 301.
The display unit 5 and the operation unit 6 are configured integrally as a display screen 1601.
The user can perform a touching operation or a dragging operation via the display screen 1601. When receiving the touching operation from the user, the stereo camera 200 displays a region 1602 including a position touched on the display screen 1601 that is indicated by a frame of broken line. The user can move the region 1602 via the operation member 301. For example, when the user instructs the operation in the up direction via the operation member 301, the region 1602 moves to the up direction. Further, the user can point a position of the region 1602 via the operation member 301.
A display operation performed on the display unit 5 by the display processor 4 is described.
The display screen 1601 displays a video signal generated by the display processor 4 (step S1701).
In this state, the stereo camera 200 determines whether a position on the display screen is pointed by the user (step S1702). The position on the display screen is pointed by the touching operation of the operation unit 6 and an operation on the region 1602 by the operation member 301. When receiving the pointing of the position, the stereo camera 200 calculates parallax information on a position pointed by the user (step S1703).
The stereo camera 200 displays the calculated parallax information on the display screen 1601 (step S1704).
The display format of parallax information on a display screen 1601 is described with reference to various examples.
When parallax information is displayed on the display screen 1601, the stereo camera 200 may be displayed on not only the vicinity of the region 1801 but also any region of the display screen 1601. For example, the stereo camera 200 may display parallax information on a lower end portion or an upper end portion of the display screen 1601.
As shown in
2-3. Display of Parallax Information about Most Popping-Out Position and Most Retreated Position Together
When receiving an operation from the user, the stereo camera 200 may, as shown in
2-4. Display in the Case where the Magnitude of Parallax Exceeds Allowable Value
When the magnitude of parallax in parallax information exceeds an allowable value, the stereo camera 200 may change the display format of parallax information on the display screen 1601. The allowable value is a limit value, for example, with which the user can view a stereo image as a three-dimensional video. The allowable value may be a value to be set by the operation member 301 in advance or a value to be set by the user.
The stereo camera 200 compares the allowable value of parallax and the magnitude of parallax in parallax information included in an image, and when the magnitude of parallax exceeds the allowable value as a result of the comparison, it changes the display format of the parallax information exceeding the allowable value. For example, as shown in
When an arrow is displayed, the stereo camera 200 may change a color according to a degree of exceeding the allowable value. For example, the stereo camera 200 may display an arrow so that as the degree that the magnitude of parallax exceeds the allowable value is larger, the color of the arrow becomes deeper red. In short, the stereo camera 200 changes the display format of parallax information to be displayed on the display screen 1601 using the allowable value as a threshold. The display format may be changed into any display format as long as the user can recognize that parallax exceeds the allowable value.
When parallax information on a plurality of regions is displayed, the stereo camera 200 may display not only parallax information on a position pointed by the user's touching operation but a predetermined number (plurality) of parallax information in decreasing order of parallax. As a result, the user can easily determine the direction and the magnitude of parallax on a plurality of positions on the screen with reference to a plurality of arrows displayed on the display unit 5. The user can intuitively adjust the magnitude and the direction of parallax through the touch panel.
2-5-1. Display with Different Marks and Bar Graph
In
2-5-2. Display with Different Zebra Patterns and Bar Graph
In
The stereo camera 200 determines a focus region based on a stereo image, and may display parallax information in the focus region.
In
These arrows are generated by the GUI generator 7 and are superimposed on a stereoscopic video signal by the display processor 4. The arrow images are generated so that the right-pointing arrow indicates the retreat direction, and the left-pointed arrow indicates the popping-out direction, and the lengths of the arrows indicate the magnitude of parallax. The arrow in the retreat direction may be shown by red, the arrow in the popping-out direction may be shown by blue, and the arrow of the focus point may be shown by green.
The user determines whether the magnitude of parallax is increased or decreased and when being increased or decreased, determines a direction of increasing or decreasing with reference to the above three arrows. In a case of
In this case, the user performs an operation for reducing the magnitude of parallax indicated by the right-pointing arrow 110 in the procedure similar to that described with reference to
When parallax is adjusted in such a manner, the user adjusts parallax while concerning about the arrow 110 and the arrow 111 indicating large parallax on the screen. However, as a result, parallax of an object on a focus point as a main object might be large. Therefore, parallax information on the focus point is displayed by, for example, a green arrow so that parallax information about the main subject can be checked during the parallax adjustment.
With the above-described operation, when the user adjusts parallax information, parallax information about the maximum parallax can be easily viewed, whereas the parallax information about the main subject can be prevented from being adjusted into a state that is different from a user's intention.
The adjustment of parallax information displayed on the display surface of the display unit 5 by the stereo camera 200 is described below with reference to the drawings.
When parallax information is displayed on the display screen of the display unit 5, the stereo camera 200 can adjust the magnitude of the displayed parallax according to an operation of the operation unit 36 performed by the user.
3-1. Adjustment of Parallax through Dragging Operation
When parallax information is displayed on the display screen of the display unit 5 and the user performs the dragging operation on the operation unit 6, the stereo camera 200 adjusts the magnitude of displayed parallax.
When the user touches the operation unit 6, the stereo camera 200 displays parallax information (arrow) 121 on a touched position on the display unit 5 as shown in
The above described that parallax information is increased by the dragging operation. However, an operation of the stereo camera 200 is not limited to the above operation and the stereo camera 200 may decrease the magnitude of parallax when the dragging operation is performed. In this case, the user performs the dragging operation in a direction opposite to the direction of the arrow.
With the above operation, the user can easily determine the magnitude and direction of parallax on a position pointed by the touching operation through the arrow displayed by the display unit, and can intuitively adjust the magnitude and direction of parallax through the touch panel.
3-2. Adjustment of Parallax through the Operation Member
The stereo camera 200 can adjust parallax information also through an operation other than the dragging operation.
For example, the stereo camera 200 can adjust the magnitude of parallax information using the operation member 301 shown in
Concretely, for example, the user performs an operation for tracing the arrow displayed on the display unit 5 to the left direction in order to reduce the parallax information indicated by the right-pointing arrow. In this case, the operation unit 6 detects this operation, the input unit 8 outputs it as operation information to the controller 9. As a result, the controller 9 controls the signal processor 3 to perform parallax adjustment to reduce the magnitude of parallax in parallax information in the retreat direction.
3-3. Adjustment of Parallax through Tap Operation
The stereo camera 200 may be configured so that an arrow displayed on the display screen of the display unit 5 is tapped sequentially in terms of time and thereby parallax information is gradually adjusted. The tap operation is for sequentially performing the touch operation at any number of times. That is to say, when the user sequentially taps a displayed arrow through the operation unit 6, the stereo camera 200 makes a control so that parallax information of the tapped arrow is adjusted by preset magnitude. When the tap operation is received sequentially twice, a control may be made so that the parallax information is increased. When the tap operation is received sequentially three times, a control may be made so that the parallax information is decreased. As shown in
When receiving the touching operation via the operation unit 6, the stereo camera 200 may make a control so that the length of the touched arrow (a magnitude of parallax) becomes 0.
In
In the stereoscopic video before adjustment, a magnitude of parallax of an innermost cube 141 is a difference 60 between the first image 1401L and the second image 1401R. An operation for displaying the cube 141 on the display screen at a time of reproduction that is desired to be performed by the user is described.
When the user touches a position 66 desired to be adjusted on the second image 1401R displayed on the display unit 5, the operation unit 6 detects this operation, and the input unit 8 outputs it as operation information to the controller 9. As a result, the controller 9 controls parallax so that a magnitude of parallax on the touched position 66 is adjusted to 0.
When the user touches the touch panel, parallax on any position on an image can be 0, namely, a portion corresponding to that position on the image can be displayed on the screen of the display device (position of parallax 0) as described above.
A process to be executed after the change of parallax information in the above manner is described.
A signal process that is executed by the stereo camera 200 in the case where parallax information is changed is described.
When parallax information is changed on the screen by a user's operation, the stereo camera 200 executes a signal process for shifting a first image and a second image so that the image is fitted to changed parallax information. This process is a shifting process.
As shown in
With such a process, the parallax information calculated from the first image and the second image can be also reduced according to the user's parallax adjustment.
3-4-2. Adjustment of Parallax Information According to Change of Shooting parameter
Parallax may be adjusted in such a manner that parallax information is adjusted not by the signal process but by adjusting a shooting parameter of the imaging unit 1.
In the position relationship between the imaging unit 1 and the object shown in
Before the adjustment of parallax information, as shown in
When the user touches a vicinity of an image of the cube 44 for adjusting parallax, parallax information about an image of the captured cube 44 is displayed as an arrow 400 as shown in
Concretely, the camera controller 230, as shown in
An object captured in this state, namely, the circular cylinder 43 and the cube 44 appear on a synthesized image signal output from the display processor 4 after the adjustment of parallax information as shown in
At this stage, the user satisfies a level at which the arrow is short, namely, a level of parallax information adjustment, and stops the operation for tracing the arrow to the left direction. Thereafter, parallax information at this time is maintained and a stereoscopic video is captured. When parallax is adjusted by the optical system in such a manner, the mask region 153 at both ends of the image shown in
As a result of adjusting actual parallax information according to the adjustment of parallax information by the user, the position having the maximum parallax on the image is occasionally changed. In this case, the stereo camera 200 may display information representing that the position having the maximum parallax is changed on the display unit 5.
In
The images of the arrows 110 and 111 are generated by the GUI generator 7, and are superimposed on a stereoscopic video signal to be displayed by the display processor 4. Arrow images are generated so that the right-pointing arrow indicates the retreat direction, the left-pointing arrow indicates the popping-out direction, and the lengths of the arrows indicate parallax information. Since two kinds of arrows are displayed, the arrow of the retreat direction may be red, and the arrow of the popping-out direction may be blue to distinguish colors of the arrows.
The user then determines whether parallax information is increased or decreased using the two arrows as references, and when increased or decreased, determines a direction. Before the adjustment in
In this case, when the user performs an operation for reducing parallax information represented by the right-pointing arrow 110 in a similar procedure to that described in
When an amount of parallax adjustment by the user is large, although parallax information on the position of the arrow 110 is maximum on the entire screen before the adjustment, the position having the maximum parallax on the screen during the adjustment is occasionally changed into the position of the arrow 113. In this case, when the controller 9 determines change of the position having the maximum parallax, it controls the GUI generator 7 so that an arrow 114 for warning the change of the maximum parallax information is displayed on the display unit 5 via the display processor 4. As to the warning, the arrow 114 may be deleted after it is blinked only for a constant time, or a warning sound may be generated simultaneously with the display.
With the above-described operation, when the position where parallax information is maximum is moved as a result of adjusting parallax information by the user, the warning of this state can prevent parallax from being adjusted until parallax information on another position is excessive due to much attention to a certain portion on the screen.
When the stereo camera 200 displays the screen shown in
The user determines whether parallax information is increased or decreased using two bar graphs 90b and 91b shown in
With this control, the magnitude of parallax in the retreat direction on the position of the mark 90 reduces, and the bar graph 90b is displayed short as shown in
When a screen shown in
The user determines whether parallax information is increased or decreased using the two bar graphs 100b and 101b shown in
The above-described operation enables adjustment of parallax information on a plurality of positions on the screen displayed on the display unit 5 to be easily and intuitively adjusted through the touch panel after the direction and the magnitude of parallax information are determined.
The other operations in the stereo camera 200 are described below with reference to the drawings.
3-5-1. Stopping of Display of Parallax during Movement
The stereo camera 200 may make a control so that parallax information is not displayed on the display unit 5 while the stereo camera 200 is moving.
In
When the user stops the operation for panning the stereo camera 200 to the right direction, the stereo camera 200 detects this operation, and arrows 143, 144, 145 and 146 as the detected results of parallax information are displayed on the display unit 5 as shown in
In the examples of
When the user performs the operation for moving the position and the attitude of the stereo camera, such as panning and zooming during the parallax adjustment, the user does not adjusts parallax but resets a shooting area (framing) during the moving operation. For this reason, when parallax adjustment display is performed, the user's operation is hindered. In the above example, the parallax adjustment display is paused, this hindrance can be prevented.
Videos to the display unit 5 and the display format of parallax information are described below.
A shading process may be executed on a region having larger parallax than parallax on a touched position. The shading process is described below with reference to
When the user touches a boundary between an object desired to be high definition and an object to be shed on the second image displayed on the display unit 5, the operation unit 6 detects the touching, the input unit 8 outputs it as operation information to the controller 9. As a result, an arrow 71 representing parallax information on the touched position is displayed on the touched position. The arrow 71 is generated by the GUI generator 7, and is superimposed on the second image by the display processor 4. The arrow images are generated so that the right-pointing arrow indicates the retreat direction, the left-pointing arrow indicates the popping-out direction, and the lengths of the arrows indicate the magnitude of parallax.
The user then touches the arrow 71 displayed on the display unit 5 at a plurality of times to instruct addition of a signal process for shading a part of an image. The operation unit 6 detects this touching, and the input unit 8 outputs it as operation information to the controller 9. As a result, the controller 9 makes a control so that the shading process is executed on a region having parallax larger than the parallax information 70 on the touched position.
Concretely, the signal processor 3 uses a low-pass filter on regions of the first image and the second image having parallax larger than the magnitude of parallax 70 on the touched position to output the signal. As a result, an object closer to the display surface than the position of the arrow 71 is displayed highly definitely, and the definition of the object that is retreated or popped out at parallax larger than that on the position of the arrow 71 is reduced (shaded). Therefore, easily viewable 3D image can be captured.
With the above-described operation, the user determine the direction and the magnitude of parallax on the screen based on the arrow shown by the display unit to be capable of easily and intuitively adjusting visibility of the 3D image through the touch panel.
3-5-2. Enlarged Display of Portion on which Parallax Information is Displayed
In
When the user touches a region where parallax is desired to be adjusted on the image displayed on the display unit 5, the operation unit 6 detects the touching, and the input unit 8 outputs the operation information to the controller 9. As a result, an arrow 2001 indicating parallax information on the touched position is displayed on the touched position.
The user then touches the arrow 2001 displayed on the display unit 5 at a plurality of times, and instructs addition of a signal process for enlarging a part of an image. The operation unit 6 detects this operation, and the input unit 8 outputs it as operation information to the controller 9. The controller 9 makes a control to execute the process for enlarging an image of a region corresponding to a touched position. At this time, the arrow 2001 is also enlarged to be displayed.
With the above-described operation, the user determines the direction and the magnitude of parallax on the screen based on the arrow displayed on the display unit to be capable of intuitively and easily adjusting visibility of a 3D image through the touch panel.
In the stereo camera 200 according to this embodiment, parallax information (the direction of parallax (the retreat direction, the popping-out direction), and the magnitude of parallax) is calculated for each region of an image, and on a position pointed by a user's operation on the touch panel, a position where parallax is maximum, or a predetermined number (plurality) of positions in decreeing order of parallax, the positions, the direction of parallax and the magnitude of parallax are displayed in a determinable format. As a result, the user determines the direction of parallax and the magnitude of parallax on the screen displayed on the display unit, and can intuitively and easily reflects the adjustment of parallax information and shooting intension through the touch panel.
When a portion with large parallax is generated on a shooting image, the user immediately selects a preferable method from countermeasure methods such as a method for shooting an image of which parallax is adjusted for each stereoscopic viewing, a method for shading a portion where stereoscopic viewing is difficult, and a method for daringly shooting even if stereoscopic viewing is slightly difficult, and reflects the selected method to a shooting image. The stereo image display device that can perform such an operation can be realized.
The stereo camera 200 according to the embodiment includes the imaging unit 1, the display unit 5, the operation unit 6, the parallax information calculator 2, and the controller 9. The imaging unit 1 obtains image data representing a stereo image. The display unit 5 displays an image based on the image data obtained by the imaging unit 1. The operation unit 6 receives pointing position on the image displayed on the display unit 5. The parallax information calculator 2 obtains parallax information on a portion corresponding to the position pointed by the operation unit 6 on the image data obtained by the imaging unit 1. The controller 9 controls the display unit 5 so that parallax information obtained by the parallax information calculator 2 as well as the image is displayed. The display unit 5 displays parallax information in a format representing a magnitude and a direction of parallax. The operation unit 6 receives a command for changing the magnitude of parallax represented by the parallax information. The controller 9 changes the magnitude of parallax on a portion corresponding to the pointed position in the image data based on the changing command received by the operation unit 6.
With this configuration, the stereo camera 200 can point the position on the image based on the image data representing a stereo image. Further, the parallax information on the portion corresponding to the pointed position can be displayed together with the image displayed by the display unit 5. As a result, the user can simultaneously check parallax information on the pointed portion while checking the image based on the image data representing the stereo image.
According to the change in the magnitude of parallax information, the stereo camera 200 can change the magnitude of parallax in image data obtained by the imaging unit 1. As a result, the user can adjust parallax of a stereo image by checking parallax information displayed on the display unit 5 and changing only the magnitude of the parallax information.
Further, for example, the controller 9 controls the display unit 5 so that parallax information obtained by the parallax information calculator 2 is displayed near the position pointed by the operation unit 6.
With this configuration, the stereo camera 200 can display the pointed position and the parallax information with them being related to each other. As a result, the user just views an image based on image data representing a stereo image to be capable of checking a position of the displayed parallax information on the position of the image.
For example, the controller 9 controls the display unit 5 so that parallax information obtained by the parallax information calculator 2 is displayed in a vector format.
With this configuration, the stereo camera 200 can display the obtained parallax information in the vector format. As a result, the user can check the position of the displayed parallax information on the image, the magnitude and direction of the parallax information by only viewing an image based on the image data representing a stereo image.
For example, the image data representing the stereo image is obtained from the imaging unit 1, the operation unit 6 receives a command for changing a magnitude of parallax represented by the parallax information, and the controller 9 controls the shooting parameter in the imaging unit so that the magnitude of parallax is changed based on the changing command.
With this configuration, the stereo camera 200 can automatically change the shooting parameter in the imaging unit 1 according to the change in the magnitude of parallax composing the parallax information obtained by the parallax information calculator 2. At this time, the shooting parameter can be set so that the parallax of the stereo image to be captured by the imaging unit 1 is the magnitude of parallax in changed parallax information. As a result, the user can adjust the shooting parameter in the imaging unit 1 by only changing the magnitude of parallax composing the parallax information displayed on the display unit 5.
For example, the stereo camera 200 further includes the operation unit 6 for setting information about the magnitude of parallax of the image data representing the stereo image. The controller 9 controls the display unit 5 so that the display format of the parallax information is changed according to a case where the magnitude of the parallax information obtained by the parallax information calculator 2 is larger than the magnitude of parallax set by the operation unit 6 and a case where the magnitude of the parallax information obtained by the parallax information calculator 2 is smaller than the magnitude of parallax set by the operation unit 6.
With this configuration, the stereo camera 200 can change the display format of parallax information displayed on the display unit 5 around information set by the operation unit 6. As a result, the user can check whether the magnitude of parallax in the displayed parallax information is larger than the magnitude of parallax in the image data representing the set stereo image by only viewing the parallax information displayed on the display unit 5.
For example, the display unit 5 and the operation unit 6 are integrally configured as the touch panel that can detect user's touching operations at least while displaying an image based on an image data.
With this configuration, parallax information can be intuitively and easily adjusted through the touch panel.
For example, when detecting the touching operation on the display portion of parallax information continuously at a plural number of times during the display of parallax information, the operation unit 6 receives the touching operation as the changing command.
With this configuration, the stereo camera 200 can regard the continuous touching operation on the display portion of the parallax information during the display of parallax information as the changing command. As a result, the user can change the magnitude of parallax in the parallax information by performing the touching operation on the display portion of the parallax information at a plural number of times, checking the parallax information displayed on the display unit 5.
For example, when the operation unit 6 detects the dragging operation on the display portion of the parallax information during the display of the parallax information, it receives the dragging operation as the changing command.
With this configuration, the stereo camera 200 can regard the dragging operation on the display portion of the parallax information during the display of the parallax information as the changing command. As a result, the user can intuitively change the magnitude of parallax through the dragging operation, checking parallax information displayed on the display unit 5 in the vector format.
For example, the parallax information calculator 2 further obtains the maximum parallax information about the maximum parallax in parallax information about parallax of image data. When the position of a portion having the maximum parallax in the image data is changed due to the position of the portion having the maximum parallax in the image data is changed based on the changing command, the controller 9 controls the display unit 5 to display information representing that the portion having the maximum parallax in the image data is changed.
With this configuration, parallax information about the maximum parallax in parallax information about the parallax of the image data obtained by the imaging unit 1 is obtained, and the change in the position of the portion having the maximum parallax can be displayed on the display unit 5. As a result, when adjusting the magnitude of parallax information, the user can automatically recognize that the portion corresponding to the maximum parallax information is changed.
For example, the parallax information calculator 2 further obtains a predetermined number of pieces of parallax information in decreasing order starting from the largest magnitude of parallax in the parallax information about the image data. The controller 9 controls the display unit 5 to display parallax information on a portion corresponding to a pointed position and a predetermined number of pieces of parallax information.
With this configuration, the stereo camera 200 obtains parallax information having the largest magnitude of parallax and at least one of another parallax information having second largest parallax or later in the parallax information about parallax of the image data obtained by the imaging unit 1, and can display another parallax information as well as parallax information on the portion corresponding to the pointed position on the display unit 5. As a result, the user can check a relationship between the parallax information of the portion corresponding to the pointed position and another parallax information as well as the image data representing the stereo image on the display unit 5.
For example, the parallax information calculator 2 further obtains parallax information on a portion corresponding to a focus region on a stereo image in image data. The controller 9 controls the display unit 5 to display the parallax information on the portion corresponding to the pointed position and the parallax information on the portion corresponding to the focus region.
With this configuration, the stereo camera 200 obtains the parallax information on the portion corresponding to the focus region on the image data obtained by the imaging unit 1, and can display this obtained parallax information as well as the parallax information on the portion corresponding to the pointed position on the display unit 5. As a result, the user can check a relationship between the parallax information on the portion corresponding to the pointed position and the parallax information on the portion corresponding to the focus region as well as the image data representing the stereo image on the display unit 5.
For example, the controller 9 detects a movement of the stereo camera 200 and controls the display unit 5 to display only the image data obtained by the imaging unit 1.
With this configuration, the stereo camera 200 can control an ON/OFF state of the display of parallax information obtained according to the movement of the stereo camera 200. As a result, only when parallax does not greatly fluctuate temporarily, the user can check the parallax information as well as the image data representing the stereo image on the display unit 5.
The stereo camera 200 according to this embodiment can display parallax information on a pointed position at least as well as an image based on the image data representing the stereo image. Further, the parallax information on the pointed position can be easily adjusted. For this reason, this embodiment can provide the stereo camera 200 that is easy-to-use for users.
In the above embodiment, an image captured by the first optical system 210 and an image captured by the second optical system 220 are converted into digital signals by the AID converter 213 and the AID converter 223, respectively, thereafter the signal processes are executed for calculating parallax information and adjusting parallax information. However, for example, parallax information may be processed inside the imaging unit 1 and in a format of an analog signal.
Further, as the method for adjusting parallax information, the method for shifting a relative position between a right-eye image and a left-eye image to change parallax information, and a method for changing an optical axis angle of the optical system to change parallax information are used. However, any methods for enlarging or reducing an image can be used as long as parallax information is changed.
In the above embodiment, parallax information is changed by the user according to the dragging operation on the operation unit 6, but parallax information may be changed according to a pinch-in operation and a pinch-out operation for changing a gap between user's two fingers on the operation unit 6.
Further, the stereo camera according to the present disclosure may include a CPU (Central Processing Unit), a system LSI (Large Scale Integration), a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), and a network interface. Further, a drive device that can carry out reading from or writing into portable recording media such as a DVD-RAM, a Blu-ray disc and an SD (Secure Digital) memory card.
The stereo camera according to the present disclosure is incorporated into a digital video camera, a digital camera and a mobile telephone as a built-in system.
Respective functions of the stereo camera may be realized by installing programs for controlling the stereo camera (hereinafter, image capturing programs) into an HDD or a ROM and executing the image capturing programs.
The image capturing programs may be recorded in a recording medium readable by a hardware system such as a computer system and an embedded system. Further, the image capturing programs may be read by another hardware system via the recording medium to be executed. As a result, the respective functions of the stereo camera can be realized in another hardware system. Examples of the recording medium readable by the computer system are optical recording media (for example, CD-ROM), magnetic recording media (for example, hard disc), magneto-optical recording media (for example, MO), and semiconductor memories (for example, memory card).
Further, the image capturing programs may be saved in a hardware system connected to a network such as an Internet and a local area network. The programs may be downloaded into another hardware system via a network to be executed. As a result, the respective functions of the stereo camera can be realized in another hardware system. Examples of the network are a terrestrial broadcasting network, a satellite broadcasting network, a PLC (Power Line Communication), a mobile telephone network, a wire communication network (for example, IEEE802.3), and a wireless communication network (for example, IEEE802.11).
In another manner, the respective function of the stereo camera may be realized by an image capturing circuit built in the stereo camera.
The image capturing circuit may be formed by a full custom TSI (Large Scale Integration), a semi-custom LSI such as an ASIC (Application Specific Integrated Circuit), a programmable logic device such as an FPGA (Field Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), or a dynamic reconfigurable device of which circuit configuration can be rewritten dynamically.
Design data for forming the respective functions of the stereo camera in an image capturing circuit may be configured by a program described by hardware description language (hereinafter, an HDL program). The design data may be configured by a net list of a gate level obtained by synthesizing logics of the HDL program. The design data may be configured by macro cell information obtained by adding arrangement information and process conditions to the net list of the gate level. The design data may be configured by mask data where a dimension, timing and the like are defined. Examples of the hardware description language are VHDL (Very high speed integrated circuit Hardware Description Language), and Verilog-HDL, and System C.
The design data may be recorded in a recording medium readable by a hardware system such as a computer system and an embedded system. The design data may be loaded into another hardware system via a recording medium to be executed. The design data read by another hardware system via the recording media may be downloaded into a programmable logic device via a download cable.
Further, the design data may be retained in a hardware system connected to a network such as an Internet or a local area network. The design data may be downloaded into another hardware system via a network to be executed. The design data obtained by another hardware system via the network may be downloaded into a programmable logic device via a download cable.
The design data may be recorded in a serial ROM to be transferred to FPGA at an electrically connected time. The design data recorded in the serial ROM may be downloaded directly into FPGA at the electrically connected time.
Further, the design data may be generated by a microprocessor at a time of electrical connection to be downloaded into FPGA.
Further, the technical idea disclosed in the above embodiment can be adapted to a television receiver which has a receiver instead of an imaging unit.
The present disclosure can be used as the stereo camera for capturing a stereoscopic video signal, and particularly as a video camera recorder that is capable of easily adjusting parallax information using a touch panel at a time of capturing a stereoscopic video signal.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-231229 | Oct 2010 | JP | national |
This is a continuation application of International Application No. PLT/JP2011/005728, with an international filing date of Oct. 13, 2011, which claims priority of Japanese Patent Application No.: 2010-231229 filed on Oct. 14, 2010, the content of which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2011/005728 | Oct 2011 | US |
| Child | 13861796 | US |
