1. Field of the Invention
The present invention relates to binoculars used by a user to view an enlarged remote scene or other objects with both eyes, and particularly to electronic binoculars that digitizes and displays images captured through imaging systems.
2. Description of the Related Art
There has been an apparatus developed or proposed as what is called digital binoculars including imaging units that convert image light beams into electric image signals and display units that display the images captured by the imaging units. The binoculars of this type are suitable to view a stationary subject or a nearly stationary subject.
That is, each of the imaging units includes an optical system, such as a lens having a relatively high magnification, enlarges and displays a remote, stationary subject, and shows the enlarged subject to a user who wears the digital binoculars. Since digital processing can be performed on the image signals, a variety of image processing operations can be carried out. For example, images captured in a dark environment are brightened before displayed, or hand-shake correction (the “hand-shake” used herein refers to a shake caused by hands) is made, as performed in a digital video camcorder.
JP-A-2004-133185 discloses an example of the electronic binoculars of this type.
The reason why such proposed electronic binoculars are used to view stationary objects is that when a moving object is viewed, it is difficult to continuously keep the object in focus even when the user tries to follow the moving object and keep it within the field of view.
That is, when an autofocus capability is equipped, for example, it is possible to some extent to follow a moving object and keep it in focus. However, consider a case where a relatively large area in a stadium where soccer or any other similar sport competition is in progress. When the user continuously follows a player, as a subject, moving across the large playing field, simply bringing the subject into focus by using an autofocus technology used in a video camcorder of related art may not be good enough to obtain images with the subject being in sharp focus. Specifically, another player in the field of view may be in focus, and which player in the field of view will be in focus disadvantageously depends on the conditions at the time of imaging.
Further, a hand-shake correction mechanism equipped in a video camcorder of related art is designed to provide a stable image by simply preventing the image being captured from being blurred due to a hand-shake. When such hand-shake correction is combined with the action of following a moving object, images viewed through the binoculars may not necessarily be appropriate.
While the above description has been made with reference to autofocusing and hand-shake correction, there have been a variety of problems with other features as well as focusing when these features are applied to electronic binoculars or image processing used in a video camcorder of related art is applied to these features.
Thus, it is desirable to provide electronic binoculars capable of comfortably viewing a moving subject.
An electronic binoculars according to an embodiment of the invention includes first and second imaging units with a predetermined horizontal distance therebetween disposed in a housing, optical members that guide image light beams to the first and second imaging units, a sensor that detects angular acceleration or acceleration acting on the housing, and an image processor. The image processor processes image signals produced by the first and second imaging units and corrects the image signals in terms of the change in motion of the housing in accordance with the angular acceleration or acceleration detected by the sensor. The embodiment further includes first and second displays with a horizontal distance therebetween disposed in the housing, and the first and second displays display the image signals processed by the image processor.
According to the embodiment of the invention, using image signals produced by the two imaging units to make motion correction in the image processor allows motion correction equivalent to what is called hand-shake correction on the housing to be made. In this case, using image signals produced by the two imaging units disposed with a horizontal distance therebetween to perform image processing allows sophisticated image processing suitable for the electronic binoculars to be performed.
According to the embodiment of the invention, using image signals produced by the two imaging units disposed with a horizontal distance therebetween to perform image processing in the image processor allows sophisticated image processing suitable for the electronic binoculars to be performed. For example, even when the electronic binoculars are inclined, motion correction can be made on the displayed images in such a way that the horizontal lines are kept oriented in a fixed direction. Alternatively, the image processor can extract only a subject of interest and display an image with the background removed.
An exemplary embodiment of the invention will be described below with reference to the accompanying drawings.
Electronic binoculars 100 according the present exemplary embodiment are integrally incorporated in a housing 120, and a user wears the electronic binoculars 100, when using it, on the face in front of the right and left eyes, for example, as shown in
The housing 120 of the electronic binoculars 100 houses a lens fixing mount 101a on the front side of the housing 120. The lens fixing mount 101a contains right and left lens systems 101L, 101R with a predetermined distance therebetween in the horizontal direction (transverse direction). Image light beams produced by the lens systems 101L and 101R are picked up by right and left imaging units 102L, 102R and converted into electric image signals. Each of the lens systems 101L and 101R is formed of a plurality of lenses including a focus lens, and moving the focus lens along the optical axis allows focus adjustment. Each of the lens systems 101L and 101R forms what is called a zoom lens the focal length of which can be changed. The zoom magnification that is typically used ranges from 2 to 10.
The imaging units 102L and 102R can be a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or any other suitable types of image sensors. A readout circuit suitable for the image sensor to be used is also provided. The distance between the two imaging units 102L and 102R is preferably at least several centimeters, which corresponds to the distance between the right and left eyes of the user who wears the electronic binoculars 100. The image sensor that forms each of the imaging units 102L and 102R has what is called a high-speed shuttering capability that allows the single-frame exposure period to be reduced.
The image signals produced by the right and left imaging units 102L, 102R are supplied to an image processor 104 and undergo a variety of image processing operations in the image processor 104 and its peripheral circuits. The imaging units 102L, 102R and the image processor 104 are disposed on the front or rear side of a substrate 103, on which a gyroscopic sensor 106 and an acceleration sensor 107 are also mounted. The outputs from the sensors 106 and 107 are used for hand-shake correction. The configuration for making the hand-shake correction will be described later in detail.
A linear motor 108 is attached to the substrate 103, and driving the linear motor 108 allows the focus lens in each of the lens systems 101L and 101R to be moved for focus adjustment. The right and left lens systems 101L, 110R are incorporated in the single lens fixing mount 101a and moved as a whole by the linear motor 108.
Right and left liquid crystal displays 110L, 110R are disposed with a predetermined distance therebetween on the rear side of the housing 120, and a sliding mechanism 109 is attached to the right and left liquid crystal displays 110L, 110R. The sliding mechanism 109 can adjust the horizontal distance between the two liquid crystal displays 110L and 110R. The detail of the adjustment mechanism will be described later. While the liquid crystal displays are used as a display means, the liquid crystal displays 110L and 110R may be replaced with any other suitable types of image displays.
As shown in
The outputs from the gyroscopic sensor 106 and the acceleration sensor 107 are supplied to the image processor 104, where image processing for hand-shake correction, which will be described later, is performed in accordance with the outputs from the sensors.
In accordance with a focus adjustment state detected in the image processor 104, a drive signal is supplied from the image processor 104 to the linear motor 108 to bring the image light beams captured by the imaging units 102L and 102R into focus.
A switch 111 is disposed at a predetermined location on the housing 120 of the electronic binoculars 100, and imaging and displaying-related processes are carried out by supplying operational instructions through the switch 111 to the image processor 104 and other components. Mode setting made by operating the switch 111 may be used to set whether or not hand-shake correction, which will be described later, and a variety of other image processing operations are carried out. The current mode setting may be displayed on the liquid crystal displays 110L and 110R.
An exemplary configuration for performing each of the image processing operations will be described below in detail.
First, the configuration for performing autofocus adjustment will be described with reference to
As shown in
The image processor 104 extracts a target subject from the image data produced by the two imaging units 102L and 102R and calculates the distance to the target subject. The calculated position is used as a target focus position, and the target position data is supplied as positional instruction data to a subtracter 114, where the current position of the optical systems 101L and 101R detected by a linear encoder 113 is subtracted from the positional instruction, and the position to be provided to the linear motor 108 is calculated. The calculated positional data is supplied to a PID controller 115 to produce a drive signal for driving the linear motor 108, and the drive signal (drive current Im) is supplied to the linear motor 108 through an amplifier 116. The PID controller 115 is a control means that performs feedback control using three elements, the deviation from the target value, the integral value of the deviation, and the derivative value of the deviation.
The configuration for making hand-shake correction will next be described with reference to
Data detected by the gyroscopic sensor 106 and the acceleration sensor 107 are digitized by analog/digital converters 117 and 118, respectively, and the converted digital data are supplied to the image processor 104, where the supplied sensor data along with the swinging state of the housing 120, which is the body of the electronic binoculars 100, are used to determine the hand-shake state. The captured image data undergo image processing according to the determined hand-shake state in the image processor 104. In the image processing, image data representing a previous frame stored in the memory 105 may be used in some cases. An example of the image processing for hand-shake correction will be described later in detail.
An exemplary configuration of the sliding mechanism 109, which adjusts the distance between the right and left liquid crystal displays 110L, 110R, will be described with reference to
The sliding mechanism 109 has screws 201L and 201R integrally disposed in series in the transverse direction, as shown in
The screw 201L disposed on the left side fits into a slidable member 202L to which the liquid crystal display 110L is attached, and rotating the screw 201L changes the position of the slidable member 202L (that is, the position of the liquid crystal display 110L) along the screw 201L. Similarly, the screw 201R disposed on the right side fits into a slidable member 202R to which the liquid crystal display 110R is attached, and rotating the screw 201R changes the position of the slidable member 202R (that is, the position of the liquid crystal display 110R) along the screw 201R.
Since the directions in which the screws 201L and 201R are threaded are opposite to each other, the two liquid crystal displays 110L and 110R slide and separate from each other (or approach) in accordance with the direction in which the screws 201L and 201R are rotated. The user can therefore arbitrarily adjust the distance between the two liquid crystal displays 110L and 110R.
The principle on which the autofocus adjustment is carried out will now be described with reference to
Now, let L0 be the distance between the two lens systems 101L and 101R, as shown in
Specifically, the discrepancy Lg between the positions of the object A in the two displayed images corresponds to the angle θx, as shown in
The thus calculated distance Lx is used as the target value to carry out the focusing processes in the processing system that has been described with reference to
The autofocus adjustment will be described using a formula. The distance Lx can be calculated by using the following approximate equation:
Distance Lx=L0/tan(θx), Equation (1)
When the distance Lx is substantially greater than the inter-lens distance L0, the above equation is approximated as follows:
tan(θx)≈K0×Lg, Equation (2)
where K0 is a positive integer determined in accordance with the magnification of the lens.
Therefore, the distance Lx is given by the following equation:
Distance Lx=L0/(K0×Lg), Equation (3)
In the present embodiment, the distance to the object is calculated based on the equation (3), and the object is automatically identified, for example, by carrying out the processes described later with reference to the example shown in
Identification of an object and a background image and image processing based on the identification performed in the electronic binoculars 100 of the present embodiment will be described with reference to
First, the principle on which an object is distinguished from a background image will be described with reference to
In this case, the image (displayed image #1) captured by the left imaging unit 102L and displayed on the liquid crystal display 110L has the object A displayed in front of the rearward background B, as shown in
The image (displayed image #2) captured by the right imaging unit 102R and displayed on the liquid crystal display 110R has the object A displayed in front of the rearward background B, as shown in
After the two displayed images #1 and #2 are obtained, one of the two images is subtracted from the other in the image processor 104 to remove the rearward background B common to the two images and produce an image containing only the object A, as shown in
As described above, providing the mode in which the background image is removed and performing the relevant image processing in the image processor 104 allow the background image to be removed and only the object to be displayed. The user is therefore provided with a very easy-to-see display mode because only the object is displayed. Further, removing the background image and extracting the object allow the object image to be readily identified, whereby the object can be identified and the distance to the object can be quickly calculated.
The principle of automatic identification of an object neighborhood and an object will be next described with reference to
An image (displayed image #2) captured by the right imaging unit 102R and displayed on the liquid crystal display 110R has the object A in front of the rearward background B, as shown in
After the two displayed images #1 and #2 are obtained, one of the two images is subtracted from the other in the image processor 104 to remove the rearward background B common to the two images and produce an image containing only the object A, as shown in
As an example of identifying an object in an image described above, for example, consider a case where a stadium is under observation. In general, a stadium has a flat ground having a uniform color, such as grass, in many cases. To extract an object under the condition, color analysis is performed on representative points (color information at 10 to 100 regularly arranged points) in the image data shown in
The positional discrepancy between the thus extracted object in the two images (Lg in
In practice, the image data only in the object neighborhood are processed to make the processing faster, whereby the computation time is reduced and realtime focusing is achieved.
Viewing region setting will next be described with reference to the images shown in
The object positioned slightly to the right of the center of the image imaged by the left imaging unit 102L as shown in
The image signals produced by the two imaging units 102L and 102R are combined as appropriate in the image processor 104 in such a way that the viewing region is located at the center.
The focusing adjustment is then carried out in such a way that the object in the viewing region is brought into focus. The processes that have been described with reference to
The processes of extracting only the image within the viewing region set on a screen and displaying an image with the background removed will be described with reference to
Similarly,
Subtracting one of the right-side object image shown in
An example of the hand-shake correction made in the electronic binoculars 100 of the present embodiment will be described with reference to
First,
It is assumed that the X and Z axes obtained in the imaging process using the image sensors in the imaging units 102L and 102R in the electronic binoculars 100 are shifted by the amount of shake θr, as shown in
In the hand-shake correction, a captured original images Vorg (xo,yo) is used to form a hand-shake corrected image Vcom (x, y), and the corrected image is displayed. The correction is made, for example, in the image processor 104 on an area set by cutting out part of the image in each frame formed by a captured image signal.
The process of converting the original image into an image with the amount of shake (θr) corrected to eliminate the effect of hand-shake will be described below by using a formula. The hand-shake angle θr is calculated by the following equation:
θr=θr0+∫(ωr)dt
(In the equation, ωr represents the sensor signal, and θr0 represents the initial horizontal angle obtained from the acceleration sensor.)
The thus measured hand-shake angle θr is used to convert the coordinates of the original image data Vorg (xo,yo) into the coordinates of the hand-shake angle corrected image Vcom (x, y) by using the following equation:
L0=√(x0̂2+y0̂2)
Θ0=arctan(x0/y0)
Vcmp(x,y)=Vorg(L0*cos(θ0+θr), L0*sin(θ0+θr))
In this way, image data with the hand-shake-related change corrected are obtained.
The shakes related to the shifts in the Z and Y axis directions can also be corrected on the same principle.
It is assumed in the present example that the hand-shake corrected image obtained as described above is an image in which the X and Z axes in the horizontal and vertical directions always coincide with the original X and Z axes. While the correction in the present example is described with reference to hand-shake correction, a state in which the electronic binoculars 100 that the user is holding is simply temporarily inclined may be similarly corrected by keeping the X and Z axes fixed.
An arrangement of the sensors necessary to make the hand-shake correction described above and the definition of the coordinates detected by the sensors will be described with reference to
As shown in
That is, the gyroscopic sensor signal (ωp, ωr, ωy) from the three-dimensional gyroscopic sensor 106 incorporated in the electronic binoculars 100 and the three-dimensional acceleration sensor signal (Ax, Ay, Az) from the acceleration sensor 107 also incorporated therein can be used to detect the rotation angles (θp, θr, θy) and the angular velocities (ωp, ωr, ωy) of the binoculars.
The acceleration sensor 107 is configured in such a way that when the Z axis is stationary and coincides with the direction in which the gravity acts, the output from the acceleration sensor (Ax, Ay, Az) is 0 [V] (zero volts) and the polarities of the sensor outputs are reversed in accordance with the direction.
When the X and Y axes are stationary in the horizontal plane in
θp=θp0+∫(ωp)dt, where θp0 represents the initial pitch angle
θr=θr0+∫(ωr)dt, where θr0 represents the initial roll angle
θy=θy0+∫(ωy)dt, where θy0 represents the initial yaw angle
The initial values θp0, θr0, and θy0 are determined by using the following equations along with the acceleration sensor signal in the stationary state.
θp0=arcsin(Ay)
θr0=arcsin(Ax)
θy0=0
The screen can be always displayed in the horizontal position by using the above equations to perform coordinate conversion in such a way that the image is always displayed in the horizontal position, as shown in
The correction described above is applicable to a case where each of the sensor signals has a frequency characteristic high enough to respond at a sufficiently high speed to the motion due to an actual hand-shake of the binoculars.
When the hand-shake motion is fast or the binoculars vibrate in the horizontal (vertical) direction, and the gyroscopic sensor 106 and the acceleration sensor 107 may not detect a shift, that is, the sensor output signals are very small, the image signals are used to detect a horizontal (vertical) shift, and images having undergone image conversion for shift correction are outputted in the form of video images. Video images of an object in question that are not affected by the vibration and shift of the binoculars are thus outputted. When a certain type of shift of the binoculars may not be detected by the sensors (the examples shown in
As seen from
In the case shown in
Making the hand-shake correction described in the present example allows the displayed image shown in
While the above figures show only the correction of the horizontal motion, the motions in the other directions are similarly corrected.
A description will be made of an example showing how the processes described above are carried out when the electronic binoculars 100 are actually used for observation with reference to the timing charts in
In the example shown in
As shown in
It is assumed in the example shown in
In this case, the images captured in a frame period K−1, which is one frame before the frame period K, and stored in the memory are read again in the frame period K and displayed on the liquid crystal displays 110L and 110R.
Thereafter, when the acceleration or the angular acceleration becomes smaller than the threshold in the following frame period K+1, the control returns to the display process using the signals captured in the frame period K+1. When the state in which the acceleration or the angular acceleration is greater than the threshold continues, the images captured and stored when the acceleration or the angular acceleration was smaller than or equal to the threshold value in the past are kept being displayed. It is noted, however, that the control may return to the display process using the current captured images after the state in which the acceleration or the angular acceleration is greater than the threshold has continued for a certain period.
In the example shown in
That is, in this example, the timing at which the right imaging unit 102R performs imaging as shown in
As shown in
It is assumed in the example shown in
In this case, the images captured by the other-side imaging unit in a frame period K−1, which is one frame before the frame period K, and stored in the memory are read again in the frame period K and displayed on the two liquid crystal displays 110L and 110R.
Thereafter, when the acceleration or the angular acceleration becomes smaller than the threshold in the following frame period K+1, the control returns to the display process using the signals captured in the frame period K+1.
As described above, performing what is called double-speed shuttering can effectively prevent blurred images due to a hand-shake from being displayed by temporarily displaying previous images in accordance with the state at the time of display. Further, performing the double-speed shuttering as shown in the present example is equivalent to imaging twice a frame even in the timing setting in which each of the imaging units performs imaging once a frame, whereby double-speed processing is achieved without increasing the amount of signal processing and power consumption necessary for the double-speed processing.
As described above, the electronic binoculars according to the present embodiment enables a sophisticated display operation different from that in electronic binoculars that has been proposed in related art. That is, performing not only the hand-shake correction but also the autofocusing enables a stable, well-defined display operation. Since the hand-shake correction is made by keeping the horizontal lines fixed, as shown in
Further, performing the autofocusing using the distance between the two imaging units achieves appropriate focusing using an intrinsic configuration of the binoculars. In particular, since an object present in a substantially central portion in the field of view is identified and used in the focusing process, even a subject moving at high speed can be brought into focus in a satisfactory manner.
Moreover, since the present embodiment allows the background image to be removed and prevents it from being displayed, only an object can be displayed. Binoculars capable of comfortably viewing only a necessary object are thus provided.
The present application contains subject matter related to that disclosed in Japanese Priority Patent Application JP 2008-159726 filed in Japan Patent Office on Jun. 18, 2008, the entire contents of which is hereby incorporated by reference.
It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.
Number | Date | Country | Kind |
---|---|---|---|
2008-159726 | Jun 2008 | JP | national |