Patent Application
20060072914
This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2003-304710, filed Aug. 28, 2003, the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to an object recognition apparatus which is used to be mounted on a movable body such as an automobile, an airplane, a ship and a robot, or a non-movable body such as an outdoor monitoring system.
2. Description of the Related Art
As an apparatus mounted on an automobile or the like, for recognizing an object in front thereof, a stereo camera has been known. This stereo camera has a function of observing a target object from different visual directions, and of measuring a distance up to the target object. The technology of the stereo camera is already broadly known in accordance with, for example, Jpn. Pat. Appln. KOKAI Publication No. 5-114099, and the like.
It is difficult in general for the stereo camera to detect an object and detect a distance appropriately in a dark place. Further, when a target object is in a low contrast even in a bright place, it is difficult to accurately find a distance up to the target object.
On the other hand, as an on-board video camera, a video camera using headlights as fill lights is known in accordance with Jpn. Pat. Appln. KOKAI Publication Nos. 5-189694 and 5-338487, and the like.
Further, a technology as well in which an emission device for emitting infrared radiation as a fill light is added to an on-board video camera, and illumination is carried out while synchronizing an image pickup timing of the video camera and a light emission timing of an infrared fill light from the emission device is well known in accordance with Jpn. Pat. Appln. KOKAI Publication No. 2002-240629 and the like.
Moreover, in Jpn. Pat. Appln. KOKAI Publication No. 2003-134509, a video camera using a laser beam as a fill light is disclosed as an on-board video camera.
In a video camera using infrared radiation or a laser beam as a fill light, it is necessary to add a light source for emitting a fill light, which results in an increase in cost for the apparatus.
Recently, a technology in which two types of range devices such as a stereo camera and a laser radar are provided together, and object recognition carried out in accordance with a predetermined algorithm on the basis of ranged results from both range devices is known in accordance with Jpn. Pat. Appln. KOKAI Publication Nos. 2000-329852 and 2003-121547, and the like.
In the complex type object recognition apparatus in which two types of range devices are provided together, highly reliable ranging performance and object recognition according to a situation are possible by utilizing the advantages possessed by the stereo camera and the laser radar, respectively. Namely, according to the stereo camera, it is impossible to accurately detect a distance up to a target object which is a long way off. However, it is possible to recognize a shape of a target object which is at a comparatively short distance in detail, and it is also possible to sense a road surface and a white line on a road. In contrast thereto, in the laser radar, a shape of a target object cannot be detected in detail, but it is possible to accurately detect a distance up to a target object which is a long way off (for example, a distance greater than or equal to 100 mm).
In above-described Jpn. Pat. Appln. KOKAI Publication Nos. 2000-329852 and 2003-121547, there is disclosed a technology relating to means for optimally integrating both range devices in order to utilize the advantages of the two.
Accordingly, an object of the present invention is to provide an object recognition apparatus which can recognize an object with extremely high reliability by both a stereo camera and an active range finder, and which can reduce an installation space, and can be manufactured at a low cost.
Note that the aforementioned active range finder means a measuring device, for example, a laser radar, which irradiates an electromagnetic wave such as a laser beam onto a target object, and receives a reflected wave therefrom, and which carries out ranging on the basis of information included in the reflected wave.
A feature of the present invention is an object recognition apparatus comprising:
an active range finder having a floodlight unit which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit which recognizes the target object on the basis of an output signal from the active range finder and an output signal from the stereo camera; and
a fill light control unit which controls the floodlight unit to operate so as to irradiate a light floodlit by the floodlight unit of the active range finder as a fill light of the stereo camera.
Another feature of the present invention is an object recognition apparatus comprising:
an active range finder having a floodlight unit which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit which recognizes the target object on the basis of an output signal from the active range finder and an output signal from the stereo camera;
a fill light control unit which controls the floodlight unit to operate so as to irradiate a light floodlit by the floodlight unit of the active range finder as a fill light of the stereo camera; and
a cover glass disposed in front of the stereo camera, wherein,
with respect to the cover glass, a predetermined region including a region into which a light that is floodlit from the floodlight unit of the active range finder, and is reflected from the target object, is incident serves as an infrared radiation permeable region, and a region other than the predetermined region serves as an infrared radiation permeability-limited region.
Still another feature of the present invention is an object recognition apparatus comprising:
an active range finder having a floodlight unit which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera to determine a distance up to the target object on the basis of image information from the target object;
a fill light control unit which irradiates a light floodlit by the floodlight unit of the active range finder as a fill light of the stereo camera onto an irradiating range in a predetermined relationship with a visual field range of the stereo camera; and
an object recognition unit which recognizes the target object on the basis of an output signal from the active range finder and an output signal from the stereo camera.
Further another feature of the present invention is an object recognition apparatus comprising:
an active range finder having a floodlight unit which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit which recognizes the target object on the basis of an output signal from the active range finder and an output signal from the stereo camera; and
a fill light control unit which controls the floodlight unit to operate so as to irradiate a light floodlit by the floodlight unit of the active range finder as a fill light of the stereo camera, wherein
the fill light control unit carries out control of the floodlight unit as a fill light irradiation unit on the basis of image information obtained from the stereo camera.
Still further another feature of the present invention is an object recognition apparatus comprising:
an active range finder having a floodlight unit which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit which recognizes the target object on the basis of an output signal from the active range finder and an output signal from the stereo camera;
a fill light control unit which controls the floodlight unit to operate so as to irradiate a light floodlit by the floodlight unit of the active range finder as a fill light of the stereo camera; and
a reliability determining unit to determine reliability in object recognition performance on the basis of an output signal from the active range finder and an output signal from the stereo camera.
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
As shown in
A floodlight window 32 in a flood light unit 30 of the active range finder 24 is disposed at an upper portion at the center of the front face of the case 22. Further, a sensor window 36 in a light receiving unit 34 of the active range finder 24 is disposed at a lower portion at the center of the front face of the case 22. Moreover, a left visual field opening 40a at a left optical input unit 40 of the stereo camera 26 is disposed at the left side portion at the front face of the case 22. In the same way, a right visual field opening 42a at a right optical input unit 42 of the stereo camera 26 is disposed at the right side portion at the front face of the case 22. An image pickup unit 46 of the stereo camera 26 is housed in a rear case of the case 22.
As shown in
Namely, if those are disposed as in
In contrast thereto, when the active range finder 24 and the stereo camera 26 are attached so as to have a space at separate places, the coincidence between a light emitting point of a fill light and a visual point of the stereo camera 26 is low. Therefore, when a target object 10 is a three-dimensional object, portions on which a fill light misses are generated, and a shadow is brought about on an image of the stereo camera 26. In other words, it is impossible to carry out efficient fill light irradiation.
Further, when object recognition is carried out in accordance with predetermined algorithm on the basis of a ranged result of the stereo camera 26 and a ranged result of the active range finder 24, a parallax between the detection systems of the both is preferably little. In the dispositions of
By the way, a windshield 16 of the vehicle 12 to which the object recognition apparatus 20 is attached constitutes a cover glass according to the invention. The cover glass 16 is disposed at the front face of the active range finder 24 and the stereo camera 26. Then, the cover glass 16 has a predetermined region E serving as an infrared radiation permeable region, the predetermined region E including a region into which a light floodlit from the floodlight unit 30 of the active range finder 24 and reflected from the target object 10 is incident.
The other region than the predetermined region E is an infrared radiation permeability-limited region. The predetermined region E is a permeable region on a part or all of a frequency domain of an infrared radiation floodlit from the floodlight unit 30.
Next, the detailed configurations of the active range finder 24 and the stereo camera 26 will be described with reference to
As shown in
The floodlight unit 30 is mounted on a position which is at the substantially center of the two optical input units 40 and 42 of the stereo camera 26. The floodlight unit 30 has a laser light source 50 and a scanning unit 52 which mechanically scans a light flux from the light source 50. The scanning unit 52 has the polygon mirror 54 in which a plurality of mirrors are arranged in a multi-angular pyramid shape, and a scanning motor 56 which carries out the scanning of a light flux by driving the polygon mirror 54 to rotate.
The polygon mirror 54 has six surfaces of U1, M1, L1, U2, M2, and L2 serving as surfaces which reflect a light from the laser light source 50 as shown in
The floodlight unit 30 has means for adjusting the irradiating range W, on which the light flux is irradiated as a fill light of the stereo camera 26, to the right and left by adjusting a relationship between a light emission timing of the laser light source 50 and an operation timing of the scanning unit 52. This point will be described in detail later.
Further, the floodlight unit 30 has an adjustment mechanism (not shown) for adjusting an irradiation angle of a light flux to be floodlit up and down by adjustably setting a supporting angle of the floodlight unit 30 upward and downward.
On the other hand, the light receiving unit 34 has a light receiving element 60 and a measuring circuit unit 62. SC denotes a control signal to be inputted to the light receiving unit 34, and SD denotes a distance output signal to be outputted from the light receiving unit 34.
The active range finder 24 receives at the light receiving unit 34 a light reflected from the target object 10, for example, of an infrared radiation emitted from the floodlight unit 30, and determines a distance up to the target object 10 on the basis of a time difference (which may be information such as a phase difference or others) from the light emission to the light reception.
The stereo camera 26 is configured to determine a distance up to the target object 10 on the basis of image information from the target object 10. The stereo camera 26 has a stereo optical system 66 and an image pickup unit 46. The stereo optical system 66 has the two optical input units 40 and 42 for acquiring image information from the target object 10 from different visual directions. The two optical input units 40 and 42 have a pair of front curvature units 40b and 42b, a pair of primary deflecting mirrors 40c and 42c, and a pair of secondary deflecting mirrors 40d and 42d.
The pair of front curvature units 40b and 42b are disposed so as to receive a light from the same target object 10 at left and right positions with a space at a predetermined interval. The pair of secondary deflecting mirrors 40d and 42d are to further reflect respective lights reflected on the primary deflecting mirrors 40c and 42c so as to be introduced into the image pickup unit 46.
Light fluxes OP1 and OP2 from the target object 10 which have been incident via the front curvature units 40b and 42b are reflected on the primary deflecting mirrors 40c and 42c. These primary deflecting mirrors 40c and 42c have sizes to such an extent that it is possible to reflect light fluxes permeated through the front curvature units 40b and 42b. Namely, the primary deflecting mirrors 40c and 42c have sizes to an extent which is substantially the same as or slightly greater than unscreened regions of the front curvature units 40b and 42b.
Further, the primary deflecting mirrors 40c and 42c are disposed so as to be inclined at about 45 degrees in the horizontal direction. Then, the primary deflecting mirror 40c is disposed in a state of being inclined downward by several degrees to the secondary deflecting mirror 40d side in the vertical direction. On the other hand, the primary deflecting mirror 42c is disposed in a state of being inclined upward by several degrees to the secondary deflecting mirror 42d side in the vertical direction. Due to the primary deflecting mirrors 40c and 42c being disposed in this way, the respective light fluxes reflected on the primary deflecting mirrors 40c and 42c are respectively incident onto the secondary deflecting mirrors 40d and 42d.
The secondary deflecting mirrors 40d and 42d are disposed so as to face the primary deflecting mirrors 40c and 42c in substantially parallel in the horizontal direction, and to cross each other at the sight from the top surface direction. Then, the secondary deflecting mirror 40d is disposed in a state of being inclined downward by several degrees to the image pickup device side in the vertical direction. In addition, the secondary deflecting mirror 42d is disposed in a state of being inclined upward by several degrees to the image pickup device side in the vertical direction. Due to the secondary deflecting mirrors 40d and 42d being disposed in this way, the respective light fluxes reflected on the secondary deflecting mirrors 40d and 42d are incident onto the image pickup unit 46.
Namely, the light flux OP1 incident from the primary deflecting mirror 40c is deflected so as to be reflected on the secondary deflecting mirror 40d to be incident into the image pickup unit 46. On the other hand, the light flux OP2 incident from the primary deflecting mirror 42c is deflected so as to be reflected on the secondary deflecting mirror 42d to be incident into the image pickup unit 46.
The image pickup unit 46 is configured to have an image pickup optical system 74 and an image pickup device 76, and obtains a parallax image 18 on the basis of a light flux incident from the stereo optical system 66.
Namely, the light flux incident via the optical system of the left side optical input unit 40 is imaged on a lower half region of the image pickup device 76 through the image pickup optical system 74. A light flux incident via the optical system of the right side optical input unit 42 is imaged on an upper half region of the image pickup device 76 through the image pickup optical system 74. Namely, the visual fields are limited by visual field masks installed on the front curvature units 40b and 42b, so that the images are imaged on the light receiving surface of the image pickup device 76 without being overlapped onto each other.
However, here, the right image is imaged on the upper half region of the image pickup device 76, and the left image is imaged on the lower half region. However, the images are imaged upside down due to the effect of the imaging lens. Then, when the relationship of the heads and tails is inverted and restored to the correct direction, a left image 18L and a right image 18R which have a parallax between the left and right are imaged so as to correspond to the upper half region and the lower half region on the light receiving surface of the image pickup device 76, as shown in
As shown in
The floodlight unit 30 of the active range finder 24 is configured to include the polygon mirror 54, a polygon mirror driver (PM driver) 56, a pulse control unit 80, a pulse generating unit 82, an power control unit 84, and the laser light source (LD) 50.
The polygon mirror driver 54 is drive means for driving the polygon mirror 54. The pulse control unit 80 is to carry out pulse emission control of the laser light source 50 while synchronizing with respect to a drive pulse from the polygon mirror driver 56. The pulse generating unit 82 is to generate a light emission pulse by receiving a signal from the pulse control unit 80. The power control unit 84 is to adjust a light emission intensity with respect to an output pulse from the pulse generating unit 82. Then, the laser light source 50 is operated to emit light by a signal outputted from the power control unit 84.
The light receiving unit 34 of the active range finder 24 has, in addition to the light receiving element 60 described above, an amplifier 90, a distance calculating unit 92, a photometric device 94, a gain controller 96, and the like.
The active range finder 24 can detect a distance of a vehicle or the like which is particularly at long range by receiving a reflected light from a so-called reflector (natural lighting reflector) of a foregoing vehicle. On the other hand, the stereo camera 26 cannot detect a distance up to an object which is far away therefrom, although depending on a size of a parallax between the left and right. Accordingly, the stereo camera 26 is effective in operation support such as avoiding other vehicles parking at a comparatively long distance, for example, during driving at high speed, by using an output from the active range finder 24.
In contrast thereto, there is a feature as will be described later in the stereo camera 26. Thus, in the present embodiment, an attempt is made to finally make good use of outputs from the both detection systems for operation support by carrying out object detection by utilizing the advantages thereof in accordance with predetermined algorithm.
The stereo camera 26 has the stereo optical system 66, the image pickup unit 46, an aperture adjustment unit 100, a shutter adjustment unit 102, a sensitivity adjustment unit 104, an exposure adjustment unit 106, a distance calculating unit 108, and the like.
The image pickup unit 46 has an image pickup device 76 as described above, and the image pickup device 76 is to convert a stereo image captured by the stereo optical system 66 into an electrical signal. The aperture adjustment unit 100 is to carry out an adjustment of an aperture mechanism (not shown) inside the stereo optical unit 66. Further, the shutter adjustment unit 102 is to adjust a charge storage period of an electronic shutter inside the image pickup device 46.
The sensitivity adjustment unit 104 is to adjust a gain of an analog image signal amplifier inside the image pickup device 76. Moreover, the exposure adjustment unit 106 is to adjust the aperture adjustment unit 100, the shutter adjustment unit 102, and the sensitivity adjustment unit 104, and to carry out a so-called exposure adjustment.
The image pickup device 46 inside the stereo camera 26 has a spectral sensitivity characteristic of a visible light. The image pickup device 46 further has an available sensitivity region at which a part or all of a frequency domain of the infrared radiation floodlit from the floodlight unit 30 can be captured in order to utilize an infrared radiation floodlit from the floodlight unit 30 as a fill light of the stereo camera 26.
The sensitivity adjustment unit 104 of the stereo camera 26 is usually set so as to obtain a standard sensitivity. However, the sensitivity adjustment unit 104 can be controlled so as to be variable in accordance with image luminance information accompanying the fill light obtained by the stereo camera 26. Moreover, the stereo camera 26 periodically acquires stereo images by periodically operating the image pickup device 76 in order to acquire image information from the target object 10, carries out a distance operation with respect to the acquired stereo image at the distance calculating unit 108, and periodically determines distances up to the target object 10.
The processing unit 28 has an object recognition unit 110, a fill light control unit 116 having a timing adjustment unit 112 and an exposure adjustment unit 114, a reliability determination unit 118, an integrated control unit 120, an information input terminal 122 for inputting operation conditions, and the like.
The object recognition unit 110 in the processing unit 28 recognizes the target object 10 on the basis of a distance output signal SD from the active range finder 24, texture image data ST outputted from the image pickup unit 46 (image pickup device 76) inside the stereo camera 26, three-dimensional image data SV outputted from the distance calculating unit 108, and information SI from an operation condition input unit 124. The operation condition input unit 124 detects an operation condition of a vehicle by a driver.
Here, the texture data ST is data of a plane image, without distance information on the object, but including a luminance and color information. The three-dimensional image data SV is data showing a distribution of a distance data string corresponding to pixels of the texture image data ST without including a luminance and color information of the object.
It is possible to detect a white line on a road, and to detect a guardrail and other obstacles, on the basis of the texture image data ST and the three-dimensional distance image data SV from the stereo camera 26. Therefore, it is possible to drive while tracing a white line, and to support an operation for driving so as to avoid obstacles. This function is a function which cannot be obtained by the active range finder 24, and is a feature of the stereo camera 26.
The timing adjustment unit 112 controls a light emission timing of a pulse light emission from the laser light source 50 by receiving a control signal from the integrated control unit 120 and outputting the control signal to the pulse control unit 80 in the floodlight unit 30. Further, the timing adjustment unit 112 carries out the control of the light receiving element 60 of the light receiving unit 34, the distance calculating unit 92, and the like by outputting the control signal to the light receiving unit 34. Moreover, the timing adjustment unit 112 carries out the control of the image pickup device 46 to drive by outputting the control signal to the image pickup device 46 in the stereo camera 26.
The exposure adjustment unit 114 controls a light emission intensity of the laser light source 50 by outputting the control signal to the power control unit 84 inside the floodlight unit 30 in accordance with algorithm which will be described later on the basis of information on an object distance outputted from the object recognition unit 110 and texture image data ST from the stereo camera 26 obtained via the object recognition unit 110. The exposure adjustment unit 114 adjusts light sensitivity by outputting the control signal to the gain controller 96 inside the light receiving unit 34. Moreover, the exposure adjustment unit 114 carries out adjustments of the aperture, the electronic shutter, and the light sensitivity inside the stereo camera 26 by outputting the control signal to the exposure adjustment unit 106 inside the stereo camera 26.
The reliability determination unit 118 carries out a comparative determination on the both ranged results by comparing a ranged result from the active range finder 24 and a ranged result from the stereo camera 26. As a result of the determination, when there is a great difference between the ranged results from the both, it is determined that a failure has been brought about in the apparatus, and the reliability determination unit 118 outputs a predetermined warning signal to the exterior.
The integrated control unit 120 receives output signals from the object recognition unit 110, the timing adjustment unit 112, the exposure adjustment unit 114, and the reliability determination unit 118, and carries out entire integrated control by outputting control signals to these object recognition unit 110, timing adjustment unit 112, exposure adjustment unit 114, and reliability determination unit 118.
The fill light control unit 116 including the timing adjustment unit 112 and the exposure adjustment unit 114 has various functions as follows.
The floodlight unit 30 is controlled to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 24. Further, the fill light control unit 116 irradiates a light floodlit by the floodlight unit 30 of the active range finder 24, as a fill light of the stereo camera 26, onto a irradiating range W in a predetermined relationship with a visual field range V of the stereo camera 26.
The fill light control unit 116 causes the floodlight unit 30 of the active range finder 24 to operate at predetermined timings as fill light irradiating means of the stereo camera 26 with respect to imaging operations of the stereo camera 26. The fill light control unit 116 further includes means for irradiating a predetermined irradiating range in one imaging operation.
Further, the fill light control unit 116 includes means for irradiating predetermined different irradiating ranges in a plurality of imaging operations. In addition thereto, the fill light control unit 116 includes means for causing the floodlight unit 30 of the active range finder 24 to selectively operate only in specific imaging operations in a plurality of imaging operations.
Moreover, the fill light control unit 116 controls the floodlight unit 30 on the basis of image information obtained from the stereo camera 26. Further, the fill light control unit 116 carries out the control for on/off of the floodlight unit 30 in accordance with the image information obtained from the stereo camera 26. Moreover, the fill light control unit 116 carries out the control for light intensity of the floodlight unit 30 in accordance with the image information obtained from the stereo camera 26.
The fill light control unit 116 carries out the conditioned control of light intensity of the floodlight unit 30 such that a luminance at an objective region is fallen into a predetermined luminance range in accordance with the image information obtained from the stereo camera 26. The fill light control unit 116 further includes means for turning the floodlight unit 30 off, or reducing the light intensity when it is determined that there is a possibility that a person exists in an objective region in accordance with the image information obtained from the stereo camera 26.
In addition, the fill light control unit 116 includes means for controlling a size of a floodlight spot from the floodlight unit 30 in accordance with the image information obtained from the stereo camera 26. Moreover, the fill light control unit 116 includes means for controlling the irradiating range W by the floodlight unit 30 in accordance with the image information obtained from the stereo camera 26.
The fill light control unit 116 includes means for gradually enhancing the floodlight unit 30 from being in an off-state or in a sate in which the light intensity is comparatively weak to being in a state in which the light intensity is comparatively strong when the means starts to operate.
The fill light control unit 116 includes means for controlling the floodlight unit 30 in accordance with the luminance information obtained from the stereo camera 26. Additionally, the fill light control unit 116 includes means for controlling the floodlight unit 30, and for adjusting light sensitivity of the stereo camera 26 in accordance with the image information obtained from the stereo camera 26.
Moreover, the fill light control unit 116 includes means for determining an irradiating range W of the floodlight unit 30 on the basis of an operation condition (for example, a rudder angle of a steering wheel) of the vehicle 12 inputted by the operation condition input unit 124.
Note that the present apparatus 20 has the following functions in addition to those described above.
Namely, the apparatus 20 has an adjustment unit for a fill light irradiating range in which a range W on which a light floodlit by the floodlight unit 30 of the active range finder 24 is irradiated as a fill light of the stereo camera 26 is adjusted with respect to a visual field range V of the stereo camera 26 or a reference position of the visual field range V.
The apparatus 20 further has a calibration execution unit which carries out calibration of a ranged result from the stereo camera 26 on the basis of a ranged result from the active range finder 24. In the optical system of the stereo camera 26, an accurate optical adjustment has been carried out. However, when it has been used for a long period, an error might be brought about in a ranged result with time due to vibration or the like.
A ranging method of the active range finder 24 is to range by measuring a period from a time light emission from the laser light source 50 is carried out up to a time a light reflected on the target object 10 returns thereto. Accordingly, a change over time is hardly brought about as compared with the stereo camera 26, and the measured values are stable. Accordingly, the calibration is preferably carried out periodically or every time the power source of the object recognition apparatus is turned on.
Further, the present apparatus 20 has a floodlight control unit constituted by the gain controller 96 and the integrated control unit 120. The floodlight control unit causes the floodlight unit 30 to operate in order to irradiate a fill light of the stereo camera 26 during an operating cycle of the image pickup device 76 in the image pickup unit 46, and within a period of time charge storage is carried out. In addition, except that the period of time charge storage is carried out, the floodlight control unit causes the floodlight unit 110 to operate in order to obtain ranged data as the active range finder 24.
Hereinafter, the operation and the effect of the first embodiment described above will be described.
The pulsed laser irradiation B shows a reference light emission timing of the laser light source 50. The laser light source 50 carries out pulse light emissions at predetermined timings so as to be synchronized with the rotation cycles of the respective inclined reflection surfaces of the polygon mirror 54. As a result, as shown in
Here, it is sufficient that the floodlight unit 30, in particular, the laser light source 50 is accurately attached to the detected visual field of the stereo camera 26. However, in other cases, as shown in
In a case where the scanning ranges are shifted to the left side as shown in
Stereo imaging in step S1 and ranging at the active range finder 24 in step S2 are, as one set, carried out at a frequency of 30 times/second by the integrated control unit 120. Namely, with one cycle being about 33 msec, the processing in step S1 is executed at the front half thereof, and the processing in step S2 is executed at the last half.
Further, sequences of object recognition in step S3 and of operation support information output in step S4 are carried out in parallel with the sequences in the steps S1 and S2. These steps S3 and S4 are carried out by using the results in the steps S1 and S2 determined due to a loop of the past such as the last time, or the last but one.
The laser light source (LD) 50 is caused to operate to emit light in timings as shown in
The sequence of stereo imaging in the step S1 will be further described in detail by using a subroutine of
First, in step S11, it is determined whether a high luminance or not. Here, under the conditions that a shutter opening period (charge storage period) is 15 msec which is a maximum length, that an aperture is a minimum aperture, and that sensitivity setting of the image pickup device 76 is minimum sensitivity, it is determined high luminance when the luminance is a luminance by which the outputs from the image pickup device 76 are saturated. Accordingly, the routine proceeds to step S12, and high luminance control is carried out.
In the high luminance control in step S12, a shutter opening period is controlled so as to be variable in accordance with a luminance between 15 msec which is a maximum second time to 1 μsec which is a minimum second time, as shown in the X item of
Here, the shutter opening period is controlled to be adjusted by giving feedback to each stereo imaging such that a MAX value of image data obtained from the image pickup device 76 is fallen in a range of 70 to 100% of a saturation level.
When it is determined not high luminance in the step S11, the routine proceeds to step S13, in which it is determined whether a low luminance or not. Here, under the conditions that a shutter opening period (charge storage period) is 15 msec which is a maximum length, that an aperture is a maximum aperture, and that sensitivity setting of the image pickup device 76 is minimum sensitivity, it is determined a low luminance when the output from the image pickup device 76 is 70% or less of a saturation level, and low luminance control in step S15 is carried out. In other cases, it is determined a medium luminance, and the routine proceeds to step S14, in which medium luminance control is carried out.
In the medium luminance control in step S14, a shutter opening period is fixed to 15 msec which is a maximum second time, as shown in the Y item of
The emission intensity of the laser light source 50 is controlled to be adjusted in accordance with a distance up to the target object 10 and a controlled value of the aperture so as to be a fill light appropriate to the target object 10. Here, the aperture is controlled to be adjusted by giving feedback from each stereo imaging to following stereo imaging such that a MAX value of image data obtained from the image pickup device 76 is fallen in a range of 70 to 100% of a saturation level.
In the low luminance control in step S15, a shutter opening period is fixed to 15 msec which is a maximum second time, as shown in the Z item of
Here, the sensitivity setting of the image pickup device 76 is controlled to be adjusted by giving feedback from each stereo imaging result to following stereo imaging such that a MAX value of image data obtained from the image pickup device 76 is fallen in a range of 70 to 100% of a saturation level.
Note that, as the conditioned control, exposure control, which is in the same way as in aperture priority, shutter speed priority, aperture or shutter speed priority, and the like at the time of general video camera photographing, may be carried out.
As described above, the conditioned control of a shutter opening period, the conditioned control of an aperture, the conditioned control for sensitivity setting of the image pickup device 76, the conditioned control of an emission intensity of the laser light source 50, and the like are carried out appropriately in accordance with an output from the image pickup device 76 in the stereo camera 26.
Next, a second embodiment of the present invention will be described.
The configurations of the object recognition apparatus in the second embodiment and embodiments described hereinafter are the same as those in the first embodiment. Therefore, only different configurations and operations will be described, the other portions which are the same as those are denoted by same reference numerals, and illustrations and descriptions thereof will be omitted.
The second embodiment is different from the first embodiment in that only the floodlight unit 30 of the active range finder 24 is installed in the vehicle so as to be integrated with the stereo camera 26, and that the light receiving unit 34 of the active range finder 24 is installed in the vicinity of the front bumper of the vehicle. In such an embodiment, an object of the present invention can be achieved.
Next, a third embodiment of the present invention will be described.
The third embodiment is an embodiment corresponding to a case where the working speed of the polygon mirror 54 is not high as that in the first embodiment described above, and hexahedral driving of the polygon mirror 54 cannot be achieved, but only tetrahedral driving thereof can be achieved during one cycle. In this case, it is impossible to set a shutter second time to about 15 msec as in the first embodiment. This is an example in which, under the photographing condition that a shutter speed is high, and a charge storage period is short less than or equal to ¼ of one cycle, a photographing operation is carried out so as to be divided into a plurality of times (three times in the present embodiment) in accordance with the light emission scanning speed of the polygon mirror 54, and one required image is obtained by synthesizing photographed images thereby.
In
Next, a fourth embodiment of the present invention will be described.
The fourth embodiment is different from the first embodiment in that a fill light is turned on at a rate of once per three times of stereo imaging.
Namely, as shown in
The fifth embodiment is different from the first embodiment in that, when it is determined that a target object in the stereo image data is a part in the visual field range of the stereo camera 26, a fill light is irradiated onto only the limited part.
As shown in
The sixth embodiment is different from the first embodiment in that a fill light is irradiated onto a limited range in accordance with information SI inputted via an input terminal 350 from operation condition input means (not shown). The operation condition input unit 124 inputs a rudder angle of the steering wheel of the automobile, and for example, when the steering wheel H is turned to the right direction as shown in
Next, a seventh embodiment of the present invention will be described.
The seventh embodiment is different from the first embodiment in that it is possible to insert a cylindrical lens (not shown) which can be inserted and demounted between the laser light source 50 and the polygon mirror 54. In the ordinary case, the cylindrical lens is not inserted in front of the laser light source 50, and a circular laser beam flux is emitted. As a result, as shown in
For example, when the target object 10 is at a long distance, and a strong fill light is required, a cylindrical lens is inserted in front of the laser light source 50, a fill light is emitted such that a laser beam flux is deformed so as to be elongated in the vertical direction, and a cycle of emission pulses from the laser light source 50 is shortened. As a result, as shown in
Although the cylindrical lens is inserted between the laser light source 50 and the polygon mirror 54 in the seventh embodiment, the cylindrical lens may be inserted at the light emission side of the polygon mirror 54. In this case, the same effect can be obtained only by inserting and demounting the cylindrical lens, even without a cycle of light emission pulses of the laser light source 50 being changed.
The eighth embodiment is different from the first embodiment in that an effect of increasing contrast on the target object is improved by further increasing unevenness of irradiation of a fill light. Namely, as shown in
In this manner, unevenness of irradiation brought about on the surface of the target object 10 is made more apparent, and an effect of increasing contrast on the target object 10 is improved.
The ninth embodiment is different from the first embodiment in that unevenness of irradiation is brought about on the target object 10 by modulating an intensity of pulse light emission emitted from the laser light source 50.
Namely, it is the same as
According to the present embodiment, it is the same as in the eighth embodiment described above in the point that the effect of increasing contrast on the target object 10 is improved. In addition thereto, because unevenness of irradiation brought about on the target object 10 is not made periodical, the risk of error in ranging is little.
By comparison, the ranging method of the stereo camera 26 is to utilize a parallax between the left and right. Thus, when unevenness of irradiation is periodical, an accurate detection is not carried out, which results in the risk of error in ranging.
Next, a tenth embodiment of the present invention will be described.
The tenth embodiment is different from the first embodiment in that, as shown in
(Characteristics in the Embodiments)
[1] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26.
In the object recognition apparatus, the floodlight unit 30 of the active range finder 24 is used as a fill light of the stereo camera 26 in the apparatus having two type of ranging units which are the active range finder 24 and the stereo camera 26. Therefore, even under the condition that the environment is dark, an appropriate stereo image can be obtained without a dedicated fill light irradiation unit being provided in particular. Additionally, the apparatus can be formed so as to be compact, and can be manufactured at a low cost, and moreover, there is an advantage from the standpoint of a installation space. Further, by providing a distribution in a floodlit pattern of a fill light, a function of projecting a pattern can be provided. Consequently, with respect to a target object having low contrast, it is possible to project required contrast under the condition that the environment is bright, and it is possible to improve a ranging accuracy.
[2] An on-board object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object; and
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26, wherein
the active range finder 24 and the stereo camera 26 are provided in a vehicle interior 14.
In the object recognition apparatus, it is possible to provide at least the stereo camera 26 at an upper position in the vehicle interior 14. Thus, the visual field is made sufficiently broad in a case of carrying out detecting a white line on a road or the like. Further, the active range finder 24 and the stereo camera 26 are provided in the vehicle interior 14, so that the object recognition apparatus can be protected from rain, wind, and dust. Moreover, when a position to which the object recognition apparatus is attached is provided at the inner side of the windshield 16 in the vehicle interior 14, the windshield 16 is made to be in a state in which rain drops are removed by a windshield wiper function even if it rains, and therefore, image data without adverse effect due to rain drops can be obtained.
[3] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
a cover glass 16 disposed in front of the stereo camera 26, wherein,
with respect to the cover glass 16, a predetermined region E including a region into which a light that is floodlit from the floodlight unit 30 of the active range finder 24, and is reflected from the target object, is incident serves as an infrared radiation permeable region, and a region other than the predetermined region serves as an infrared radiation permeability-limited region.
In the object recognition apparatus, a necessary infrared radiation reflected from the target object is incident after permeating through the predetermined region E. Therefore, a harmful effect due to an unnecessary infrared radiation making an invasion into the vehicle interior 14 can be avoided.
[4] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal from the active range finder 24 and an output signal from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
a cover glass 16 disposed in front of the stereo camera, wherein
the cover glass 16 has an available sensitivity region through which a part or all of a frequency domain of an infrared radiation floodlit from the floodlight unit 30 is permeable.
In the object recognition apparatus, a fill light can be effectively applied to the stereo camera 26 provided in the vehicle interior 14.
[5] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26, wherein
the floodlight unit 30 of the active range finder 24 and the stereo camera 26 are housed in a same housing 22 so as to be unitized.
In the object recognition apparatus, calibration such as position adjustment between the detected visual field of the stereo camera 26 (overlapped region of the visual fields of the left and right cameras) V and the fill light irradiating range (radar scanning region) W, and the like can be executed as a single unit at a stage before attaching it to a movable body such as the vehicle 12.
[6] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [5], wherein
the stereo camera 26 has two optical input units 46, 66 in order to acquire image information on the target object from different visual directions, and the floodlight unit 30 is mounted at a substantially central position between the two optical input units 46, 66 of the stereo camera 26.
In the object recognition apparatus, it is possible to efficiently irradiate a fill light P onto the detected visual field of the stereo camera 26 (overlapped region of the visual fields of the left and right cameras) V. Further, the integrated unit 20 of the object recognition apparatus can be formed so as to be compact, so that an installation space for the unit 20 can be suppressed to the minimum necessary. Note that, when the target object is a three-dimensional object, a part onto which a fill light is not irradiated is brought about in a shadow part of the target object. However, as described above, the floodlight unit 30 is disposed at substantially the central position between the two optical input units 46 and 66, and consequently, the region onto which a fill light is not irradiated can be made as small as possible.
[7] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
a fill light control unit 116 which irradiates a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26 onto an irradiating range in a predetermined relationship with a visual field range V of the stereo camera 26; and
an object recognition unit 110 which recognizes the target object on the basis of an output signal from the active range finder 24 and an output signal from the stereo camera 26.
In the object recognition apparatus, because the detected visual field range V of the stereo camera 26 and the fill light irradiating range W are adjusted in a predetermined relationship, it is possible to efficiently irradiate a fill light onto the target object of the stereo camera 26.
[8] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal from the active range finder 24 and an output signal from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
an adjustment unit which adjusts a range W onto which a light floodlit by the floodlight unit 30 of the active range finder 24 is irradiated as a fill light of the stereo camera 26, with respect to a visual field range V or a reference position of the visual field range V of the stereo camera 26.
[9] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [8], wherein
the floodlight unit 30 has a light source 50 and a scanning unit 52 which mechanically scans a light flux from the light source 50, and adjusts the range W to be illuminated as a fill light by adjusting a relationship between a light emission timing of the light source 50 and an operation timing of the scanning unit 52.
[10] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [8], wherein
the floodlight unit 30 has an adjustment unit which adjusts an irradiation angle of a light flux floodlit from the floodlight unit 30.
[11] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
a fill light control unit 116 which causes the floodlight unit 30 of the active range finder 24 to operate as a fill light irradiation unit of the stereo camera 26 at predetermined timings, with respect to imaging operations of the stereo camera 26; and
an object recognition unit 110 which recognizes the target object on the basis of an output signal from the active range finder 24 and an output signal from the stereo camera 26.
In the object recognition apparatus, the floodlight unit 30 operates as a fill light irradiation unit at predetermined timings with respect to photographing operations of the stereo camera 26. Therefore, a fill light is effectively irradiated during only a necessary period within the period of the imaging operations.
[12] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [11], wherein
the fill light control unit 116 irradiates a predetermined irradiating range in one imaging operation.
[13] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [11], wherein
the fill light control unit 116 irradiates predetermined different irradiating ranges in a plurality of imaging operations.
In the object recognition apparatus, in a case where an object luminance is comparatively bright, and when an exposure period of the image pickup unit (charge storage period) is extremely short (in such a case, it is impossible to irradiate a fill light onto the entire necessary irradiating range during one imaging operation [during an exposure period]), a fill light is irradiated onto the necessary irradiating range separately in a plurality of times of imaging operations.
[14] An object recognition apparatus described in embodiment is the object recognition apparatus according to the item [11], wherein
the fill light control unit 116 causes the floodlight unit 30 of the active range finder 24 to selectively operate in only specific imaging operations among a plurality of imaging operations.
In the object recognition apparatus, irradiations of a fill light are prevented from being carried out unnecessarily many times. Therefore, it is possible to prolong the lifetime of the floodlight unit 30.
[15] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object, the stereo camera being provided in a vehicle interior 14;
an object recognition unit 110 to recognize the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit to operate so as to irradiate a light floodlit by the floodlight unit 110 of the active range finder 24 as a fill light of the stereo camera 26, wherein
the stereo camera 26 has spectral sensitivity at which a part or all of a frequency domain of an infrared radiation floodlit from the floodlight unit 30 is captured as an available sensitivity region.
In the object recognition apparatus, an overlapped region is generated on a spectral sensitivity of the stereo camera 26 and a spectral sensitivity of the floodlight unit 30. Therefore, the floodlight unit 30 of the active range finder 24 can be effectively utilized as a fill light irradiation unit of the stereo camera 26.
[16] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26, wherein
the stereo camera 26 further has an image signal amplifier, and a gain of the image signal amplifier is controlled so as to be variable in accordance with image luminance information accompanying the fill light obtained from the stereo camera 26.
In the object recognition apparatus, when stereo image photographing using a fill light cannot be sufficiently carried out due to a situation of outside light (a situation of infrared disturbance light is made different in accordance with fine weather, night, or the like), and a relationship of a distance up to the target object, gain-up is automatically carried out. As a result, sufficient fill light photographing is always possible.
[17] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26, wherein
the fill light control unit 116 carries out control of the floodlight unit 30 as a fill light irradiation unit on the basis of image information obtained from the stereo camera 26.
In the object recognition apparatus, fill light irradiations necessary for the stereo camera 26 are carried out in an appropriate manner. Therefore, there is an advantage that an attempt can be made to effectively utilize energy.
[18] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 carries out ON/OFF control of the floodlight unit 30 as a fill light irradiation unit in accordance with image information obtained from the stereo camera 26.
In the object recognition apparatus, a fill light is controlled to be turned off when it is determined that a fill light is unnecessary on the basis of stereo image information. Therefore, an attempt at efficient use of energy or the like can be made.
[19] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 carries out light intensity control of the floodlight unit 30 as a fill light irradiation unit in accordance with image information obtained from the stereo camera 26.
In the object recognition apparatus, when it is determined that an intensity of a fill light is not required to be so strong, or when it is determined that there is some sort of negative effect because the intensity of the fill light is strong on the basis of stereo image information, the intensity of the fill light is automatically made weaker. On the other hand, when it is determined that a strong fill light is required on the basis of stereo image information, the intensity of the fill light is automatically made stronger. Accordingly, an appropriate intensity of the fill light according to a situation can be set.
[20] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [19], wherein
the fill light control unit 116 carries out conditioned control of a light intensity of the floodlight unit 30 such that a luminance of the target object falls into a predetermined luminance range, in accordance with image information obtained from the stereo camera 26.
[21] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 turns the floodlight unit 30 off, or makes a light intensity weak when it is determined that there is a possibility that a person exists in a region to be an object, in accordance with image information obtained from the stereo camera 26.
In the object recognition apparatus, the human body can be prevented from being damaged by being irradiated with a fill light.
[22] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 controls a size of a floodlight spot from the floodlight unit 30 as a fill light irradiation unit, in accordance with image information obtained from the stereo camera 26.
Usually, when an attempt is made to detect a target object which may be at a long distance by a laser radar, a ranging region which can be covered is made greater as a result of making a diameter of a floodlight spot large, and a reflector of a vehicle is detected by the laser radar. In the object recognition apparatus, the information on the stereo image can be used simultaneously, and thus, when a position of the reflector of the vehicle can be detected, it is possible to irradiate a laser with a small diameter whose beam intensity is strong onto the position. As a result, ranging performance by the laser radar is improved.
[23] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 controls an irradiating range W by the floodlight unit 30 as a fill light irradiation unit, in accordance with image information obtained from the stereo camera 26.
In the object recognition apparatus, a fill light can be irradiated onto only a necessary irradiating range. Therefore, an attempt at efficient use of energy can be made.
[24] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 gradually enhances the floodlight unit 30 as a fill light irradiation unit from being in an off-state or in a state in which a light intensity is comparatively weak to being in a state in which the light intensity is comparatively stronger, at the time of starting operation.
In the object recognition apparatus, it is possible to reduce a loss in energy which is brought about at the time of operating of a sudden with a strong light intensity.
[25] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [22], wherein
the fill light control unit 116 gradually enhances the floodlight unit 30 as a fill light irradiation unit from being in an off-state or in a state in which a light intensity is comparatively weak to being in a state in which the light intensity is comparatively stronger, at the time of starting operation.
In the object recognition apparatus, it is possible to reduce a loss in energy which is brought about at the time of operating of a sudden with a strong light intensity.
[26] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [17], wherein
the fill light control unit 116 controls the floodlight unit 30 as a fill light irradiation unit in accordance with luminance information obtained from the stereo camera 26.
In the object recognition apparatus, it is possible to irradiate a fill light such that the intensity thereof is made weak when the luminance is high, and that the intensity thereof is made strong when the luminance is low, in accordance with luminance information. Therefore, a necessary intensity of a fill light can be set in accordance with a luminance, and an attempt at efficient use of energy can be made.
[27] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26, wherein
the fill light control unit 116 carries out control of the floodlight unit 30 as a fill light irradiation unit in accordance with image information obtained from the stereo camera, and carries out an adjustment of light receiving sensitivity of the stereo camera 26.
In the object recognition apparatus, it is possible to accurately irradiate a fill light necessary for the stereo camera 26. Additionally, it is possible to process a signal in accordance with a size of the output signal S2 from the stereo camera 26. Accordingly, an attempt at efficient use of energy can be made, and moreover, it is possible to ensure a necessary dynamic range for the output signal S2 from the stereo camera 26.
[28] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
a calibration executing unit which carries out calibration of a distance output signal of the stereo camera 26 by a distance output signal S1 from the active range finder 24.
Generally, a stereo camera has a complicated and delicate optical mechanism, and there is a possibility that an alignment of an optical system is shifted over time due to factors such as vibration. In contrast thereto, the active range finder 24 is comparatively resistant to a change over time. In the object recognition apparatus, calibration with respect to the distance output signal S2 of the stereo camera 26 is appropriately carried out by the distance output signal S1 of the active range finder 24. Therefore, it is possible to realize the object recognition apparatus with less change over time.
[29] An on-board object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 116 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
an operation condition input unit to input an operation condition of a vehicle 12, wherein
the fill light control unit 116 determines an irradiating range W of the floodlight unit 30 as a fill light irradiation unit on the basis of an operation condition inputted by the operation condition input unit.
In the object recognition apparatus, it is possible to irradiate a fill light intensively onto a necessary region or an important region in the detected visual field range V of the stereo camera 26 on the basis of an operation condition input by the operation condition input means. Therefore, an attempt at efficient use of energy or the like can be made.
[30] An object recognition apparatus described in the embodiment is the object recognition apparatus according to the item [29], wherein
the operation condition input unit is an input unit which inputs a steering angle of a steering wheel of a vehicle, and
the fill light control unit 116 determines an irradiating range of the floodlight unit 30 as a fill light irradiation unit on the basis of a steering angle of a steering wheel inputted from the operation condition input unit.
In the object recognition apparatus, it is possible to irradiate a fill light onto only a region corresponding to a direction in which a steering wheel has been turned in the detected visual field V of the stereo camera 26. Therefore, efficient use of energy or the like is possible.
[31] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 116 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
a reliability determining unit 118 to determine reliability in object recognition performance on the basis of the output signal S1 from the active range finder 24 and the output signal S2 from the stereo camera 26.
In the object recognition apparatus, determination on the reliability is carried out by comparing the distance output signal S1 from the active range finder 24 and the distance output signal S2 from the stereo camera 26. Thus, when the reliability is damaged, it is possible to rapidly and accurately handle the problem.
[32] An object recognition apparatus described in the embodiment comprises:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 including an image pickup unit 46 to acquire image information from the target object, and to periodically determine a distance up to a target object by causing the image pickup unit 46 to periodically operate;
an object recognition unit 116 which recognizes the target object on the basis of an output signal S1 from the active range finder 24 and an output signal S2 from the stereo camera 26;
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26; and
a floodlight control unit 96, 120 which, during a period of an operation cycle of the image pickup unit 46, causes the floodlight unit 30 to operate as a fill light irradiation unit of the stereo camera 26 within a time frame of carrying out charge storage, and causes the floodlight unit 30 to operate as the active range finder 24 out of a time frame of carrying out charge storage.
In the object recognition apparatus, imaging operations of the stereo camera 26 using a fill light are periodically carried out, and original ranging operations of the active range finder 24 are carried out between intervals thereof. Accordingly, both detection systems of the active range finder 24 and the stereo camera 26 are effectively operated within a limited period.
[33] An object recognition apparatus described in the embodiment is the object recognition apparatus according to one of the items [1] to [8], and [10] to [32], wherein
the floodlight unit 30 includes a light source 50 and a scanning unit 52 (54, 56) which mechanically scans a light flux from the light source 50.
[34] An object recognition apparatus described in the embodiment is the object recognition apparatus according to one of the items [1] to [32], wherein
the active range finder 24 is a laser radar.
[35] The embodiment describes a calibration method in an object recognition apparatus comprising:
an active range finder 24 having a floodlight unit 30 which floodlights a light onto a target object, and determines a distance up to a target object on the basis of a reflected light therefrom;
a stereo camera 26 to determine a distance up to the target object on the basis of image information from the target object;
an object recognition unit 110 which recognizes the target object on the basis of an output signal from the active range finder 24 and an output signal from the stereo camera 26; and
a fill light control unit 116 which controls the floodlight unit 30 to operate so as to irradiate a light floodlit by the floodlight unit 30 of the active range finder 24 as a fill light of the stereo camera 26, wherein
calibration of a distance output signal of the stereo camera 26 is carried out by using a distance output signal obtained from the active range finder 24.
The object recognition apparatus shown in the embodiments includes the following modified example.
There is provided a structural requirement that a light emission intensity of the laser light source 50 is made to be in a state of being off or comparatively weak immediately after operation, and is made stronger by degrees from the state.
Because a laser beam having a strong intensity can be avoided from being suddenly emitted in a state in which the environment is bright, and when a fill light by the laser light source 50 is not necessarily required, it is possible to prevent energy from being wasted.
Use of the apparatus makes it possible to recognize the surrounding circumstances with great accuracy. Therefore, for example, the apparatus is mounted on a movable body such as a vehicle or the like, so that the apparatus is made extremely useful at the time of making an attempt to safely drive the movable body.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2003-304710 | Aug 2003 | JP | national |
This is a Continuation application of PCT Application No. PCT/JP2004/011927, filed Aug. 19, 2004, which was published under PCT Article 21(2) in Japanese.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP04/11927 | Oct 2004 | US |
| Child | 11286278 | Nov 2005 | US |
