In the accompanying drawings:
Referring to
As shown in
Of these, the on-board camera 10 is mounted on an own vehicle to allow imaging of the front of the own vehicle. The on-board camera 10 includes, for example, an image sensor of which a light-receiving element is a charge-coupled device (CCD). The on-board camera 10 is mounted on the own vehicle and fixed so that an imaging direction of the on-board camera 10 matches a predetermined reference direction (for example, a horizontal direction) when imaging the front of the own vehicle.
The on-board camera 10 is configured so that the shutter speed, the frame rate, the gain of a digital signal outputted to a vehicle-detection controlling unit 30, and the like, can be adjusted depending on an instruction from an internal controlling unit (not shown). The on-board camera 10 outputs a digital signal indicating the luminance (brightness or signal level) of each pixel within the picked-up image to the vehicle detection controlling unit 30 as image data, in addition to a horizontal and vertical synchronizing signal of the image.
A vehicle behavior sensor 20 includes a stroke sensor mounted, for example, on a four-wheel suspension system of the own vehicle. The vehicle behavior sensor 20 detects behavioral changes in a pitch direction and a roll direction of the own vehicle. When the own vehicle runs and the velocity in the back-and-forth and lateral directions of the vehicle accelerate the body of the own vehicle swings or moves in the pitch direction and the roll direction. In accompaniment with the swinging motions, the imaging direction of the on-board camera 10 also shifts from the reference direction. The vehicle behavior sensor 20 provides the vehicle-detection controlling unit 30 with vehicle behavior information used to calculate how much the imaging direction of the on-board camera 10 has shifted from the reference direction.
The vehicle-detection controlling unit 30 performs image processing on the image data inputted from the on-board camera 10, taking into consideration the above-described vehicle behavior information. As a result, the vehicle-detection controlling unit 30 identifies to which, among the rear lights (taillights) of the preceding vehicle, the headlight of the oncoming vehicle, and the reflector mounted on the side of the road, a light source included in the image corresponds. When the vehicle-detection controlling unit 30 identifies the light source in the image to be corresponding to the rear light of the preceding vehicle or the headlight of the oncoming vehicle, the vehicle-detection controlling unit 30 outputs detection information related to the preceding vehicle or the oncoming vehicle to a headlight controlling apparatus 40.
The headlight controlling apparatus 40 controls the direction of headlights HL (headlamps) of the vehicle (refer to
At the same time, when the distance to the preceding vehicle or the oncoming vehicle is equal to or more than the predetermined distance or the preceding vehicle or the oncoming vehicle is not detected, the headlight controlling apparatus 40 sets the direction of the headlight to a high beam. As a result, the headlight controlling apparatus 40 ensures the driver of the own vehicle has visibility for a longer distance. Because detection is performed based on the image data from the on-board camera 10, the preceding vehicle and the oncoming vehicle that are a relatively long distance away (for example, 600 meters) can be detected. Therefore, the headlight controlling apparatus 40 can appropriately control the direction of the headlight.
Next, a principle behind the preceding vehicle and oncoming vehicle detection performed by the vehicle-detection controlling unit 30 will be described.
When the preceding vehicle and the oncoming vehicle are a same distance away from the own vehicle along the running direction at night and the on-board camera 10 images the preceding vehicle and the oncoming vehicle (refer to
At the same time, the farther the headlight of the oncoming vehicle, the rear light of the preceding vehicle, and the light from the reflector are, the darker they are observed to gradually become by the image sensor receiving these lights. A reason for this is as follows. When the light source such as the headlight is a relatively close distance away from the own vehicle, the light from the light source is picked up over a plurality of light-receiving elements within the image sensor. Therefore, the luminance is a constant value. However, when the light source is far, the light is only picked up within a smaller amount of light-receiving elements within the image sensor. In this state, the percentage of the light source in the light-receiving elements within the image sensor decreases as the distance becomes longer. Therefore, the luminance from the light source decreases.
Here, the size of the light on the vehicle is constant to a certain extent. The brightness of the light is also regulated by law. Therefore, a certain correlation is established between the distance from the own vehicle and the luminance observed by the image sensor in the on-board camera 10. The luminance of the light also differs if the light source changes. Therefore, different correlations are established between, for example, when the reflector is the light source and when the light on the vehicle is the light source.
(1) The headlight of the oncoming vehicle is a so-called discharge light. When the direction of the headlight is the low beam (HID[Lo]) and when the direction is the high beam (HID[Hi]) are shown.
(2) The headlight of the oncoming vehicle is a halogen light. When the direction of the headlight is the low beam (Halogen[Lo]) and when the direction is the high beam (Halogen[Hi]) are shown.
(3) The rear light of the preceding vehicle is a bulb light. When a brake light is simultaneously illuminated (Rear[Bulb]+Brake) and when only the bulb light is illuminated (Rear[Bulb]) are shown.
(4) The rear light of the preceding vehicle is a light-emitting diode (LED) light. When a brake light is simultaneously illuminated (Rear[LED]+Brake) and when only the LED light is illuminated (Rear[LED]) are shown.
(5) When the shape of the reflector is a small circle (Reflector[Circle S]), a large circle (Reflector[Circle L]), a rectangle (Reflector[Rect]), and a large square (Reflector[Square L]) are respectively shown.
As described above, if the light source type differs, the correlation between the distance to the light source and the luminance also differs. Taking advantage of this point, according to the embodiment, the light sources that are the headlight of the oncoming vehicle and the rear light of the preceding vehicle are identified from the image picked up by the on-board camera 10. Specifically, based on the measurement results such as those shown in
The distance to the light source and the luminance of the light source are detected from the image data captured by the on-board camera 10. The relationship between the detected distance and luminance is compared with the stored correlations regarding various light sources. The correlation with which the relationship has a highest probability of correspondence is judged. Then, when the relationship is judged to have the highest probability of correspondence with the correlation corresponding to the headlight of the oncoming vehicle or to the rear light of the preceding vehicle, the light source in the image data is detected as the oncoming vehicle or the preceding vehicle.
Hereafter, specific procedures performed by the vehicle-detection controlling unit 30 for vehicle detection will now be described with reference to
First, at Step S210, the vehicle-detection controlling unit 30 loads the image data of the front of the own vehicle picked up by the image sensor in the on-board camera 10 into the memory. The image data includes a signal indicating the luminance of each pixel in the image. Next, at Step S220, the vehicle-detection controlling unit 30 loads information related to the behavior of the own vehicle from the vehicle behavior sensor 20.
At Step S230, the vehicle-detection controlling unit 30 extracts a light source area with high luminance thought to be the light source, from the image data loaded into the memory. The light source area is a bright area within the image data having a higher luminance than a predetermined threshold luminance. A plurality of light source areas is often present. All of the light source areas are extracted. Details of the light source area extracting process will be described with reference to
In the light source area extracting process, first, at Step S310, the vehicle-detection controlling unit 30 performs a binarization process by comparing the luminance of each pixel with the predetermined threshold luminance. Specifically, the vehicle-detection controlling unit 30 assigns ‘1’ to a pixel having a luminance that is equal to or more than the predetermined threshold luminance and ‘0’ to other pixels, thereby creating a binary image. Next, at Step S320, when the pixels to which ‘1’ is assigned are near each other within the binary image, a labeling process is performed to recognize these pixels as a single light source area. As a result, the light source area formed from a group of a plurality of pixels is extracted as one light source area.
Returning to
In the distance calculating process shown in
First, to perform the light-pair-dependent distance calculating process, at Step S410, the vehicle-detection controlling unit 30 performs a light pair creating process to create a pair of lights. The pair of lights including the left light and the right light fulfills the following conditions in the image data picked up by the on-board camera 10. The lights are positioned close together and at almost the same height. The areas of the light source areas are almost the same. The shapes of the light source areas are the same. Therefore, the light source areas fulfilling these conditions are the light pair. A light source area that cannot be paired is considered to be a single light.
When the light pair is formed as shown in
x=f·{W0/(W1·Rh)} (1)
where a reference Rh [mm/pix] is a resolution of the captured image. This calculation is established on the assumption that the own vehicle almost directly faces the preceding vehicle or the oncoming vehicle.
At the same time, when the oncoming vehicle or the preceding vehicle is positioned a long distance away from the own vehicle, the image sensor cannot identify the left light and the right light. The lights are recognized as a single light source, mostly as a single elliptical light source, as illustrated in
The single-light-dependent distance calculating process at Step S430 is a process to calculate the distance when both left and right lights are recognized as a signal light source. This calculation can also be made on the foregoing formula (I), in which w1 is a width of the single elliptical light on the captured image (refer to
Alternatively, the single-light-dependent distance calculating process may be done using information indicating a position of the lights on the captured image. The farther the distance, the closer the lights come to the upper end of the captured image. As shown in
x=h·f/((iy−PIH/2)·Rv−θ0·f) (2),
where h [mm] is the height of the on-board camera 10, f [mm] is a focal distance of the on-board camera 10, Rv [mm/pix] is a resolution of the captured image, PIH [pix] is an height of the captured image, and θ0 [rad] is a depression angle shown in
As described above, the on-board camera 10 is mounted on the own vehicle so that the imaging direction is the reference direction determined in advance. When the lights source is the headlight or the rear light of the vehicle, the height of the light source can be approximated by a rough constant distance (for example, 80 centimeters) from the ground. Therefore, under the assumption that the road is flat, the actual distance to the single light source can be calculated from the distance from the lower edge of the image data to the position of the single light source.
However, when the velocities in the back-and-forth and lateral directions of the vehicle are accelerated, the vehicle swings or moves in the pitch direction or the roll direction. In accompaniment with the swinging motions, the imaging direction of the on-board camera 10 also shifts from the reference direction. Therefore, the vehicle-detection controlling unit 30 determines the distance to the single light source by detecting the behavior of the own vehicle using the vehicle behavior sensor 20 and correcting the position of the single light source by the amount of shifting of the imaging direction.
Returning to
As a result, the vehicle-detection controlling unit 30 can identify whether the light source picked up by the image sensor is the headlight of the vehicle, the rear light of the vehicle, or a reflector that is a disturbance light. Regarding the luminance of each light source, a maximum luminance within the light source area or an average luminance can be used.
At Step S520, the vehicle-detection controlling unit 30 extracts the light source area of which the light source type is still unidentified from the memory. At Step S530, the vehicle-detection controlling unit 30 extracts a maximum luminance value of the light source area. At Step S540, the distance calculated at Step S420 or Step S430 in the flowchart in
Then, at Step S550, the calculated distance and the maximum luminance value are applied to the relationships between distance and luminance range stored in advance. The probability of correspondence is calculated for each stored relationship. The light source area is judged to be the light source type with the highest probability of correspondence.
In the case shown in
In addition, when if the curves “Rear (LED)” and “Reflector (Circle L) are switched to each other in
Instead of giving probabilities of “headlight, rear light, reflector”=“10, 45, 45” % with 10% noise, probabilities on interpolation may be given depending on a condition where a correlational point between the luminance and the distance is positionally closer to which one of the mutually switched curves “Rear (LED)” and “Reflector (Circle L) in
The exemplary embodiment of the present invention has been described above. However, the present invention is not limited in any way to the above-described embodiment. Various modifications can be made to the invention without departing from the scope of the invention.
For example, according to the embodiment, the light source type is identified from the relationship between the luminance of the light source and the distance to the light source, such as those shown in
According to the embodiment, the distance to the light source is determined by the light-pair-dependent distance calculating process or the single-light-dependent distance calculating process being performed. However, regarding the distance to the light source, for example, the distance can be detected using a so-called stereo camera or using another sensor (a distance detecting sensor such as a laser radar sensor).
The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiment and modifications are therefore to be considered in all respects as illustrative and not restrictive, the scope of the present invention being indicated by the appended claims rather than by the foregoing description and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Number | Date | Country | Kind |
---|---|---|---|
2006-211324 | Aug 2006 | JP | national |