Patent Application
20150262365
The disclosure of Japanese Patent Application No. 2014-049456 filed on Mar. 12, 2014 including the specification, drawings and abstract is incorporated herein by reference in its entirety.
1. Field of the Invention
The present invention relates to an image processing device and an image processing method.
2. Description of Related Art
Conventionally, there is a technology that detects a particular object such as a human face based on the disparity. For example, Japanese Patent Application Publication No. 2007-280088 (JP 2007-280088 A) discloses a technology that generates a depth map based on the disparity between two images, sets a detection area of a size corresponding to a distance value in the depth map, and determines whether the object in the detection area is a human face.
It is desired to reduce a reduction in accuracy when extracting a target based on the disparity of image data captured by an imaging device. For example, when the difference between the distance from the imaging device to a desired target and the distance from the imaging device to the background is small, the disparity value becomes small. This reduction in disparity, in turn, reduces the extraction accuracy when the target area is extracted from the image data. When the contour is extracted from a wider range using the disparity information including not only the face but also the body and legs of a person, the extraction target includes a part where the distance to the background is short and a part where the distance to the background is long. In the part where the distance to the background is short, the difference between the disparity value obtained from the image data in that part and the disparity value obtained from the image data in the nearby background position becomes small, sometimes reducing the separation/extraction accuracy. This reduction tends to increase the divergence between the extracted shape and the real shape of the target in the part where the distance to the background is short. Therefore, when comparison such as pattern matching is performed based on the extracted shape, the accuracy of pattern matching is reduced if the divergence between the extracted shape and the real shape is large.
The present invention provides an image processing device and an image processing method that can increase the extraction accuracy when the area of a target is extracted from image data based on the disparity.
An image processing device according to a first aspect of the present invention includes an imaging device and an electronic control unit. The imaging device is configured to capture an image of surroundings. The electronic control unit includes a first extraction unit, a disparity calculation unit, a second extraction unit, and an object determination unit. The first extraction unit is configured to extract a main area, which is an image area including a candidate for a target, based on image data captured by the imaging device. The disparity calculation unit is configured to calculate a disparity based on image data captured by the imaging device. The second extraction unit is configured to extract a candidate area, which is an image area of the candidate, from the main area based on the disparity. The object determination unit is configured to determine whether an image of the candidate area is an image of the target by comparing a shape of the candidate area and a predetermined shape of the target. An extraction of the candidate area from a lower part area in the main area is restricted more strictly than an extraction of the candidate area from an upper part area in the main area. The lower part area constitutes a lower part of the main area and is provided below a predetermined position in the main area. The upper part area constitutes an upper part of the main area and is provided above the predetermined position in the main area.
The image processing device according to the first aspect of the present invention achieves the effect that the extraction accuracy for extracting the area of a target from image data based on the disparity is increased.
In the image processing device described above, the predetermined position may be a position that is set as a reference position.
In the image processing device described above, the electronic control unit may further include a distance identification unit configured to identify a distance from the imaging device to the candidate for the target.
In the image processing device described above, in the main area, the reference position when the distance identified by the distance identification unit is large may be set above the reference position when the distance is small.
In the image processing device described above, the second extraction unit may be configured to extract only a part of the main area as the candidate area. In the part of the main area, the distance between the candidate for the target and the imaging device is within a predetermined range with respect to the distance identified by the distance identification unit. In addition, the predetermined range of the lower part area may be smaller than the predetermined range of the upper part area.
In the image processing device described above, a degree of divergence between the predetermined range of the lower part area in the main area and the predetermined range of the upper part area may be larger when the distance identified by the distance identification unit is large than when the distance is small.
In the image processing device according to the first aspect of the present invention, a degree of restriction on the extraction of the candidate area from the lower part area when the distance identified by the distance identification unit is large may be set higher than a degree of restriction on the extraction of the candidate area from the lower part area when the distance is small.
In the image processing device according to the first aspect of the present invention, the second extraction unit may be configured to extract a part of the main area as the candidate area. In the part of the main area, the distance between the candidate for the target and the imaging device is within a predetermined range with respect to the distance identified by the distance identification unit. In addition, an extraction of the candidate area may be inhibited in the lower part area.
In the image processing device according to the first aspect of the present invention, the extraction of the candidate area may be inhibited in the lower part area.
An image processing method according to a second aspect of the present invention includes: capturing, by an imaging device, an image of surroundings to generate image data; extracting, by an electronic control unit, a main area, which is an image area including a candidate for a target, based on the image data; calculating, by the electronic control unit, a disparity based on the image data; extracting, by the electronic control unit, a candidate area, which is an image area of the candidate, from the main area based on the disparity; determining, by the electronic control unit, whether an image of the candidate area is an image of the target by comparing a shape of the candidate area and a predetermined shape of the target; and restricting, by the electronic control unit, the extraction of the candidate area from a lower part area in the main area more strictly than an extraction of the candidate area from an upper part area in the main area. The lower part area constitutes a lower part of the main area and is provided below a predetermined position in the main area. The upper part area constitutes an upper part of the main area and is provided above the predetermined position in the main area.
The image processing method according to the second aspect of the present invention achieves the effect that the extraction accuracy for extracting the area of a target from image data based on the disparity is increased.
Features, advantages, and technical and industrial significance of exemplary embodiments of the invention will be described below with reference to the accompanying drawings, in which like numerals denote like elements, and wherein:
An image processing device and an image processing method in embodiments of the present invention are described below in detail with reference to the drawings. The embodiments below are not intended to limit the scope of the present invention. Elements described in the embodiments include their variations readily thought of by those skilled in the art and substantially equivalent elements.
[First embodiment] A first embodiment is described below with reference to
As shown in
As shown in
Each of the right-side camera 2R and the left-side camera 2L is connected to the ECU 3. The image data captured by the cameras 2R and 2L is output to the ECU 3. The ECU 3 has the function as an image-processing device that processes the image data captured by the stereo camera imaging device 2. The ECU 3 is an electronic control unit that includes a computer. The ECU 3 sends an imaging command to the right-side camera 2R and the left-side camera 2L at a predetermined time interval. It is desirable that the capturing times be synchronized between the right-side camera 2R and the left-side camera 2L. The ECU 3 acquires image data captured and generated by the cameras 2R and 2L in response to the imaging command.
As shown in
An example of the disparity calculation method is described with reference to
In
The disparity calculation unit 4 calculates the degree of correlation between the comparison target block BKC, which is shifted various shift amounts from the corresponding block BK0, and the source block BK. For example, the disparity calculation unit 4 shifts the comparison target block BKC in the horizontal direction or the vertical direction, one pixel at a time, and calculates the degree of correlation each time the comparison target block BKC is shifted. After that, the disparity calculation unit 4 calculates the disparity of the target pixel PX based on the comparison target block BKC that has the highest degree of correlation with the source block BK. For example, the disparity calculation unit 4 calculates the difference in the brightness of each corresponding pixels between the source block BK and the comparison target block BKC and calculates the sum (or the sum of squares) of the brightness differences. The smaller the calculated sum of brightness differences is, the higher the degree of correlation between the source block BK and the comparison target block BKC is.
The disparity calculation unit 4 determines the comparison target block BKC with the highest degree of correlation with the source block BK, and determines the shift amount, by which the comparison target block BKC is shifted from the corresponding block BK0, to be the disparity of the target pixel PX. The maximum shift amount Smax for shifting the comparison target block BKC from the corresponding block BK0 is determined in advance.
The disparity calculation unit 4 may calculate the disparity using the method described in H. Hirschmuller, “Accurate and efficient stereo processing by semi-global matching and mutual information”, CVPR, 2005, pp. 807-814.
The first extraction unit 5 extracts a main area (see the symbol R1 in
The first extraction unit 5 extracts the main area R1 based on the brightness of the pixels of the image data DR and DL. The first extraction unit 5 in this embodiment extracts the main area R1 based on one of the right image data DR and the left image data DL. To extract the main area R1, the first extraction unit 5 first extracts a pixel area PX1 which is an area included in the right image data DR and the brightness of which is largely different from that of its surrounding. The first extraction unit 5 matches the contour shape of the extracted block of the captured pixel area PX1 with the pedestrian's contour shape, stored in advance, to determine whether the image of the pixel area PX1 is an image generated by capturing a pedestrian.
The first extraction unit 5 may extract a pedestrian based on the brightness using the method described in N. Dalaland B. Triggs, “Histogram of oriented gradient for human detection”, CVPR, 2005.
If the degree of correlation between the contour shape of the pixel area PX1 and the contour shape of the pedestrian is high, the first extraction unit 5 temporarily determines that the image of the pixel area PX1 is the image of a pedestrian. In addition, as will be described later in this embodiment, the extraction condition is adjusted for the pixel area PX1, which is temporarily determined as the pixel area of a pedestrian, based on the position in the height direction. The image processing device 100 extracts the final candidate area for pattern matching based on the adjusted extraction condition.
After the image in the pixel area PX1 is temporarily determined as the image of a pedestrian, the first extraction unit 5 extracts the main area RI. The main area R1 is extracted so that the pixel area PX1 is included as shown in
The distance identification unit 6 identifies the distance from the stereo camera imaging device 2 to the target candidate based on the image data captured by the stereo camera imaging device 2. The distance identification unit 6 in this embodiment calculates the distance based on the disparity calculated by the disparity calculation unit 4. In this embodiment, the distance identification unit 6 sets a distance calculation area R11 in the main area R1. The distance identification unit 6 calculates the distance between the stereo camera imaging device 2 and the target candidate based on the disparity information on the pixels in the distance calculation area R11.
The distance calculation area R11 is a rectangular area as shown in
The distance identification unit 6 calculates the distance between the stereo camera imaging device 2 and the imaging object, temporarily determined as a pedestrian, from the disparity values of the pixels in the distance calculation area R11. The distance identification unit 6 calculates the representative distance as a value that represents the distance between the stereo camera imaging device 2 and the target candidate temporarily determined as a pedestrian. The value of the disparity between a pixel in the right image data DR and a pixel in the left image data DL corresponds to the distance between the imaging object of the pixel and the stereo camera imaging device 2. For example, the distance identification unit 6 calculates the median or the average of the disparities of the pixels, included in the distance calculation area R11, as the representative disparity. This representative disparity corresponds to the representative distance between the target candidate and the stereo camera imaging device 2. The distance identification unit 6 may output the representative disparity value directly as the value corresponding to the representative distance or may convert the representative disparity to a distance and then output the representative distance. The distance identification unit 6 in this embodiment calculates the representative distance L0 and outputs the calculated representative distance L0 to the extraction range setting unit 7. The larger the distance between the stereo camera imaging device 2 and the target candidate is, the smaller the disparity value is. Conversely, the smaller the distance is, the larger the disparity value is. Therefore, when the distance identification unit 6 outputs the representative disparity value as a value corresponding to the representative distance L0, the processing by the extraction range setting unit 7 and so on should be performed based on the above-described correlation between the disparity value and the distance.
The extraction range setting unit 7 sets a range that is included in the main area R1 and is used for extracting an actual target. First, the problem that arises when a target is extracted based on the disparity is described with reference to
As shown in
To address this problem, the image processing device 100 in this embodiment changes the vertical-direction ranges, which are used for extracting the shape of a solid object, according to the distance to a target candidate as described below. Changing the vertical-direction ranges in this manner enables the shape of a target candidate to be extracted accurately, resulting in a reduction in accuracy when a candidate for a distant target is extracted. With reference to
As shown in
The extraction range setting unit 7 sets the extraction permission area R12 by referring to the map shown in
As shown in
Therefore, when the same target candidate is extracted, the aspect ratio of the extraction permission area R12 changes in this embodiment according to the representative distance L0. The more distant a target is from the vehicle 1, the smaller is the area in which the target is captured in the image. Therefore, the more distant the target is, the smaller is the size of the main area R1. In this embodiment, the vertical-direction width hp of the extraction permission area R12 is reduced at a reduction rate higher than the reduction rate at which the main area R1 is reduced when the distance to a target becomes large. Therefore, the nearer the target candidate is to the stereo camera imaging device 2, the thinner the extraction permission area R12 becomes in the vertical direction. On the other hand, the more distant the target candidate is, the smaller does the ratio of the vertical frame width hp to the horizontal frame width of the extraction permission area R12 become.
The value of the height h, which depends on the distance d, may be determined experimentally in advance. For example, for a combination of the height from the road surface to the stereo camera imaging device 2, distance d, and height h, the extraction accuracy of target extraction is evaluated based on the experimental result. As a result, the map shown in
The second extraction unit 8 extracts the candidate area PX3, which is the image area of a target candidate, from the main area R1 based on the disparity. The second extraction unit 8 separates the target candidate and the background based on the disparity values of the pixels in the extraction permission area R12. The extraction method used by the second extraction unit 8 to extract a target candidate is described below with reference to
From the extraction permission area R12, the second extraction unit 8 extracts an area, where the distance between the imaging object and the stereo camera imaging device 2 is within a predetermined range of the distance identified by the distance identification unit 6 (representative distance L0), as a candidate area. For example, if the distance from the stereo camera imaging device 2 is within a range of a predetermined distance σ of the representative distance L0 in the depth direction as shown in
If a pixel is included in the extraction permission area R12 and its distance corresponding to the disparity value is within a predetermined range of the representative distance L0, the pixel is determined to be a pixel in which the target candidate is captured. More specifically, for each pixel in the extraction permission area R12, the second extraction unit 8 converts the disparity value of the pixel to the distance LP from the stereo camera imaging device 2. The second extraction unit 8 determines that, if the calculated distance LP below satisfies expression (1) below, the pixel is a pixel in the candidate area.
L0−σ≦LP≦L0+σ (1)
As shown in
The object determination unit 9 compares the shape of the candidate area PX3 with the shape of a predefined target to determine whether the image of the candidate area PX3 is the image of the target. The object determination unit 9 in this embodiment performs comparison via pattern matching. The ECU 3 stores the shape of a pedestrian as the shape of the target in advance. The ECU 3 stores the shapes of a plurality of pedestrians having different facing directions, poses, and physiques. The object determination unit 9 performs pattern matching between these stored shapes and the shape of the candidate area PX3. Based on the result of comparison between the shape of the candidate area PX3 and the stored shapes, the object determination unit 9 determines whether the candidate area PX3 is the image of a pedestrian. In this embodiment, an area which is in the main area R1 and in which the lower part below the boundary line LB, indicating a predetermined position, is captured (that is, an area in which the extraction accuracy is relatively low) is the extraction inhibition area R13. That is, the extraction of the candidate area PX3 from the extraction inhibition area R13 is restricted more strictly than the extraction of the candidate area PX3 from the extraction permission area R12 in which the upper part above the boundary line LB, which is the reference position, is captured. This reduces the possibility that a low-accuracy extraction result is included in the candidate area PX3, allowing the object determination unit 9 to perform high-accuracy pattern matching.
The operation of the image processing device 100 in this embodiment is described below with reference to
In step S2, the disparity calculation unit 4 calculates the disparity. The disparity calculation unit 4 calculates the disparity between the right image data DR and the left image data DL through the stereo disparity processing as described above by referring to
In step S3, the first extraction unit 5 extracts the main area R1. The first extraction unit 5 extracts the main area R1 as described above by referring to
In step S4, the distance identification unit 6 calculates the distance to the detection object detected as a candidate for a pedestrian. The distance identification unit 6 sets the median of the disparity values, calculated for the pixels in the distance calculation area R11, as the distance from the stereo camera imaging device 2 to the detection object as described above by referring to
In step S5, the extraction range setting unit 7 sets the range in which the contour shape can be detected. The extraction range setting unit 7 sets the range in step S5 based on the value of the representative distance L0 calculated by the distance identification unit 6. For example, the extraction range setting unit 7 sets the extraction permission area R12 as described above by referring to
In step S6, the second extraction unit 8 extracts the contour shape using the disparity in the range that is set in step S5. The second extraction unit 8 extracts the candidate area PX3 from the extraction permission area R12 based on the disparity data on the pixels in the extraction permission area R12 that is set in step S5. For example, as described with reference to
After the candidate area PX3 is determined, the second extraction unit 8 extracts the contour shape of the candidate area PX3. For example, a pixel, which is one of the pixels on each row in the candidate area PX3 and is at the end of continuous pixels, is determined to be a pixel that forms the contour. Similarly, a pixel, which is one of the pixels on each column in the candidate area PX3 and is at the end of continuous pixels, is determined to be a pixel that forms the contour. The second extraction unit 8 outputs the extracted contour shape to the object determination unit 9. After step S6 is performed, the processing proceeds to step S7.
In step S7, the object determination unit 9 determines whether the contour shape is that of a pedestrian. The object determination unit 9 performs pattern matching between the contour shape, extracted by the second extraction unit 8, and a pre-stored pedestrian model. For pattern matching with the model, the object determination unit 9 increases or decreases the size of the contour shape so that the scale of the extracted contour shape matches the scale of the model. For example, the object determination unit 9 may perform this determination via Support_Vector_Machines (SVM) by replacing the part of Histogram_of oriented_gradient (HOG) of the method, described in N. Dalaland B. Triggs, “Histogram of oriented gradient for human detection”, CVPR, 2005, with a binary image.
The object determination unit 9 determines whether the contour shape, extracted in step S6, is the shape of a pedestrian and outputs the determination result. After step S7 is performed, the control flow is terminated.
As described above, the image processing device 100 in this embodiment restricts the extraction of the candidate area PX3 as follows. That is, with the boundary line LB as the predetermined position in the main area R1, the image processing device 100 restricts the extraction of the candidate area PX3 from the area (extraction inhibition area R13), in which the lower part below the boundary line LB is captured, more strictly than the extraction of the candidate area PX3 from the area (extraction permission area R12) in which the upper part above the predetermined position is captured. In this embodiment, the degree of restriction on the extraction of the candidate area PX3 is the maximum in the extraction inhibition area R13 and, in this area, the extraction of the candidate area PX3 is inhibited. In this way, this method efficiently reduces the loss in the extraction accuracy of the candidate area PX3.
In the image processing device 100 in this embodiment, the boundary line LB when the distance (representative distance L0) identified by the distance identification unit 6 is large is set at an upper position in the main area R1 than when the distance is small. This is implemented by the following two, (i) and (ii). (i) The height h, shown in
The ECU 3 in this embodiment may further include a support unit that supports the driver's driving operation. The support unit supports the driver based on the determination result in step S7. The support method includes a pedestrian's presence transmission method performed by the information transmission unit that transmits the pedestrian's presence information to a driver and a driving operation support method performed by the operation support unit. The pedestrian's presence transmission method includes the transmission of an alert or a warning to a driver based on the relative position and the relative speed between a detected pedestrian and the vehicle 1 as well as the suggestion of the avoidance operation for avoiding an approach to a pedestrian. The driving operation support method includes the avoidance of an approach to a detected pedestrian by assisting a driver in performing the input operation for the driving source (engine, motor generator, etc.), the brake device, and the steering device. The ECU 3, which has the support unit, can function as a control device that controls the vehicle 1.
According to the image processing device 100 in this embodiment, the candidate area PX3 is extracted by narrowing a candidate to an image area (extraction permission area R12) where the candidate can be extracted accurately, based on the distance to the extracted pedestrian candidate. Therefore, when the candidate for a pedestrian is distant from the vehicle 1, the upper part of the pedestrian (for example, the head or the upper half of the body) is extracted as the candidate area PX3. Even when a pedestrian is distant from the vehicle, the upper part of the pedestrian can be accurately extracted because the disparity with the background is relatively large. By pattern performing matching with a pedestrian model based on the candidate area PX3 that is accurately extracted as described above, the image processing device 100 can accurately determine whether the candidate area PX3 is a pedestrian. That is, from the time a pedestrian is distant from the vehicle, the presence of the pedestrian can be determined accurately. In addition, inhibiting the lower part of a pedestrian from being extracted reduces the possibility that a low-accuracy extraction result is included in the candidate area PX3. Therefore, this method reduces the possibility that, though actually a pedestrian, the candidate area PX3 is incorrectly determined not to be a pedestrian.
As the distance between the vehicle 1 and a pedestrian becomes smaller, the extraction permission area R12 is extended in the main area R1 in the direction of the lower end of the pedestrian. Therefore, as the vehicle 1 approaches the pedestrian, it becomes possible to extract the candidate area PX3 that includes a larger part of the pedestrian. This ability allows the more detailed information on the pedestrian, such as the facing direction and the pose of the pedestrian, to be obtained through pattern matching. For example, it becomes possible to determine not only whether the extracted candidate is actually a pedestrian but also whether the pedestrian is a child or whether the pedestrian is crossing the road in front of the vehicle 1. This detailed information enables the pedestrian to be classified more accurately.
(Image processing method) An image processing method is disclosed in the first embodiment. The image processing method includes the following procedures: a capturing procedure for capturing the image of the surroundings; a first extraction procedure for extracting a main area that is an image area including a target candidate based on the image data captured by the capturing procedure; a disparity calculation procedure for calculating the disparity based on the image data captured by the capturing procedure; a second extraction procedure for extracting a candidate area, which is the image area of the candidate, from the main area; and an object determination procedure for determining whether the image of the candidate area is the image of a target based on a comparison between the shape of the candidate area and the shape of the predetermined target. The extraction of the candidate area from the area, which is in the main area and in which the lower part below the predetermined position is captured, is restricted more strictly than the extraction of the candidate area from the area in which the upper part above the predetermined position is captured.
The capturing procedure is performed, for example, by the stereo camera imaging device 2. The first extraction procedure is performed, for example, by the first extraction unit 5. The disparity calculation procedure is performed, for example, by the disparity calculation unit 4. The second extraction procedure is performed, for example, by the second extraction unit 8. The object determination procedure is performed, for example, by the object determination unit 9.
[Modification of first embodiment] A modification of the first embodiment is described below.
As shown in
In this modification, the height h that determines the extraction permission area R12 is changed in three stages. Instead of this, the height h that determines the extraction permission area R12 may be changed in two stages or in four or more stages.
[Second embodiment] A second embodiment is described below with reference to
As shown in
As shown in
On the other hand, for a pixel in the lower part area R1L, if the difference between the distance LP, from the imaging object of the pixel to the stereo camera imaging device 2, and the representative distance L0 exceeds the second predetermined distance σ2, the pixel is not determined to be a pixel in which the target candidate is captured. For the upper part area R1U, the second extraction unit 8 extracts the candidate area PX3 based on the first predetermined distance σ1. On the other hand, for the lower part area R1L, the second extraction unit 8 extracts the candidate area PX3 based on the second predetermined distance σ2. The other operations of the image processing device 100 are the same as those in the first embodiment described above.
In this embodiment, the predetermined distance σ differs between the upper part and the lower part of the main area R1 as described above. The predetermined range (range of ±σ2) for the lower part area R1L, which is included in the main area R1 and in which the lower part below the predetermined position is captured, is smaller than the predetermined range (range of ±σ1) for the upper part area R1U in which the upper part above the predetermined position is captured. This difference in the predetermined ranges (±σ2 and ±σ1) restricts the extraction of the candidate area PX3 from the lower part area R1L more strictly than the extraction of the candidate area PX3 from the upper part area R1U. Therefore, when extracting the lower part of the pedestrian PE, this embodiment reduces the possibility that the background, such as the road surface, is incorrectly detected as a part of the pedestrian PE.
In addition, the second predetermined distance σ2 changes in this embodiment according to the distance identified by the distance identification unit 6. The second predetermined distance σ2 when the representative distance L0 from the stereo camera imaging device 2 to the pedestrian PE is large is smaller than the second predetermined distance σ2 when the representative distance L0 is small. That is, the degree of restriction on the extraction of the candidate area PX3 from the lower part area R1L when the distance (representative distance L0) identified by the distance identification unit 6 is large is set higher than the degree of restriction on the extraction of the candidate area PX3 from the lower part area R1L when the distance is small. In other words, the error range of the distance or disparity, which is allowed when the candidate area PX3 is determined to be the image of a pedestrian in the lower part area R1L, is set smaller when the pedestrian PE is distant from the vehicle than when the pedestrian PE is near the vehicle. This means that, when the pedestrian PE is distant from the vehicle, the degree of restriction on the extraction of the candidate area PX3 in the lower part area R1L is increased.
When the representative distance L0 becomes larger and, as a result, the second predetermined distance σ2 is set smaller, the divergence between the first predetermined distance σ1 and the second predetermined distance σ2 is increased. That is, the degree of divergence between the predetermined range (range of ±σ2) for the lower part area R1L and the predetermined range (range of ±σ1) for the upper part area R1U becomes larger when the representative distance L0 is large than when the representative distance L0 is small. Therefore, considering the characteristics of the stereo camera method that tends to decrease in extraction precision as the distance to the object becomes larger, the extraction condition for the lower part area R1L when the detection object is distant from the vehicle can be made stricter.
A change in the second predetermined distance σ2 according to a change in the representative distance L0 may be made continuously or in stages. When the representative distance L0 is a distance equal to or larger than a predetermined value, the second predetermined distance σ2 may be set to 0. When second predetermined distance σ2 is set to 0, the candidate area PX3 almost similar to that in the first embodiment described above can be obtained.
In the second embodiment, the boundary line between the upper part area R1U and the lower part area R1L may be set at a fixed point regardless of the representative distance L0. In addition, the main area R1 may be divided into three or more areas in the vertical direction. In this case, it is desirable that the value of the predetermined distance σ be largest in the uppermost area and that the value of the predetermined distance σ be decreased as the area is lower.
In addition, the second embodiment may be performed in combination with the first embodiment. For example, the upper part area R1U and the lower part area R1L similar to those in the second embodiment may be provided in the extraction permission area R12. This configuration inhibits the candidate area PX3 from being extracted in the extraction inhibition area R13 and, in the extraction permission area R12, restricts the extraction of the candidate area PX3 in an area in which a relatively lower part is captured.
[First modification of second embodiment] A first modification of the second embodiment is described below.
As shown in
According to this modification, the predetermined distance σ can be changed, as necessary, according to the vertical-direction distance from the contact surface such as the road surface. This allows the extraction condition to be changed according to the ease with which the disparity between the pedestrian PE and the background is obtained, thereby reducing the loss in extraction accuracy. It is also possible to change the second predetermined distance σ2 according to the representative distance L0 in the same manner as when the predetermined distance σ of the intermediate area R1M is changed. In addition, when the representative distance L0 is equal to or larger than a predetermined distance, the second predetermined distance σ2 may be set to 0. It is also possible to change the position of the intermediate area R1M according to the representative distance L0. For example, when the representative distance L0 is large, the intermediate area R1M may be positioned higher than when the representative distance L0 is small.
In addition, this modification may be performed in combination with the first embodiment. For example, the upper part area R1U, the intermediate area R1M, and the lower part area R1L, such as those used in this modification, may be provided in the extraction permission area R12.
[Second modification of second embodiment] A second modification of the second embodiment is described below.
For example, the value of the predetermined distance σ changes linearly according to a position in the vertical direction as shown in
In this modification, the minimum value σ3 of the predetermined distance σ is variable. The value of the minimum value σ3 changes according to the representative distance L0. The minimum value σ3 when the representative distance L0 is large is smaller than the minimum value σ3 when the representative distance L0 is small. That is, the more distant the pedestrian PE is from the vehicle 1, the smaller the minimum value σ3 is. On the other hand, the maximum value σ4 of the predetermined distance σ is constant. The more distant the pedestrian PE is from the vehicle 1, the smaller the minimum value σ3 is. Therefore, the more distant the pedestrian PE is from the vehicle 1, the more difficult it becomes to extract the feet of the pedestrian PE as the candidate area PX3. In addition, the more distant the pedestrian PE is from the vehicle 1, the higher the area, where it is difficult to extract it as the candidate area PX3, is extended upwards in the pedestrian PE. When the representative distance L0 is equal to or larger than a predetermined distance, the minimum value σ3 may be set to 0.
In addition, this modification may be performed in combination with the first embodiment described above. For example, the predetermined distance o may be decreased in the extraction permission area R12 as the position becomes lower.
[Modifications of above embodiments] Although mounted on the vehicle 1 in the first embodiment and the second embodiment described above, the image processing device 100 need not always be mounted on the vehicle 1. The image processing device 100 may be mounted on a moving body other than a vehicle or on a structure other than a moving object. The image processing device 100 may be handled singly as a unit. The image processing device 100 may be configured by a special device or at least a part of its components may be configured by general-purpose components. For example, the stereo camera imaging device 2 may be configured by a combination of two general-purpose cameras. Instead of the ECU 3, a general-purpose computer (for example, a PC) may be used. The stereo camera imaging device 2 may capture a stereo image, not with two cameras, but with one camera.
Although the area in front of the vehicle 1 is captured by the stereo camera imaging device 2 in the embodiments and the modifications described above, the capturing direction is not limited to the front. For example, the stereo camera imaging device 2 may capture an image at the side or in the back of the vehicle 1. Instead of calculating the distance from the imaging device to a candidate for a target based on the disparity, the distance identification unit 6 may identify the distance to a candidate for a target with a distance information acquisition unit such as a radar detector.
Although the target to be determined through pattern matching is typically a pedestrian, other targets may also be a target to be determined. Other targets include moving objects, which move in the traffic environment around the vehicle, such as a preceding vehicle and a bicycle. Other targets may also include fixed objects such as a telegraph pole or a pole.
The contents disclosed in the embodiments and the modifications described above may be performed as necessary in any combination.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2014-049456 | Mar 2014 | JP | national |
