The present invention relates to a control apparatus used to monitor the surroundings of a vehicle, and a vehicle surrounding monitoring apparatus provided with that control apparatus.
Apparatuses for monitoring the surroundings of a vehicle by taking images of the surroundings of the vehicle using cameras installed in the vehicle have been proposed heretofore. Also, methods for informing a driver of the presence of obstacles or people (hereinafter “obstacles”) which the vehicle might hit near the vehicle (especially on the path of an ongoing vehicle), have been proposed heretofore. For example, patent literature 1 discloses, dividing a display area on a display apparatus into two, displaying images taken by a camera in the first display area and displaying obstacles detected by image recognition in the second display area.
Furthermore, there is a conventional image monitoring apparatus that monitors the situation in a photographing range and issues a warning only when the image in a detection area defined in advance by the monitoring party using a penlight or the like changes. Patent literature 2 discloses preventing detection errors by issuing a warning only when an intruding object found in a detection area does not touch the outer frame of the detection area.
Patent Literature
[PTL 1]
Japanese Patent Application Laid-Open No. 2008-174076
[PTL 2]
Japanese Patent Application Laid-Open No. HEI4-311186
Technical Problem
However, generally speaking, in obstacle detection by image recognition, there is a possibility that detection results reveal detection errors and missed detections due to noise caused by external factors such as the photographing environment or due to the limit of image recognition processing technology. Then, there are cases where these detection errors and missed detections make the monitoring party confused. For example, when a pedestrian is present in a detection area and yet a warning is not issued due to missed detection, it is difficult for the monitoring party to judge whether a warning is not issued because the pedestrian is outside the detection area or because the system is malfunctioning. Consequently, the monitoring party is not able to intuitively sense the risk of collision with obstacles based on the operating conditions of the monitoring apparatus. In other words, simply informing the monitoring party of an obstacle detection result only when an obstacle is detected, can improve the usability for the monitoring party only to a certain limit.
It is therefore an object of the present invention to provide a control apparatus that improves the usability of a vehicle surrounding monitoring apparatus without confusing the monitoring party while monitoring the surroundings of a vehicle, and a vehicle surrounding monitoring apparatus provided with that control apparatus.
A control apparatus according to the present invention comprises: a setting section that sets a detection area subject to obstacle detection in a photographing range of an imaging section installed in a vehicle; a detection section that detects an obstacle from the detection area set in an image taken by the imaging section; and a processing section that performs image processing so that information for distinguishing between inside and outside of the set detection area is displayed superimposed on the image taken by the imaging section.
According to the present invention, it is possible to improve the usability of a vehicle surrounding monitoring apparatus without confusing the monitoring party while monitoring the surroundings of a vehicle.
Now, embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
(Embodiment 1)
The vehicle surrounding monitoring apparatus of
The imaging section 1, having one or a plurality of cameras installed in a vehicle, takes images of the surroundings of the vehicle and inputs data of photographed images (hereinafter “photographed data”) to the obstacle detection section 4 and synthesized data generating section 5. The kinds of cameras that can be used include CCD (Charge Coupled Device) cameras and CMOS (Complementary Metal Oxide Semiconductor) cameras.
In the following descriptions, the vehicle in which the imaging section 1 is installed in order to monitor the surroundings, will be refereed to as “subject vehicle.”
The imaging section 1, provided as an imaging means, is constituted by, for example, as shown in
In the following description of the present embodiment and in all of the subsequent descriptions of embodiments, cases will be described where the imaging section is constituted by a rear camera 14 alone, for ease of explanation.
The detection area setting section 2, imaging parameter setting section 3, obstacle detection section 4 and synthesized data generating section 5 are implemented by executing a software program in an electronic control unit (ECU) 100 installed in the subject vehicle. The control apparatus according to the present embodiment is formed by combining at least the detection area setting section 2, obstacle detection section 4 and synthesized data generating section 5.
The detection area setting section 2, provided as a setting means, sets an obstacle detection area in the photographing range of the imaging section 1. That is to say, a detection area that can be set up may be the same as the photographing range or may be part of the photographing range. The detection area setting section 2 inputs detection area data showing the detection area having been set, to the obstacle detection section 4 and synthesized data generating section 5. The detection area data represents, for example, a group of coordinates to define the detection area.
For example, a method of inputting specific numerical values is a possible setting method. To be more specific, for example, as shown in
If a detection area is set using a different coordinate system from a photographed image, coordinate conversion needs to be performed for the detection area. This is necessary when, for example, a detection area is set by designating a relative position with respect to the subject vehicle, as shown in
Furthermore, a method of converting three-dimensional coordinates of a detection area into two-dimensional coordinates of a photographed image using imaging parameters described later, is another possible conversion method. For example, if external parameters such as the angle and position for setting a camera on a tripod and internal parameters such as the focal distance are known in advance, it is possible to use a general coordinate conversion method using a pinhole camera model.
The imaging parameter setting section 3 sets, for example, installation parameters upon installing the imaging section 1 in the vehicle. To be more specific, the installation angle and installation position of a camera with respect to a vehicle, information for cutting an, input image upon generating an output image (the angle of view, the number of pixels and so on), and various parameters for geometric transformation, are possible parameters. The imaging parameter setting section 3 inputs imaging parameters having been set, in the detection area setting section 2.
The obstacle detection section 4, provided as a detection means, detects obstacles in a detection area set in a photographed image by means of image recognition processing using photographed data and detection area data received as input from imaging section 1 and detection area setting section 2, respectively. The obstacle detection section 4, upon detecting an obstacle, reports that result to synthesized data generating section 5 and warning section 7. Various items of attribute information, including the number, position, size, moving speed and moving direction of detected obstacles, may be reported as detection results.
As a method of image recognition processing that can be used for obstacle detection, there is background subtraction, whereby an obstacle is detected based on the level of aging of the image in a detection area. Furthermore, a method of storing the pattern of a detection target object in a storage apparatus (not shown) as learning data and searching for an area that matches or resembles the learning data in an input image, can be also used. Even other techniques of image recognition processing are also possible, including a method of using moving object recognition following an optical flow, a method of using a three-dimensional object recognition using a stereo method, and a method combining these. The image recognition processing technique to use can be selected depending on system configurations.
The synthesized data generating 5, provided as a processing means, performs image processing to generate display data for drawing a superimposing detection area on a display image, using detection area data received as input from the detection area setting section 2 and photographed data received as input from the imaging section 1. That is to say the synthesized data generating section 5 synthesizes a photographed image with a display object that makes the detection area visible (for example, a line to define the outer edges of the detection area). Display data therefore refers to data to show a synthesized image. The synthesized data generating section 5 inputs display data having been generated, in the display section 6.
The warning section 7 issues a warning based on an obstacle detection result in the obstacle detection section 4. When the obstacle detection section 4 detects an obstacle in the detection area, the warning section 7 issues a warning to inform the driver of the presence of the obstacle. The kinds of warnings that can be use include sound, voice and light, and these can be changed as appropriate depending on the situation.
The display section 6, provided as a display means, is a display apparatus such as, for example, a liquid-crystal display apparatus. A dedicated in-vehicle monitor or a general television monitor can be used, for example. The display section 6 displays images based on display data received as input from the synthesized data generating section 5. The content to be displayed is a result of combining a detection result in the obstacle detection section 4, a detection area set up in the detection area setting section 2, and an input image photographed in the imaging section 1.
The display apparatus used as the display section 6, should preferably have touch-panel functions that make possible input operations for specifying the detection area.
Several examples of display images will be shown now.
In the example shown in
The details of processing will be described using
In step 4, in the detection area setting section 2, three-dimensional coordinates of the detection area are converted into image coordinates on an input image, using external parameters in the imaging parameters. This conversion is calculated based on, for example, a pinhole camera model used in general camera image conversion. In step S5, in the synthesized data generating section 5, based on an input image and the detection area subjected to coordinate conversion in S4, the detection area is superimposed upon the input image. The method of this superimposition uses frame display/painting, broken lines/solid lines, and so forth, as described earlier. In step S6, synthesized data generating section 5 converts the input image on which a detection area is superimposed, into an output image for display. As for the method of this conversion, the input image is converted into an output image based on information related to cutting, in the imaging parameters. Referring to input image P5 shown in
By this means, according to the present embodiment, image processing for surrounding an area subject to obstacle detection (i.e. detection area) in a display image by frames or painting the area by a transparent color is performed, so that the detection area is shown to the driver with a photographed image, distinctly, in a visual fashion. Consequently, even when detection fails or is missed during obstacle detection processing, the deriver is prevented from unnecessary confusion. The drive is able to judge at ease whether a pedestrian is outside the detection area or the system is malfunctioning.
(Embodiment 2)
The vehicle surrounding monitoring apparatus of embodiment 2 of the present invention will be described now. The vehicle surrounding monitoring apparatus of this embodiment has basically the same configuration as the above embodiment. Therefore, the same components as those of the above embodiment described earlier, will be assigned the same reference numerals and will not be described in detail.
The vehicle surrounding monitoring apparatus of the present embodiment has a configuration adding a detection area operation input section 201 to the configuration described in embodiment 1, as shown in
The detection area operation input section 201 receives an operation input for changing the detection area set in the detection area setting section 2, and has this input reflected in the detection area setting. Operation input reception is made possible by, for example, displaying touch panel switch SW that is operable, in display image P8, on a display apparatus having touch panel functions, as shown in
By this means, according to the present embodiment, the detection area can be set up on a variable basis, according to operation inputs from the monitoring party (e.g. driver). By this means, the actual detection area can reflect the area the monitoring party wants to monitor, thereby improving the usability for the monitoring party even more. Furthermore, a detection area that is displayed can be operated on a display image, thereby allowing the monitoring party to perform changing operations at ease.
(Embodiment 3)
The vehicle surrounding monitoring apparatus of embodiment 3 of the present invention will be described now. The vehicle surrounding monitoring apparatus of the present embodiment has basically the same configuration as the above embodiments. Therefore, the same components as those of the above embodiments described earlier, will be assigned the same reference numerals and will not be described in detail.
The vehicle surrounding monitoring apparatus of the present embodiment has a configuration adding a driving state information acquiring section 301 to the configuration described in embodiment 1, as shown in
The driving state information acquiring section 301 acquires a subject vehicle driving state detection result as driving state information. The driving sate information acquiring section 301 has this detected driving state in the obstacle detection area setting.
This driving state information refers to, for example, information showing physical quantities in the driving state. Examples of physical quantities to show the driving state include, for example, driving speed and traveling direction. Means that can be used to measure the traveling direction include, for example, a steering angle sensor and an angular velocity sensor, and means that can be used to measure the driving speed include, for example, a vehicle speed meter and accelerometer.
By this means, when the state of the steering wheel can be acquired as a state of driving, the obstacle detection area can be adjusted according the turning of the steering wheel.
Furthermore, it is possible to change the detection area depending on the driving speed detected. That is, the detection area can be set on an arbitrary basis by, for example, confining the obstacle detection area to a nearby area of the subject vehicle when the driving speed is slower, and extending the obstacle detection area further when the driving speed is faster.
Thus, when the driving speed can be acquired as a state of driving, it is possible to adjust the obstacle detection area according to the traveling speed.
Thus, according to the present embodiment, it is possible to optimize the detection obstacle detection area for the driving state of the subject vehicle. Furthermore, an optimized detection area is displayed on the display section 6, and the monitoring party can see and check it.
(Embodiment 4)
The vehicle surrounding monitoring apparatus of embodiment 4 of the present invention will be described now. The vehicle surrounding monitoring apparatus of this embodiment has basically the same configuration as the above embodiments. Therefore, the same components as those of the above embodiments described earlier, will be assigned the same reference numerals and will not be described in detail.
The vehicle surrounding monitoring apparatus of the present embodiment has a configuration adding a photographing environment information acquiring section 401, to the configuration described in embodiment 1, as shown in
The photographing environment information acquiring section 401 acquires photographing environment detection results as photographing environment information. Then, the photographing environment information acquiring section 401 has this detected photographing environment in the obstacle detection area setting.
This photographing environment information refers to information showing external environmental conditions of the subject vehicle. To be more specific, if an illuminance sensor or a means for sensing the brightness in the external environment from time information and switching information of the subject vehicle's lights is provided in the subject vehicle, this detection result is acquired as photographing environment information. Furthermore, if a raindrop sensor, image sensor, radar sensor and other appropriate information communications means that can detect the occurrence of rainfall, snowfall and fog are provided in the subject vehicle, these detection results are acquired as photographing environment information.
By this means, the obstacle detection area can be changed depending on the photographing environment detected. That is, the detection area can be set on an arbitrary basis by, for example, confining the obstacle detection area to a nearby area of the subject vehicle when it is darker around subject vehicle, and extending the obstacle detection area further when it is brighter around the subject vehicle. For example, also in the event of the occurrence of rainfall or fog, the obstacle detection area may be confined to a nearby area of the subject vehicle. By this means, the accuracy of obstacle detection can be maintained at a certain level or above regardless of the photographing environment.
Thus, according to the present embodiment, it is possible to optimize the detection obstacle detection area for the photographing environment. Furthermore, an optimized detection area is displayed on the display section 6, and the monitoring party can see and check it.
(Embodiment 5)
The vehicle surrounding monitoring apparatus of embodiment 5 of the present invention will be described now. The vehicle surrounding monitoring apparatus of this embodiment has basically the same configuration as the above embodiments. Therefore, the same components as those of the above embodiments described earlier, will be assigned the same reference numerals and will not be described in detail.
The vehicle surrounding monitoring apparatus of the present embodiment has a configuration adding a map information acquiring section 501 to the configuration described in embodiment 1, as shown in
The map information acquiring section 501 acquires map information of the surroundings of the subject vehicle. The map information acquiring section 501 has this acquired map information in the obstacle detection area setting.
This map information here includes various items of information that can be acquired using a car-navigation system or other appropriate radio communications means, such as the form of roads (e.g. intersections), road signs (e.g. pedestrian crossing) and facility attributes (e.g. grade school).
By this means, when information about the form of roads can be acquired as map information, the obstacle detection area can be adjusted according the turning of the steering wheel.
Thus, according to the present embodiment, it is possible to optimize the detection obstacle detection area based on map information of the surroundings of the vehicle. Furthermore, an optimized detection area is displayed on the display section 6, and the monitoring party can see and check it.
Embodiments of the present invention have been described above. The above embodiments can be implemented with various changes. The above embodiments can also be implemented in various combinations.
The disclosure of Japanese patent application No. 2009-233321, including the specification, drawings and abstract, is incorporated herein by reference in its entirety.
The control apparatus of the present invention provides an advantage of improving the usability of a vehicle surrounding monitoring apparatus without confusing the monitoring party while monitoring the surroundings of a vehicle, and is applicable to a vehicle surrounding monitoring apparatus.
1 Imaging section
2 Detection area setting section
3 Imaging parameter setting section
4 Obstacle detection section
5 Synthesized data generating section
100, 200, 300, 400, 500 ECU
201 Detection area operation input section
301 Driving state information acquiring section
401 Photographing environment information acquiring section
501 Map information acquiring section
Number | Date | Country | Kind |
---|---|---|---|
2009-233321 | Oct 2009 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2010/001959 | 3/18/2010 | WO | 00 | 1/7/2011 |