This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2009-272860, filed on Nov. 30, 2009, the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to an on-vehicle device mounted on a vehicle and a recognition support system including the on-vehicle device.
2. Description of the Related Art
Conventionally, there is known an on-vehicle device provided with a blind corner monitor (hereinafter described as “BCM”) for displaying an image obtained by imaging a side ahead of a vehicle or a rearward thereof, by a camera, which is a blind corner for a driver.
For example, Japanese Patent Application Laid-open No. 2009-67292 discloses an on-vehicle device that switches, when it is detected that an own vehicle is about to enter an intersection based on position information of the own vehicle and road information, from a screen displayed on a display unit or from a map as a navigation function (hereinafter described as a “navigation screen”) to a camera image, and displays the camera image. This allows a driver to visually recognize an area becoming a blind corner for him or her through the camera image.
There is also known an on-vehicle device that switches, if a running speed of an own vehicle becomes low, for example, if it becomes 10 kilometers per hour or less, from a navigation screen to a camera image of a side ahead of the vehicle imaged by a camera mounted on the own vehicle, and displays the camera image.
However, in the on-vehicle devices described above, when the own vehicle approaches the intersection or if the running speed of the own vehicle becomes low, the switching to the camera image is always performed. Therefore, there is a problem that the switching to the camera image is performed even if there is no vehicle approaching the own vehicle.
Besides, in the on-vehicle devices described above, because the switching to the camera image is performed frequently (each time the own vehicle approaches an intersection), this switching may annoy particularly the driver who wants to keep on checking the navigation screen.
Moreover, if the switching to the camera image is frequently performed, then this causes the driver to become less conscious of caution against an approaching vehicle, and thus, there is also a problem that even if the camera image is displayed, the driver neglects checking of the camera image.
Thus, it remains a big challenge how to achieve an on-vehicle device and a recognition support system capable of allowing a driver to reliably recognize the presence of a moving object approaching an own vehicle from a blind corner for the driver while causing the driver to maintain a sense of caution against a dangerous object.
It is an object of the present invention to at least partially solve the problems in the conventional technology.
An on-vehicle device according to one aspect of the present invention is mounted on a vehicle and includes an image acquisition unit that acquires an image obtained by imaging a peripheral image around the vehicle, a moving-object detector that detects whether there is a moving object approaching the vehicle as an own vehicle from the peripheral image based on own-vehicle information indicating running conditions of the own vehicle, a switching unit that switches between images in a plurality of systems input to a display unit, and a switching instruction unit that instructs the switching unit to switch to the peripheral image when the moving object is detected by the moving-object detector.
A recognition support system according to another aspect of the present invention includes an on-vehicle device mounted on a vehicle and a ground server device that performs wireless communication with the on-vehicle device. The ground server device includes a transmission unit that transmits peripheral information around the vehicle to the vehicle. The on-vehicle device includes a reception unit that receives the peripheral information from the ground server device, an image acquisition unit that acquires an image obtained by imaging a peripheral image around the vehicle, a moving-object detector that detects whether there is a moving object approaching the vehicle as an own vehicle from the peripheral image based on the peripheral information received by the reception unit and own-vehicle information indicating running conditions of the own vehicle, a switching unit that switches between images in a plurality of systems input to a display unit, and a switching instruction unit that instructs the switching unit to switch to the peripheral image when the moving object is detected by the moving-object detector.
The above and other objects, features, advantages and technical and industrial significance of this invention will be better understood by reading the following detailed description of presently preferred embodiments of the invention, when considered in connection with the accompanying drawings.
Preferred embodiments of the on-vehicle device and the recognition support system according to the present invention will be explained in detail below with reference to the accompanying drawings. In the following, an overview of the on-vehicle device and the recognition support system according to the present invention will be explained with reference to FIGS. 1A to 1C-2, and then the embodiments of the on-vehicle device and the recognition support system according to the present invention will be explained with reference to
First, the overview of the on-vehicle device and the recognition support system according to the present invention will be explained with reference to FIGS. 1A to 1C-2.
As shown in
Then, only when the determined risk of collision satisfies predetermined conditions, the on-vehicle device and the recognition support system according to the present invention switch from a screen displayed on a display provided in the own vehicle or from a navigation screen to a camera image, and display the camera image thereon.
More specifically, the on-vehicle device and the recognition support system according to the present invention are mainly characterized in that only when the risk of collision is high, switching to the camera image is performed, which makes it possible to reduce a switching frequency and perform recognition support for the driver while causing the driver to maintain a sense of caution against a dangerous object.
The characteristic points will be specifically explained below. As shown in
For example, as shown in
Here, as shown in
Then, the on-vehicle device according to the present invention detects whether there is a moving object based on the image imaged by the super-wide angle camera (hereinafter described as “camera image”). The on-vehicle device according to the present invention also acquires own vehicle information including a running speed, a running direction, and a running position of the own vehicle based on information of various sensors, and acquires peripheral information based on information received from various radars mounted on the own vehicle. Here, the peripheral information includes a distance between the own vehicle and the moving object, and a moving direction of the moving object with respect to the own vehicle and a moving speed thereof.
The on-vehicle device according to the present invention determines whether the moving object is approaching the own vehicle based on the own vehicle information and the peripheral information.
Subsequently, the on-vehicle device according to the present invention, when it is determined that the moving object is approaching the own vehicle, predicts how much time is left before the own vehicle and the moving object collide with each other based on the distance between the own vehicle and the moving object and the running speed or the like, and calculates the time as a collision prediction time.
Thereafter, the on-vehicle device according to the present invention, when the calculated collision prediction time is a predetermined threshold value or less, determines that the risk of collision between the own vehicle and the moving object approaching the own vehicle is very high.
Therefore, the on-vehicle device according to the present invention switches from the screen already displayed on the display unit such as a display provided in the on-vehicle device, herein, from the navigation screen (see
Furthermore, as shown in
On the other hand, in
Thus, the on-vehicle device and the recognition support system according to the present invention detect the moving object based on the image imaged by the camera mounted on the own vehicle, and calculate, if the detected moving object is approaching the own vehicle, a collision prediction time indicating how much time is left before the own vehicle and the moving object collide with each other.
Then, when the calculated collision prediction time is the predetermined threshold value or less, the on-vehicle device and the recognition support system according to the present invention switch from a screen already displayed on the display unit to a camera image, and display the camera image thereon.
Therefore, according to the on-vehicle device and the recognition support system of the present invention, it is possible to allow the driver to reliably recognize the presence of the moving object that is approaching the own vehicle from the blind corner for the driver while causing the driver to maintain a sense of caution against the dangerous object.
An example of the on-vehicle device and the recognition support system whose overview has been explained with reference to FIGS. 1A to 1C-2 will be explained in detail below. First, a configuration of an on-vehicle device 10 according to the present embodiment will be explained below with reference to
As shown in
The camera 11 can image a peripheral image around the own vehicle. For example, the super-wide angle camera can capture an image in a wide field of view (here, 190 degrees) through a special-purpose lens with short focal length or the like. The camera 11 is mounted on the front of the own vehicle, and captures frontward, leftward, and rightward images of the vehicle. The present embodiment explains the case where the camera 11 is mounted on the front of the vehicle, however, the camera 11 may be mounted on the rear side, the left side, or the right side of the vehicle.
Here, a mounting pattern of the camera 11 will be explained below with reference to
As shown in
Referring back to the explanation of
The own-vehicle information acquisition unit 12 acquires own-vehicle information including a running speed, a running direction, and a running position of the own vehicle. More specifically, the own-vehicle information acquisition unit 12 acquires angle information detected by the gyro sensor, and acquires the running direction of the own vehicle based on to which direction a steering wheel of the own vehicle is directed detected by the rudder angle sensor. In addition, the own-vehicle information acquisition unit 12 acquires the running position of the own vehicle through information received from the GPS receiver, and acquires the running speed of the own vehicle through information from the speed sensor. The own-vehicle information acquisition unit 12 also performs the process of transferring the acquired own-vehicle information to the moving-object detector 15b.
The storage unit 13 is configured with storage devices such as a nonvolatile memory and a hard disk drive. The storage unit 13 stores therein a mounting position of the camera 11 (see
The display 14 is a display device that displays an image imaged by the camera 11 and displays an image received from any device other than the on-vehicle device 10. Here, the display 14 receives a navigation image 20 indicating a road map and a route to a destination from a car navigation device and displays the received image. However, the display 14 may receive an image from a DVD (Digital Versatile Disk) player or the like and display the received image. Although the car navigation device and the DVD player are provided separately from the on-vehicle device 10, they may be integrated into the on-vehicle device 10.
The control unit 15 controls the entire on-vehicle device 10. The image acquisition unit 15a is a processor that performs a process of acquiring an image imaged by the camera 11 (hereinafter described as “camera image”). The image acquisition unit 15a also performs a process of transferring the acquired camera image to the moving-object detector 15b.
The moving-object detector 15b is a processor that detects a moving object approaching the own vehicle by calculating optical flows based on the camera image and that sets the degree of risk based on the risk information 13b.
Here, the specific moving-object detection process executed by the moving-object detector 15b and the risk information 13b will be explained with reference to
First, the moving-object detector 15b detects feature points from the image at the time t. Here, four points indicated by dashed line circles are detected as the feature points. Subsequently, the moving-object detector 15b detects feature points from the image at the time t′. Here, four points indicated by solid line circles are detected as the feature points. Then, the moving-object detector 15b detects vectors from the feature points at the time t to the feature points at the time t′ as optical flows respectively.
In order to specify directions of the optical flows relative to the own vehicle based on the mounting position of the camera 11, the moving-object detector 15b acquires the camera mounting position information 13a to specify the directions of the optical flows with respect to the own vehicle. By subtracting the movement of the own vehicle from the generated optical flows, the moving-object detector 15b can detect the movement vector of the object (hereinafter simply described as “movement vector”). The moving-object detector 15b may detect the moving object by correcting the camera image, the movement vector, or the like using the mounting pattern of the camera 11 explained with reference to
The moving-object detector 15b then detects whether there is a moving object approaching the own vehicle based on the detected movement vector. If the length of the movement vector is longer than 0, then the moving-object detector 15b recognizes the object as being moving and thus determines the object as a moving object.
The moving object is detected based on the length of the movement vector as 0, however, the moving object may be detected using a predetermined threshold value as reference. Furthermore, there is no need to use all the detected feature points for a predetermined object. As shown in
The moving-object detector 15b detects the moving object by calculating the optical flows, however, may detect the moving object using a pattern matching method or a clustering method.
Subsequently, the risk information 13b used when the moving-object detector 15b executes the risk setting process will be explained below with reference to
Each of
First, a case where the mounting position of the camera 11 is on the front side will be explained. In
Subsequently, optical flows in directions different from these in
In
Subsequently, a case where the mounting position of the camera 11 is not on the front side will be explained below. For example, if the image of
If the image of
Meanwhile, if the image of
In this manner, the risk information 13b includes degrees of risks which are preset in association with the situations. The risk information 13b is not limited thereto, and thus the degrees of risks are set in association with various situations. In addition, the explanation is made so that each situation is set as a high degree of risk or a low degree of risk, however, the degrees of risks may be subdivided and set according to each degree of risk.
The moving-object detector 15b acquires the degree of risk in association with the situation stored in the risk information 13b based on the directions of the optical flows of the detected moving object, the mounting position of the camera 11, and the own-vehicle information, and sets the acquired degree of risk as a degree of risk of the moving object.
In this manner, the moving-object detector 15b detects the moving object approaching the own vehicle based on the optical flows, and sets the degree of risk with respect to the detected moving object based on the risk information 13b.
Referring back to the explanation of
The switching display unit 15d is a processor that performs the process of switching to the camera image acquired by the image acquisition unit 15a and displaying the camera image on the display 14 when it is determined by the switching determination unit 15c that the switching is performed from the navigation image 20 to the camera image.
Meanwhile, when it is determined by the switching determination unit 15c that the switching is not performed from the navigation image 20 to the camera image, the switching display unit 15d continuously acquires the navigation image 20 and displays it on the display 14.
The switching display unit 15d may highlight the moving object to be displayed on the display 14. For example, the switching display unit 15d may superimpose the moving object on the camera image and display the speed of the moving object and the distance between the moving object and the own vehicle.
The switching display unit 15d may also highlight the moving object according to the degree of risk of the moving object set by the moving-object detector 15b, to be displayed on the display 14. For example, the switching display unit 15d may display an enclosing frame around the moving object with a high degree of risk, may blink the enclosing frame or the entire image, or may change the color for display. Furthermore, the switching display unit 15d may emit alarm sound and vibrate a seat belt based on the degree of risk to inform the driver of the risk.
The switching display unit 15d also performs processes for switching to a camera image, displaying the camera image on the display 14, and after a predetermined time passes, returning to the image previous to the switching (here, navigation image 20).
In the above, the display is returned to the navigation image 20 after the passage of the predetermined time. The present invention, however, is not limited thereto. For example, the switching display unit 15d may return the display to the navigation image 20 in response to detection that an accelerator of the own vehicle is in an on-state, or may continuously display the camera image during detection of a moving object approaching the own vehicle even if the acceleration becomes on.
Next, the processes executed by the on-vehicle device 10 and the recognition support system according to the present embodiment will be explained below with reference to
As shown in
Furthermore, the moving-object detector 15b acquires the camera mounting position information 13a and the risk information 13b stored in the storage unit 13 (Step S103). Then, the moving-object detector 15b detects whether there is a moving object based on the camera image acquired at Step S101, the own-vehicle information acquired at Step S102, and the camera mounting position information 13a and the risk information 13b acquired at Step S103 (Step S104).
The switching determination unit 15c determines whether the moving-object detector 15b has detected the moving object (Step S105), and determines, when the moving object has been detected (Yes at Step S105), whether the detected moving object is approaching the own vehicle (Step S106).
Then, when it is determined that the detected moving object is approaching the own vehicle (Yes at Step S106), then the switching display unit 15d switches to a camera image and displays the camera image on the display (Step S107), and ends the recognition support process executed by the on-vehicle device 10.
Meanwhile, when it is determined that the detected moving object is not approaching the own vehicle (No at Step S106), then the switching display unit 15d does not switch to the camera image, but displays the navigation image 20 as it is (Step S108), and ends the process.
Furthermore, when the moving object is not detected (No at Step S105), the switching display unit 15d does not also switch to the camera image and displays the navigation image 20 as it is (Step S108), and ends the process.
Incidentally, the present embodiment has explained the case where it is determined whether the display is to be switched to the camera image, based on the camera image, the own-vehicle information, and also based on the camera mounting position information 13a and the risk information 13b. However, the present invention is not limited thereto. Therefore, there will be explained below, with reference to
First, a configuration of a recognition support system according to the modification will be explained below with reference to 7.
As shown in
The ground system 30 is a system that detects a vehicle running along a road by various sensors such as infrastructure sensors installed on the road, and manages information for road conditions such as congestion on the road and an accident. The ground system 30 also has a function for performing wireless communication with the on-vehicle device 10′ and transmitting the information for the road conditions to the on-vehicle device 10′.
The communication I/F 16 of the on-vehicle device 10′ is configured with communication devices for data transmission/reception through wireless communication with the ground system 30. For example, the on-vehicle device 10′ receives congestion situation of the road from the ground system 30 through the communication I/F 16.
The radar group 17 is a group of radar devices such as a millimeter-wave radar and a laser radar, that transmits an electromagnetic wave to an object and measures a reflective wave of the object to thereby acquire a distance to the object and a direction thereof. The radar group 17 acquires peripheral information around the own vehicle including a distance between the own vehicle and the moving object, an approaching direction of the moving object with respect to the own vehicle, and a moving speed of the moving object. The radar group 17 also performs a process of transferring the acquired peripheral information to the peripheral-information acquisition unit 15e. It should be noted that the radar group 17 may be configured with a single radar device.
The peripheral-information acquisition unit 15e is a processor that performs a process of acquiring the peripheral information around the own vehicle from the ground system 30 and the radar group 17. The peripheral-information acquisition unit 15e also performs a process of transferring the acquired peripheral information around the own vehicle to the collision prediction time calculator 15f.
The collision prediction time calculator 15f is a processor that performs processes of predicting how much time is left before the own vehicle and the moving object collide with each other based on the own-vehicle information received from the moving-object detector 15b and the peripheral information acquired by the peripheral-information acquisition unit 15e, and of calculating the time as a collision prediction time.
A switching determination unit 15c′ is a processor that performs a process of determining whether the display is to be switched from the navigation image 20 to the camera image by comparing the collision prediction time calculated by the collision prediction time calculator 15f with the threshold value 13c.
More specifically, the switching determination unit 15c′, when the collision prediction time is the threshold value 13c or less, recognizes that the risk of collision between the own vehicle and the moving object approaching the own vehicle is very high and determines that the display is switched to the camera image.
The threshold value 13c used for determination on switching may be varied by the switching determination unit 15c′ based on the peripheral information acquired by the peripheral-information acquisition unit 15e. Here, a method of varying the threshold value 13c executed by the switching determination unit 15c′ will be explained below with reference to
However, when the information indicating that the intersection where the own vehicle enters is congested is obtained through the peripheral information acquired by the peripheral-information acquisition unit 15e, the threshold value 13c for the determination on switching may be increased. For example, the switching determination unit 15c″ may vary the threshold value 13c from 3.6 seconds to 5.0 seconds and perform the determination on switching.
When the information indicating that the intersection where the own vehicle enters is an “intersection where accidents occur frequently” is obtained from the peripheral information acquired by the peripheral-information acquisition unit 15e, the threshold value 13c for determination on switching due to the variation may be prolonged from 3.6 seconds to 6.0 seconds. By varying the threshold value 13c depending on the situation in this manner, safety can be secured.
Referring back to the explanation of
It should be noted that the switching display unit 15d′ may highlight the moving object according to the collision prediction time calculated by the collision prediction time calculator 15f to be displayed on the display 14. For example, the switching display unit 15d′ may display an enclosing frame around the moving object of which collision prediction time is very short, blink the enclosing frame or the entire image, or change the color for display.
Moreover, the switching display unit 15d′ may emit alarm sound or vibrate a seat belt according to the collision prediction time to inform the driver of the risk.
In this manner, in the present modification, determination accuracy is improved by determining whether switching to the camera image is performed according to the collision prediction time calculated based on the peripheral information acquired from the ground system 30 and the radar group 17 in addition to the determination on the switching executed in the embodiment. This allows the switching frequency of the image to be reduced and recognition support for the driver to be performed while causing the driver to maintain a sense of caution against a dangerous object.
Next, processes executed by the on-vehicle device 10′ and the recognition support system according to the modification will be explained below with reference to
At Step S206, the peripheral-information acquisition unit 15e, when the moving object is detected by the moving-object detector 15b (Yes at Step S205), acquires peripheral information (Step S206).
Then, the collision prediction time calculator 15f calculates a collision prediction time based on the own-vehicle information and the peripheral information (Step S207), and the switching determination unit 15c′ determines whether the collision prediction time is shorter than the threshold value 13c (Step S208).
Subsequently, the switching display unit 15d′, when the collision prediction time is shorter than the threshold value 13c (Yes at Step S208), switches from the navigation image 20 to a camera image (Step S209), and ends the recognition support process executed by the on-vehicle device 10′.
Meanwhile, the switching display unit 15d′, when it is determined that the collision prediction time exceeds the threshold value 13c (No at Step S208), does not switch to the camera image of the moving object but displays the navigation image 20 as it is (Step S210), and ends the process.
As explained above, in the on-vehicle device and the recognition support system according to the present embodiment and the present modification, the on-vehicle device is configured so that the camera acquires an image obtained by imaging a peripheral image around a vehicle, the moving-object detector detects whether there is a moving object approaching the vehicle as an own vehicle from the peripheral image based on own-vehicle information indicating running conditions of the own vehicle, the switching display unit switches between images in a plurality of systems input to the display unit, and when the moving object is detected by the moving-object detector, the switching determination unit instructs the switching display unit to switch to the peripheral image. Therefore, it is possible to allow the driver to reliably recognize the presence of a moving object approaching the own vehicle from a blind corner for the driver while causing the driver to maintain a sense of caution against a dangerous object.
In the embodiment and the modification, when it is determined that the risk is high, the display is performed by switching from the navigation image to the camera image of the moving object, however, the display may be preformed by superimposing the camera image within the navigation image.
Furthermore, the embodiment and the modification have explained the example of performing display by switching a display screen like the navigation image which is not the camera image to the camera image. However, when a power supply for the display is off, the power supply for the display may be turned on to display the camera image.
As explained above, the on-vehicle device and the recognition support system according to the present invention are useful to cause the driver to maintain the sense of caution against a dangerous object, and are particularly suitable for the case where it is desired to allow the driver to surely recognize the presence of a moving object approaching the own vehicle from a blind corner for the driver.
Although the invention has been described with respect to specific embodiments for a complete and clear disclosure, the appended claims are not to be thus limited but are to be construed as embodying all modifications and alternative constructions that may occur to one skilled in the art that fairly fall within the basic teaching herein set forth.
Number | Date | Country | Kind |
---|---|---|---|
2009-272860 | Nov 2009 | JP | national |