Patent Grant
10744942
The present invention relates to a vehicle peripheral display device provided with an image acquisition unit for acquiring a peripheral image captured from a vehicle by an image pickup device provided to the vehicle, and a display device for displaying the peripheral image, the display device being provided inside the vehicle cabin.
In conventional techniques in this type of vehicle peripheral display device, indicator lines serving as a guide for driving are displayed over a peripheral image captured from a vehicle by an image pickup device, and a driver is thereby assisted in driving. For example, Patent Document 1 discloses a technique whereby an indicator line indicating a predicted trajectory of travel to a target parking position is superimposedly displayed on the basis of such factors as the driver's steering angle during parking of a vehicle.
[Patent Document 1] Japanese Laid-open Patent Application No. 11-334470
In the vehicle peripheral display device described above, indicator lines are displayed two-dimensionally on a peripheral image of the vehicle which has three-dimensional depth. Therefore, although the indicator lines should lie on the plane of the road pictured in the peripheral image, the positional relationship between such a peripheral image and the indicator lines is difficult to recognize. It can also be difficult to distinguish between lines drawn on the road and the indicator lines, for example, and the indicator lines are sometimes difficult to recognize intuitively.
The present invention was developed in view of the foregoing problems, and an object of the present invention is to provide a peripheral display device whereby indicator lines are easily recognized.
A first aspect of the present invention comprises an image acquisition unit for acquiring a peripheral image of a vehicle, the peripheral image being captured by an image pickup device provided to the vehicle; a display device for displaying the peripheral image, the display device being provided inside a vehicle cabin; and an indicator line output unit for causing an indicator line serving as a guide for a driver during a driving operation to be displayed superimposed on the peripheral image, at least one of a shadow portion and a side surface portion adjacent to the indicator line being added to the indicator line.
According to this aspect, when the shadow portion is provided, a visual effect can be created in which the indicator line appears to float over the vehicle peripheral image, which is endowed with three-dimensional depth by the shadow portion. In other words, the visual effect created is as if an indicator line actually placed on a road surface or the like pictured in the peripheral image is seen by a vehicle occupant from the driver seat. As a result, it is possible to ensure that the driver comprehends the indicator line, and the indicator line can be appropriately presented to the driver.
When the side surface portion is provided, since the indicator line superimposed on the peripheral image is displayed in three-dimensional fashion by the side surface portion, the indicator line is displayed three-dimensionally on the vehicle peripheral image, which has three-dimensional depth, and the positional relationship between the peripheral image and the indicator line can easily be recognized. For color-blind drivers, since the side surface portion is added to the displayed indicator line, the color surface area of the indicator line increases, and the indicator line is easier to recognize. On the other hand, for drivers who are not color blind, the three-dimensional display of the indicator line prevents the thickness of the line from being emphasized. As a result, the vehicle occupant can clearly recognize the positional relationship between the peripheral image and the indicator line.
When both the shadow portion and the side surface portion are provided, the indicator line is displayed even more three-dimensionally, and the positional relationship between the peripheral image and the indicator line is easily recognized.
According to a second aspect of the present invention, the indicator line output unit adds and displays the shadow portion adjacent to the side surface portion.
According to this aspect, since the shadow portion is displayed adjacent to the side surface portion, the display on the display device is more organized, and the positional relationship between the peripheral image and the indicator line is easier to recognize than in a case in which the side surface portion and the shadow portion are separated.
According to a third aspect of the present invention, the indicator line output unit adds and displays the side surface portion on the side of the indicator line close to the image pickup device.
According to this aspect, it is possible to create a visual effect which is as if a three-dimensional indicator line actually placed on a road surface or the like pictured in the peripheral image is seen by a vehicle occupant from the driver seat. As a result, the vehicle occupant can more reliably recognize the positional relationship between the peripheral image and the indicator line. The peripheral image and the indicator line can also be even more clearly distinguished.
According to a fourth aspect of the present invention, the indicator line output unit adds and displays the shadow portion on the side of the indicator line close to the image pickup device.
According to this aspect, a visual effect is created in which the indicator line appears to float over the peripheral image, and a visual effect can be created which is as if an indicator line actually placed on a road surface or the like pictured in the peripheral image is seen by a vehicle occupant from the driver seat.
According to a fifth aspect of the present invention, the indicator line is a broken line.
According to this aspect, the region of the peripheral image that is covered by the indicator line can be reduced in size. The vehicle occupant can therefore accurately comprehend the peripheral image.
According to a sixth aspect of the present invention, the indicator line is intermittently arranged and drawn as an indicator having an arbitrary shape.
According to this aspect, it is possible to clearly distinguish between the indicator line and lines which are original to the peripheral image, such as white lines on a road or parking space lines, for example.
According to a seventh aspect of the present invention, the indicator line output unit displays the indicator line and the side surface portion or shadow portion added to the indicator line so as to have a smaller width the greater the distance is from the image pickup device.
According to this aspect, perspective can be imparted to the indicator line, and the vehicle occupant can more reliably comprehend the positional relationship of the indicator line in the peripheral image.
According to an eighth aspect of the present invention, the indicator line is a parking frame line for indicating a parking space.
According to this aspect, a parking space can easily be recognized on the screen of the display device, and the operation of parking the vehicle can be made easier for the vehicle occupant during parking.
According to a ninth aspect of the present invention, the indicator line is an predicted trajectory line which extends in the direction of travel of the vehicle and indicates the predicted trajectory of movement of the vehicle in accordance with a steering angle.
According to this aspect, the predicted trajectory line is easily recognized on the screen of the display device.
According to a tenth aspect of the present invention, the indicator line is an extension from the vehicle width, extending in the direction of travel of the vehicle.
According to this aspect, the extension line of the vehicle width is easily recognized on the screen of the display device. The “vehicle width” referred to herein is the maximum width of the vehicle in the left-right direction. However, for safety purposes, this vehicle width may be set to a value larger than the actual maximum width.
According to an eleventh aspect of the present invention, the indicator line is a distance reference line as a reference of forward distance from a front end of the vehicle or rearward distance from a rear end of the vehicle.
According to this aspect, the distance reference line is easily recognized on the screen of the display device.
Following is a description of an example in which the peripheral display device of the present invention is applied to a parking assistance device for assisting in the operation of parking a vehicle.
A monitor 20 (display device) in which a touch panel 23 is formed in a display portion 21 is provided at the top of a console in the vicinity of the driver seat. In the present embodiment, the monitor 20 is a liquid crystal monitor provided with a backlight. The monitor 20 is also provided with a speaker 22. A pressure-sensitive panel or electrostatic panel is used as the touch panel 23, and the position of touch by a finger or the like is outputted as location data. As described hereinafter, the touch panel 23 of the monitor 20 is used as an instruction inputting means at the start of parking assistance in the present embodiment. In a case in which the vehicle 30 is equipped with a navigation system, the monitor 20 is preferably also used as the display device of the navigation system.
The monitor 20 may also be a plasma display or CRT display, and the speaker 22 may be provided in another location, such as inside a door. When another switch or the like is provided as instruction inputting means at the start of parking assistance, the touch panel 23 of the monitor 20 need not necessarily be used as the instruction inputting means of the parking assistance device.
A steering wheel sensor 14 for measuring a steering direction and amount is provided to the operating system of the steering wheel 24. A shift position sensor 15 for determining the shift position is provided to the operating system of a shift lever 25. An accelerator sensor 16 for measuring the accelerator position is provided to the operating system of the accelerator pedal 26. A brake sensor 17 for detecting whether the brake is being operated is provided to the operating system of the brake pedal 27.
Rotation sensors 18 for measuring the rotation amount of at least one of the front wheels 28f and the rear wheels 28r are provided as movement distance sensors. In the present embodiment, a case is described in which rotation sensors 18 are provided to the rear wheels 28r. A configuration may also be adopted in which the movement distance is measured by measuring the amount of movement of the vehicle 30 from the amount of rotation of the drive system in the speed change mechanism 34. An ECU (electronic control unit) 10 for travel control including parking assistance is provided in the vehicle 30 as the central core of the parking assistance device of the present invention.
A camera 12 for capturing an image of the scene to the rear of the vehicle 30 is provided at the rear of the vehicle 30. The camera 12 houses a CCD (charge coupled device), CIS (CMOS image sensor) or other imaging element, and is a digital camera for outputting information imaged by the imaging element as video information in real time. The camera 12 is provided with a wide-angle lens, and has a field of view of approximately 140 degrees to the left and right, for example. The camera 12 is set so as to have a depression angle of about 30 degrees, for example, toward the rear of the vehicle 30, and captures an image of a region extending about 8 m generally to the rear. The captured image is inputted to the ECU 10 and utilized for such purposes as parking assistance.
The image acquisition unit 2 is a functional unit for acquiring a peripheral image captured by the camera 12 (image pickup device) from the vehicle 30. The image acquisition unit 2 has frame memory for image storage, a synchronizing separator circuit, and other components. The display controller 3 is a functional unit for displaying the peripheral image captured by the camera 12 from the vehicle 30 on the monitor 20 (display device) inside the vehicle cabin.
The computation unit 1 sets the parking target position P3 (see
As shown in
The operation of the parking assistance device will next be described using as an example a case of parallel parking in a parking space E between two parked vehicles 41, 42, as shown in
The driver advances the vehicle 30 from position P1 to position P2 at which the foreground of the parking space E appears on the monitor 20, and once the driver stops the vehicle, a speed sensor 7b detects that the vehicle 30 is stopped and transmits this detection result to the computation unit 1 of the parking assistance device. When the driver sets the shift lever to the reverse position, the shift position sensor 15 detects that the shift lever is set to reverse and transmits this detection result to the computation unit 1. When the two detection results mentioned above are transmitted to the computation unit 1, the parking target setting unit 4 sets the parking target position P3 on the basis of the peripheral image. The method for setting the parking target position will not be described in detail, but the parking target position is set, for example, through image recognition of the partition lines provided to the parking space E, or of obstacles such as the parked vehicles 41, 42 in front and to the rear.
When the parking target position P3 is set, the guidance unit 5 calculates a guidance path on the basis of the current position P2 of the vehicle and the parking target position P3 set by the parking target setting unit 4. As shown in
When there is no need to change the parking target position P3 first set, the parking target position P3 is confirmed when the vehicle occupant touches the “OK” display. When the vehicle occupant wishes to change the set parking target position P3, the vehicle occupant can change the parking target position P3 by operating the touch panel, for example. The parking target position P3 may also be changed by operating the steering wheel.
Once the parking target position P3 is confirmed, the guidance unit 5 detects that the shift lever is set to reverse and that the vehicle occupant has released the brake, and controls the ECU 10 so as the guide the vehicle 30 toward the parking target position P3. A configuration may also be adopted in which the display “Cancel parking assistance” is provided on the screen of the monitor 20, and the driver can cancel parking assistance as needed by touching the display. A configuration may also be adopted in which parking assistance is cancelled when the vehicle occupant operates the brake, the steering wheel, or another component, for example. When the ECU 10 reaches the parking target position, the ECU 10 is controlled so as to stop the vehicle 30 and end parking assistance.
(1) In the first embodiment, an example is described in which the vehicle 30 is guided by automatic steering to the set parking target position, but this configuration is not limiting. For example, a configuration may be adopted in which a predicted course line of the vehicle is displayed superimposed as an indicator line on the peripheral image, and the driver parks the vehicle by such operations as operating the steering wheel on the basis of the course prediction line, as shown in
In the present embodiment, the rearward prediction lines 51 are displayed superimposed on the peripheral image via the display controller 3, as shown in
Vehicle rearward lines 55 are also displayed superimposed on the peripheral image. The vehicle rearward lines 55 are indicator lines which indicate a predetermined position to the rear of the vehicle 30 regardless of the steering angle of the vehicle 30. Consequently, this indicator line is displayed superimposed on a predetermined position fixed in the peripheral image according to the optical relationship with the camera 12 mounted to the vehicle 30. The vehicle rearward lines 55 in the present embodiment are composed of vehicle width extension lines 55a which are extensions of the vehicle width, extending to the rear of the vehicle, and a 1 m reference line 56a as a distance reference line. In the present embodiment, the vehicle rearward lines 55 are displayed in green, for example, or another color other than that of the rearward prediction lines 51 in order to be more easily distinguishable from the rearward prediction lines 51. The “vehicle width” herein is the maximum width of the vehicle 30 in the left-right direction, but may be set to a value larger than the actual maximum width for safety purposes.
The indicator line output unit 6 adds profile lines and shadow lines to the rearward prediction lines 51 and vehicle rearward lines 55 in the present embodiment as well. Specifically, profile lines 51b and shadow lines 51c are added on the insides of the pair of left and right predicted trajectory lines 51a of the rearward prediction lines 51, and profile lines 54b, 53b, 52b and shadow lines 54c, 53c, 52c are also added along the sides of the 5 m reference line 54a, 3 m reference line 53a, and 1 m warning line 52a close to the vehicle 30. Profile lines 55b, 56b and shadow lines 55c, 56c are also added on the insides of the pair of left and right vehicle width extension lines 55a and on the vehicle 30 side of the 1 m reference line of the vehicle rearward lines 55. The rearward prediction lines 51 and the vehicle rearward lines 55 are thus three-dimensionally displayed superimposed on the peripheral image.
The driver operates the steering wheel to back the vehicle 30 so that the vehicle 30 does not strike the parked vehicles 41, 42 or other obstacles, and guides the vehicle 30 into the parking space E while referring to the rearward prediction lines 51 and the vehicle rearward lines 55 which are displayed superimposed on the peripheral image. When the driver then confirms that the vehicle 30 is in the parking space E and the vehicle rearward lines 55 are substantially parallel to the parking space E, the driver vehicle 30 stops the vehicle.
In the present embodiment, rearward prediction lines and vehicle rearward lines are described for moving the vehicle 30 in reverse, but this configuration is not limiting. For example, although not shown in the drawings, forward prediction lines and vehicle forward lines may be displayed superimposed on the peripheral image during forward movement of the vehicle 30. In this case, predicted trajectory lines extending in front of the vehicle 30 indicate the predicted trajectory of movement of the vehicle 30 in accordance with the steering angle. Vehicle width extension lines in this case indicate extensions of the vehicle width, and extend forward from the vehicle 30. Furthermore, distance reference lines serve as references of distance forward from the front end of the vehicle 30.
(2) Cases have so far been described in which the indicator lines are solid lines. However, the indicator lines may be broken lines, for example, or other non-solid lines. A combination of solid lines and broken lines, for example, or other non-solid lines may also be used.
In the present embodiment as well, the indicator line output unit 6 adds and displays the profile lines and shadow lines along the sides of the vehicle rearward lines 55 close to the image pickup device (camera 12). In other words, the profile lines 55b, 56b and shadow lines 55c, 56c are added along the vehicle 30 sides of the pair of left and right vehicle width extension lines 55a and the 1 m reference line 56a of the vehicle rearward lines 55. The profile lines 55b and shadow lines 55c are also displayed as broken lines on the broken vehicle width extension lines 55a, and are added in positions corresponding to the broken vehicle width extension lines 55a.
Although not particularly limited, the vehicle width extension lines 55a are displayed in blue, and the 1 m reference line 56a is displayed in red, for example. The profile lines 55b and shadow lines 55c of the vehicle width extension lines 55a are also displayed in blue, but the brightness progressively decreases, i.e., the color becomes closer to black, in the stated order of the vehicle width extension lines 55a, the profile lines 55b, and the shadow lines 55c. The profile lines 56b and shadow lines 56c of the 1 m reference line 56a are also displayed in red, but the brightness progressively decreases in sequence in the 1 m reference line 56a, profile lines 56b, and shadow lines 56c. Of course, this configuration is not limiting, and the lines may be yellow or another color other than described above, or each line may be of a different hue. The vehicle rearward lines 55 are thereby displayed three-dimensionally in superimposed fashion on the peripheral image.
(3) In a case in which all or some of the indicator lines are displayed as broken lines in the manner described above, the indicator lines may be intermittently arranged and drawn as indicators having an arbitrary shape.
(4) Examples have thus far been described in which the indicator line output unit 6 adds side surface portions (profile lines) and shadow portions (shadow lines) to the displayed indicator lines, but both the side surface portions (profile lines) and the shadow portions (shadow lines) need not necessarily be displayed, and a configuration may be adopted in either the side surface portions (profile lines) or the shadow portions (shadow lines) are displayed.
(5) In the first embodiment described above, the indicator line output unit 6 may be configured so that the width of the indicator lines, profile lines, and shadow lines decreases as the distance from the camera 12 increases. In other words, in the case of the first embodiment described above (with reference to
In the case of the second embodiment described above (with reference to
(6) In the embodiments described above, examples are described in which the vehicle peripheral display device is applied to a parking assistance device, but this configuration is not limiting. The vehicle peripheral display device can also be applied in cases in which indicator lines are displayed superimposed on a peripheral image during driving operations other than parking.
The vehicle peripheral display device of the present invention can be applied to such devices as a parking assistance device for assisting a driver with a driving operation, whereby the indicator lines displayed by the display device are easily recognized, and the driver can clearly recognize the positional relationship between the peripheral image and the indicator lines.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2008-122557 | May 2008 | JP | national |
| 2008-276859 | Oct 2008 | JP | national |
This application is a divisional of U.S. application Ser. No. 12/888,157, filed Sep. 22, 2010, which is a divisional of U.S. application Ser. No. 12/933,994, which is a National Stage of International Application No. PCT/JP2009/058193 filed Apr. 24, 2009, claiming priority based on Japanese Patent Application Nos. 2008-122557 filed May 8, 2008 and 2008-276859 filed Oct. 28, 2008, the contents of all of which are incorporated herein by reference in their entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6999602 | Yoshida et al. | Feb 2006 | B2 |
| 7643935 | Sakakibara et al. | Jan 2010 | B2 |
| 8218007 | Lee et al. | Jul 2012 | B2 |
| 10315568 | Kadowaki | Jun 2019 | B2 |
| 20010030688 | Asahi et al. | Oct 2001 | A1 |
| 20020026269 | Tanaka et al. | Feb 2002 | A1 |
| 20020110262 | Iida et al. | Aug 2002 | A1 |
| 20040119823 | Takagi et al. | Jun 2004 | A1 |
| 20040153243 | Shimazaki et al. | Aug 2004 | A1 |
| 20060022810 | Inoue et al. | Feb 2006 | A1 |
| 20060287826 | Shimizu et al. | Dec 2006 | A1 |
| 20070239357 | Mori et al. | Oct 2007 | A1 |
| 20080100614 | Augst | May 2008 | A1 |
| 20110181724 | Kadowaki | Jul 2011 | A1 |
| Number | Date | Country |
|---|---|---|
| 1522892 | Aug 2004 | CN |
| 101167104 | Apr 2008 | CN |
| 1 160 146 | Dec 2001 | EP |
| 64-014700 | Jan 1989 | JP |
| 11-334470 | Dec 1999 | JP |
| 2001-010428 | Jan 2001 | JP |
| 2001-027535 | Jan 2001 | JP |
| 2001-180405 | Jul 2001 | JP |
| 2002-160597 | Jun 2002 | JP |
| 2002-297029 | Oct 2002 | JP |
| 2002-314991 | Oct 2002 | JP |
| 2004-056219 | Feb 2004 | JP |
| 2004-147083 | May 2004 | JP |
| 2004-203126 | Jul 2004 | JP |
| 2004-284370 | Oct 2004 | JP |
| 2004-345554 | Dec 2004 | JP |
| 2005-045602 | Feb 2005 | JP |
| 2006-040008 | Feb 2006 | JP |
| 2006-284458 | Oct 2006 | JP |
| 2007333502 | Dec 2007 | JP |
| 2007333502 | Dec 2007 | JP |
| Entry |
|---|
| Japanese Office Action, dated Mar. 7, 2013, issued in corresponding Japanese Application No. 2008-276859. |
| Japanese Office Action dated Jul. 26, 2012 issued in Japanese Patent Application No. 2008276859. |
| Extended European Search Report issued in the corresponding European Application No. 09742683.7 dated Mar. 27, 2012. |
| Chinese Office Action issued in a corresponding Chinese Application No. 200980110661.7 dated Dec. 15, 2011. |
| International Search Report of PCT/JP2009/058193 dated Jun. 2, 2009. |
| International Preliminary Report on Patentability issued in PCT/JP2009/058193, dated Dec. 23, 2010. |
| Number | Date | Country | |
|---|---|---|---|
| 20190152394 A1 | May 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12888157 | Sep 2010 | US |
| Child | 16253451 | US | |
| Parent | 12933994 | US | |
| Child | 12888157 | US |
