The present invention relates to a driving lane recognizing apparatus mounted on a motor vehicle (hereinafter referred to as the vehicle-onboard driving lane recognizing apparatus) for detecting lane markings to recognize a driving lane.
Heretofore, there have been proposed and developed a great variety of vehicle-onboard driving lane recognizing apparatuses which are designed for detecting lane markings or the like existing in front of a running motor vehicle to recognize a driving lane therefor.
In general, the conventional vehicle-onboard driving lane recognizing apparatus known heretofore is comprised of a CCD (Charge-Coupled Device) camera serving as an image pick-up means (also known as the imaging means) and a recognition processing unit serving as a recognizing means. The CCD camera is installed on, for example, a ceiling portion of the motor vehicle located in the vicinity of a driver seat for picking up image scenes such as landscapes or the like making appearance in front of the motor vehicle in the course of running.
In that case, the imaging range of the CCD camera is oriented or set such that a road extending for a predetermined distance before the motor vehicle can be covered by the landscape or scene picked up when the motor vehicle is running on the road.
On the other hand, the recognition processing unit is comprised of an analog-to-digital converter (ADC), a pre-processing ASIC (Application Specified IC), an image memory, a CPU (Central Processing Unit), a ROM (Read-Only memory), a communication IC (Integrated Circuit) and others.
The analog-to-digital converter serves for converting an analog image signal outputted from the CCD camera into digital image data. The pre-processing ASIC executes a predetermined pre-processing on the image data outputted from the analog-to-digital converter. As such pre-processing, there may be mentioned, by way of example, an edge emphasizing or enhancing filter processing. The image memory is destined for storing the image data resulting from the pre-processing.
The CPU executes a processing for the stored image data to recognize the lane markings contained therein.
The RAM serves as a work area for the CPU, while the communication IC serves to transmit to a relevant external system or equipment the data such as, for example, the result of recognition of the lane markings transferred from the CPU. For more particulars, reference may have to be made to Japanese Patent Application Laid-Open Publication No. 213155/1999 (JP-A-11-213155).
In the conventional vehicle-onboard driving lane recognizing apparatus mentioned above, the analog-to-digital converter, the pre-processing ASIC, the image memory and the RAM are implemented individually as discrete chips, respectively, as a result of which the circuit scale of the vehicle-onboard driving lane recognizing apparatus becomes large and involves high cost in the implementation thereof, giving rise to a problem.
In the light of the state of the art described above, it is contemplated with the present invention as an object thereof to solve the problem mentioned above by providing an inexpensive vehicle-onboard driving lane recognizing apparatus of a small circuit scale.
In view of the above and other objects which will become apparent as the description proceeds, there is provided according to a general aspect of the present invention a vehicle-onboard driving lane recognizing apparatus which includes an image pick-up unit mounted on a motor vehicle running on a road for picking up images of scenes making appearance in front of the motor vehicle, an image data selecting unit for selecting image data only of a predetermined region from the image picked up by the image pick-up unit, an image data storing unit for storing the image data, and a driving lane recognizing unit for detecting lane markings on the road from the image data stored in the image data storing unit to thereby recognize a driving lane extending along the lane markings, wherein at least the image data selecting unit, the image data storing unit and the driving lane recognizing unit are all implemented in a single chip.
By virtue of the arrangement described above, there can be realized inexpensively the vehicle-onboard driving lane recognizing apparatus of a small circuit scale while making it unnecessary to install an external image data storing unit, to advantageous effect.
The above and other objects, features and attendant advantages of the present invention will more easily be understood by reading the following description of the preferred embodiments thereof taken, only by way of example, in conjunction with the accompanying drawings.
In the course of the description which follows, reference is made to the drawings, in which:
The present invention will be described in detail in conjunction with what is presently considered as preferred or typical embodiments thereof by reference to the drawings. In the following description, like reference characters designate like or corresponding parts throughout the several views.
Now, description will be made of the vehicle-onboard driving lane recognizing apparatus according to a first embodiment of the present invention.
The vehicle-onboard driving lane recognizing apparatus is installed on a motor vehicle running on a road. Referring to
An image data storing means 3 is designed to store therein the image data selected by the image data selecting means 2. A driving lane recognizing means 4 is designed to detect lane markings by making use of the image data stored in the image data storing means 3 for thereby recognizing the driving lane extending along the lane markings as detected.
At this juncture, it should be noted that the image data selecting means 2, the image data storing means 3 and the driving lane recognizing means 4 are realized en bloc in a single or unitary chip 5 as indicated by enclosing with a single-dotted broken line.
Incidentally, the image pick-up means 1 is shown as being provided externally of the chip 5. However, it goes without saying that the image pick-up means 1 may also be implemented on the chip 5 similarly to the other components or means mentioned above, if it is possible.
Next, description will be made in detail of the individual functional blocks.
At first, the image pick-up means 1 will be elucidated. The image pick-up means 1 is designed for taking picture or image of scenes making appearance in front of the motor vehicle in the course of running on a road. In general, two types of the image pick-up means are well known in the art. They are a CCD (Charge-Coupled Device) imaging device and a CMOS (Complementary Metal Oxide Semiconductor) imaging device. In this conjunction, it is presumed that the CMOS imaging device incorporating therein an analogue-to-digital converter (ADC) is employed as the image pick-up means in the vehicle-onboard driving lane recognizing apparatus according to the instant embodiment of the invention.
Incidentally, the image pick-up means 1 may be constituted by a CCD serving as the imaging device, a CCD driving IC (Integrated Circuit) and an analogue-to-digital converter (ADC).
Next, description will be directed to the image data selecting means 2.
Since the circuit arrangement of the image data selecting means 2 is self-explanatory from
A horizontal counter 11 designed for counting the number of pixels in the horizontal direction increments or counts up a count value upon every inputting of the dot clock signals DCK. The count value of the horizontal counter 11 is cleared once when the horizontal synchronizing signal HD is inputted.
The count value outputted from the horizontal counter 11 is compared by a comparator 14a with a horizontal start address (start address in the horizontal direction) preset at a horizontal start address hold module 12. When the count value becomes greater than the horizontal start address, the output of the comparator 14a changes from a level “L (low level)” to a level “H (high level)”.
Further, the count value outputted from the horizontal counter 11 is compared by a comparator 14b with a horizontal end address (end address in the horizontal direction) placed in a horizontal end address hold module 13. So long as the count value remains smaller than the horizontal end address, the output of the comparator 14b assumes the level “H”, whereas when the count value becomes greater than the horizontal end address, the output of the comparator 14b changes to the level “L” from “H”.
A logical AND (logical product) circuit 15a determines a logical product of the outputs of the comparator 14a and the comparator 14b. The output signal of the logical AND circuit 15a serves as the select signal in the horizontal direction. Thus, in the horizontal direction, the pixel data located from the horizontal start address to the horizontal end address are selected from the image data picked up by the image pick-up means 1.
Similarly, in the vertical direction, a vertical counter 16 designed for counting the pixel number in the vertical direction increments the count value thereof in response to inputting of the horizontal synchronizing signal HD. The count value of the vertical counter 16 is once cleared when the vertical synchronizing signal VD is inputted.
The count value outputted from the vertical counter 16 is compared by a comparator 14c with a vertical start address (start address in the vertical direction) preset at a vertical start address hold module 17. When the count value becomes greater than the vertical start address, the output of the comparator 14c changes from a level “L” to a level “H” to be outputted.
Further, the count value outputted from the vertical counter 16 is compared by a comparator 14d with a vertical end address (end address in the vertical direction) placed in a vertical end address hold module 18. So long as the count value remains smaller than the vertical end address, the output of the comparator 14d assumes the level “H”, whereas when the count value becomes greater than the vertical end address, the output of the comparator 14d changes to the level “L” from “H”.
A logical AND circuit 15b determines a logical product of the outputs of the comparator 14c and the comparator 14d. The output signal of the logical AND circuit 15b serves as the select signal in the vertical direction. In other words, in the vertical direction, the pixel data located from the vertical start address to the vertical end address are selected from the image data picked up by the image pick-up means 1.
Finally, the output of the logical AND circuit 15a representing the select signal in the horizontal direction and the output of the logical AND circuit 15b representing the select signal in the vertical direction are logically ANDed by a logical AND circuit 15c, whereby the image data select signal for one rectangular select range is generated.
By making use of the generated select signal, the image data is selected to be subsequently stored in the image data storing means 3 of the succeeding stage.
Incidentally, the image data selecting means shown in
Further, although it has been described that the image data selecting means shown in
In this conjunction, it should further be added that the image data selecting means 2, the image data storing means 3, and the driving lane recognizing means 4 may be realized by a microcomputer, as will be described hereinafter. In that case, it is preferred to realize the image selection by making use of the output signal of the timer, because then the circuit scale can be made small with the cost being reduced.
Besides, by selecting the image by means of software incorporated in the microcomputer, it is possible to change dynamically the select range of the image data in dependence on the recognized state of the driving lane.
Additionally, the whole image may be selected by designating the start address and the end address in both the horizontal and vertical directions, as the case may be.
Next, referring to
According to a method adopted generally for recognizing the driving lane, search zones are set for detecting lane markings 21 painted on a road. In
In this conjunction, it is noted that although a single search region or zone is defined by a rectangular region of (M×N) pixels with M pixels in the horizontal direction and N pixels in the vertical direction, where M and N represent given natural number, respectively, the search zone may be derived as a search line with the pixel number N in the vertical direction being one.
As shown in
The image data of the individual search zones or regions selected by the image data selecting means 2 are stored in the image data storing means 3.
The driving lane recognizing means 4 detects the lane markings by processing the image data stored in the image data storing means 3 to thereby recognize a driving lane extending along the lane markings.
Next, description will be directed to the recognition of the driving lane as carried out by the driving lane recognizing means 4. As described previously, the position at which the lane marking exists in the selected search zone or region is detected. As the lane marking detecting method, there may be mentioned a conventional filter processings such as template matching or the like.
For effectuating the template matching, a template which exhibits a luminance distribution similar to that of the object to be detected is prepared, and the one exhibiting the highest correlation with the template is selected as the object concerned.
By way of example, presuming that the search zone shown in
On the basis of the positions of the lane markings detected in the individual search zones, respectively, the parameter representing the driving lane (road geometry) which can be represented by a polynomial or the like can be identified to be the result of recognition of the driving lane.
As described previously, by selectively setting the individual search zones such that they are disposed substantially with equidistance in the traveling direction, as viewed from the top, the detected positions of the lane markings will also lie with equidistance therealong. Thus, the parameter of the driving lane expressed by the polynomial or the like can be identified without any difficulty.
Next, description will be made of the single-dotted line block 5 shown in
In recent years, the capacity of the RAM incorporated integrally in the microcomputer tends to increase. However, the microcomputer incorporating the RAM of a large capacity capable of storing all the image data of, for example, 640×480 pixels, which is termed generally VGA (video graphics array) according to the video standards for PC, while allowing the program to run, is extremely expensive.
Such being the circumstances, according to the teachings of the present invention incarnated in the instant embodiment thereof, it is taught that the image data selecting means 2, the image data storing means 3 and the driving lane recognizing means 4 at the least are realized in one and the same chip, wherein only the image data of the specific region required for the recognition of the driving lane is selected by the image data selecting means 2 to be subsequently transferred to the image data storing means 3 (i.e., the so-called built-in RAM). By virtue of this arrangement, the built-in RAM can be of a small capacity and thus the chip can be realized significantly at very low cost. In other words, according to the teaching of the present invention incarnated in the instant embodiment, the vehicle-onboard driving lane recognizing apparatus of small circuit scale and low cost can be realized advantageously for practical applications.
At this juncture, it should also be added that so far as all the image data required for the recognition of the driving lane can be stored in the built-in RAM, there arises no necessity of providing an extraneous RAM externally of the chip to another advantage.
Furthermore, in the conventional vehicle-onboard image processing system, the image data is not intactly stored but a primitive processing called a so-called pre-processing such as typified by a simple filtering processing of the image data is performed by employing a dedicated IC such as an ASIC (application-specific integrated circuit) or the like. By contrast, in the case of the vehicle-onboard driving lane recognizing apparatus according to the instant embodiment of the invention, all the image data 1 are saved in the RAM incorporated in the chip 5 or the microcomputer chip. Accordingly, the pre-processing mentioned above can wholly be carried out softwarewise by taking advantage of high-seed access capability.
As is obvious from the above, extraneous external RAM and the dedicated IC such as application-specific integrated circuit can be spared, whereby the size or scale as well as the cost of the apparatus can remarkably be reduced.
In the vehicle-onboard driving lane recognizing apparatus according to the first embodiment of the present invention described above, no consideration has been paid to the transfer rate of the image data. According to the teaching of the invention incarnated in a second embodiment, there is adopted an arrangement for converting the transfer rate of the data derived from the output of the image pick-up means 1.
Referring to
At this juncture, it should be mentioned that the single-dotted broken line block 5 may be implemented as a single chip. In that case, only the image pick-up means 1 is provided externally of the chip 5. It goes however without saying that in case the image pick-up means 1 can be incorporated in the chip 5, it may be built in the chip similarly to the other means or components. Furthermore, although the block 5 may be constituted by a dedicated IC for the image processing, it is presumed that in the vehicle-onboard driving lane recognizing apparatus according to the instant embodiment of the invention, the block 5 is constituted by a microcomputer which includes as the indispensable components a CPU, a RAM and a timer at the least.
Description will now be directed to the operation of the pixel data transfer rate converting means 31. In the case where the output rate of the pixel data is lower than the transfer rate inherent to the microcomputer, it is possible to input the image data delivered from the image pick-up means intactly to the RAM incorporated in the microcomputer. On the other hand, when the output rate of the pixel data is higher than the transfer rate inherent to the microcomputer, the image data can be fetched by the microcomputer by lowering the transfer rate of the pixel data by adopting the arrangement shown in
More specifically, by making use of a dot clock signal which changes by one cycle every time the pixel data of one pixel (picture element) is outputted from the image pick-up means, the pixel data can be transferred to the built-in RAM of the microcomputer at the transfer rate corresponding to 1/M of the output rate of one pixel data, where M represents a given natural number.
The pixel data outputted sequentially from the image pick-up means 1 are supplied to input ports D[7 . . . 0] of D-flip-flops (DFFs) 36 and 37 to be stored in the D-flip-flops 36 and 37 under the timing of the dot clock signal DCK which changes by one cycle every time one pixel of the pixel data is outputted.
In this conjunction, it is to be noted that the pixel data outputted from the output port Q[7 . . . 0] of the D-flip-flop 37 temporally proceeds by one pixel relative to the pixel data outputted from the output port Q[7 . . . 0] of the D-flip-flop 36.
On the other hand, the frequency of the dot clock signal DCK is lowered to a half by means of a frequency by-two-division circuit which may be realized by a T-flip-flop (TFF) 38. Relation between the original dot clock signal DCK and the transfer clock whose frequency is equal to a quotient resulting from division of the dot clock DCK by two is such as illustrated in
When pixel data of two pixels have been inputted with two dot clocks DCK, the pixel data of two pixels are transferred to the microcomputer.
In this manner, two pixel data each of 8 bits which temporally continue to each other are made available from the outputs of the D-flip-flops 36 and 37 to be transferred to the built-in RAM of the microcomputer from the 16-bit transfer port under the timing of the transfer clock signal.
As is apparent from the foregoing, with the circuit arrangement described above, it is possible to fetch the image data on a real-time basis even when the transfer rate at which the pixel data is transferred to the built-in RAM of the microcomputer is lower than the output rate of the pixel data outputted from the image pick-up means 1 by latching a predetermined number of pixel data and by transferring en bloc the pixel data when the predetermined number of the pixel data have been latched.
In this manner, by realizing the pixel data transfer rate converting means 31 for converting the transfer rate of the image data with a simplified structure, the select range of the microcomputer can be broadened, which in turn means that the microcomputer of low cost and low transfer rate can be employed while mitigating limitation imposed on the built-in RAM capacity of the microcomputer similarly to the case of the first embodiment of the invention, whereby the total cost demanded for the vehicle-onboard driving lane recognizing apparatus can be reduced.
In the case of the vehicle-onboard driving lane recognizing apparatus first and second embodiments of the present invention, the image data outputted from the image pick-up means 1 are fetched for use. According to the teaching of the present invention incarnated in a third embodiment thereof, image data available from an external system or equipment can also be made use of.
Referring to
At this juncture, it should also be mentioned that the single-dotted broken line block 5 may be implemented as in the form of a single chip. In that case, only the image pick-up means 1 is provided externally of the chip 5. It goes however without saying that in case the image pick-up means 1 can be incorporated in the chip 5, it may be built in the chip similarly to the other means or constituents.
Now referring to
The decision signal EXTIN is used as a gate signal for three-state buffers 47a and 47b, respectively. Unless the image data is inputted from the external system, the image data derived from the output of the image pick-up means 1 is delivered intactly from the output of the three-state buffer 47a since the decision signal EXTIN is of “L” level, while the output of the three-state buffer 47b exhibits a high impedance.
Consequently, in the above-mentioned case, the image data outputted from the image pick-up means 1 are transferred to the image data storing means 3 by way of the image data selecting means 2 and at the same time the image data outputted from the image pick-up means 1 are supplied to the external system through image output line.
On the other hand, in the case where the image data are inputted from the external system, the decision signal EXTIN will then assume the level “H”. In that case, the output of the three-state buffer 47a exhibits high impedance, whereby the externally inputted image data delivered from the output of the three-state buffer 47b are transferred to the image data storing means 3 via the image data selecting means 2. Further, the externally inputted image data is supplied to the external system through the image output line as well.
By virtue of the arrangement of the vehicle-onboard driving lane recognizing apparatus described above, even through only the image data of a given limited region or zone can be stored internally of the apparatus, the whole image can also be observed or confirmed because all the image data are outputted externally.
Furthermore, in the case where the image data is available from the external system or equipment, the image data can be transferred to the image data storing means 3 incorporated in the apparatus. Then, the image data taken upon running or driving test or for the other purpose may be inputted from a relevant external system or equipment to thereby logically examine the driving lane recognition processing on a desk.
At this juncture, it should be mentioned that substantially only the three-state buffers 47a and 47b are additionally incorporated in the apparatus for realizing the function described above. In other words, the functions for checking or observing the whole image as well as for the logical examination on the desk can be added without increasing appreciably the control signal of the whole apparatus.
In the description of the first to third embodiments of the invention, no consideration has been paid to the basic clock signal supply. The vehicle-onboard driving lane recognizing apparatus according to a fourth embodiment of the present invention is so arranged that the individual means thereof are driven on the basis of a basic clock signal supplied from a single oscillator.
Referring to
In operation, the basic clock signal supplied to the image pick-up means 1 from the oscillator 51 is inputted to the dedicated driving signal generating means 1b incorporated in the image pick-up means 1.
The dedicated driving signal generating means 1b is designed to generate various driving signals such as the horizontal synchronizing signal HD and the vertical synchronizing signal VD for driving the image-pickup device 1a and others.
The generated driving signals are inputted to the image-pickup device 1a for driving the same. The image-pickup device 1a outputs the image data which are then stored in the image data storing means 3 by way of the image data selecting means 2.
The driving lane recognizing means 4 executes the processing for recognizing the driving lane by making use of the image data stored in the image data storing means 3. Parenthetically, operation of the driving lane recognizing means 4 is similar to that described hereinbefore in conjunction with the preceding embodiments of the invention. Accordingly, repeated description will be unnecessary.
At this juncture, it is to be noted that the basic clock signal for driving the image pick-up means 1 is in common to the clock for driving the image data selecting means 2, the image data storing means 3 and the driving lane recognizing means 4.
In the case of the conventional vehicle-onboard driving lane recognizing apparatus, the basic clock signal for driving the image pick-up means 1 is prepared separately from the clock signal for driving the other units than the image pick-up means 1 in view of the video signal standards. By contrast, in the vehicle-onboard driving lane recognizing apparatus according to the instant embodiment of the invention, the basic clock signal is commonly used for driving not only the image pick-up means 1 and the other portions (e.g. block 5). In other words, employment of the single oscillator is sufficient. Owing to this feature, the timing circuit design can be simplified and facilitated.
Further, for driving the image pick-up means 1 together with the other units with the common basic clock signal, such arrangement as shown in
In the vehicle-onboard driving lane recognizing apparatus shown in
In the vehicle-onboard driving lane recognizing apparatus shown in
As can be understood from the above, by driving the image pick-up means 1 and the other components with a common clock signal, it is sufficient to employ only one oscillator 51, which means that the vehicle-onboard driving lane recognizing apparatus can be manufactured at low cost. Besides, because the components or means incorporated in the vehicle-onboard driving lane recognizing apparatus are driven by the common clock signal, the arrangement for timing properly the operation of these means can easily be designed and implemented.
Further, by generating the driving signal for driving the image-pickup device 1a by means of the built-in timer 52 incorporated in the microcomputer as in the case of the vehicle-onboard driving lane recognizing apparatus shown in
It should however be mentioned that although the image pick-up means 1 and the other components or means are operated under the timing of the one and the same clock signal in the vehicle-onboard driving lane recognizing apparatus according to the instant embodiment of the invention, such arrangement may of course be adopted that the basic clock signal for driving the image pick-up means 1 is N times as high as the clock signal for operating the other means or components or alternatively the frequency of the basic clock signal for driving the image pick-up means 1 may be 1/N of that of the clock signal for driving the other means or components, where N represents a given natural number.
The block 5 indicated as enclosed by the single-dotted broken line may be realized in one chip. In that case, only the image pick-up means 1 is provided externally of the single-dotted line block 5. It should however be appreciated that the image pick-up means 1 may be incorporated in the chip 5 together with the other means or components if it is possible.
In general, the exposure control based on the luminance information of the whole image will incur essentially no problem. However, in the case of the vehicle-onboard image processing apparatus, it is preferred to prevent the luminance information of the unnecessary image region from being reflected in the exposure control. In other words, the exposure control should ideally be performed by making use of the luminance information of a concerned or relevant portion of the image. In that case, a luminance measuring timing signal is required for measuring luminance of the concerned portion of the image in synchronism with the operation timing of the image pick-up means 1. In this conjunction, it should be noted that because the same basic clock signal is employed commonly for all the components of the vehicle-onboard driving lane recognizing apparatus realized in the arrangement shown in
Many modifications and variations of the present invention are possible in the light of the above techniques. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described.
Number | Date | Country | Kind |
---|---|---|---|
2002-283589 | Sep 2002 | JP | national |
This is a divisional of application Ser. No. 11/683,247 filed Mar. 7, 2007, now abandoned which is a divisional of application Ser. No. 10/379,531 filed Mar. 6, 2003 now abandoned.
Number | Name | Date | Kind |
---|---|---|---|
5926164 | Terakawa et al. | Jul 1999 | A |
6108765 | Caudel et al. | Aug 2000 | A |
6704434 | Sakoh et al. | Mar 2004 | B1 |
6829388 | Sakurai | Dec 2004 | B1 |
6925206 | Akutagawa | Aug 2005 | B2 |
20020131620 | Shirato | Sep 2002 | A1 |
20030063773 | Muramatsu et al. | Apr 2003 | A1 |
20030103674 | Publicover | Jun 2003 | A1 |
20040062418 | Ishikura et al. | Apr 2004 | A1 |
Number | Date | Country |
---|---|---|
05-164569 | Jun 1993 | JP |
09-223218 | Aug 1997 | JP |
11-213155 | Aug 1999 | JP |
2000-099896 | Apr 2000 | JP |
2001-022638 | Jan 2001 | JP |
Number | Date | Country | |
---|---|---|---|
20100098296 A9 | Apr 2010 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11683247 | Mar 2007 | US |
Child | 12055132 | US | |
Parent | 10379531 | Mar 2003 | US |
Child | 11683247 | US |