Patent Application
20130202155
In various embodiments, the present invention relates, in general, to image processing and, more specifically, to detecting road-lane markers in images.
Automated road-navigation systems provide various levels of assistance to automobile drivers to increase their safety and/or to reduce their driving effort. Various techniques have been developed to gather information about a vehicle's location, moving path, and/or surrounding environment. For example, vision-based road-lane tracking systems may be used to detect lane markers for adaptive cruise control, vehicle tracking, obstacle avoidance, lane-departure warning, and/or driving-pattern detection. In the lane-tracking systems, cameras may be mounted to the front of a vehicle to capture images of the roadway ahead of the vehicle, and image-processing software may be used to identify the lane markers in the images.
A Hough-transform algorithm may be used to identify lines in an acquired image, especially when the signal-to-noise ratio of the image is low and/or the variation of brightness in the image is large. The Hough transform converts a line in the image into a single point having two parameters: ρ (representing the shortest distance between the line and the origin) and θ (representing the angle between the shortest line and the x-axis). An image consisting of many shapes may therefore be converted into a plurality of (ρ,θ) pairs (which may be stored in a two-dimensional array of ρ and θ values), and analyzed to detect which shapes are lines. Because the Hough transform requires an unpredictable, random access to the two-dimensional array, however, it requires a large local memory or cache to hold the entire image and/or array in order to operate quickly and efficiently. If the Hough transform is run on a digital-signal, low-power, or other type of process having limited local memory, the entire image and/or array cannot be stored locally, resulting in an unacceptable number of calls to a much slower main memory. Additionally, the Hough Transform is able to detect only straight lane markers, not curved ones.
Other techniques, such as the so-called B-Snake road model and the probabilistic-fitting model, have been proposed to detect curved lane markers. They all, however, involve random memory accesses and thus require the entire image to be stored in the local memory to run efficiently and are similarly unsuitable for use with a processor having limited internal memory. Consequently, there is a need for real-time detection of both straight and curved lane markers using a low-cost, low-power processor having limited internal memory.
In various embodiments, the present invention relates to systems and methods for quickly and accurately detecting straight and/or curved road-lane markers using only a part of a received roadway image (or images), thereby providing real-time vehicle-position information, relative to the road-lane markers, without the need for a processor having a large internal/local memory. In one embodiment, a road-lane marker detector first scans through at least one horizontal line of the received image. The position of any road-lane markers in the received image is determined by computing and analyzing the intensity gradient of the scanned line; changes in the intensity gradient may indicate presence of one or more lane markers. The positions of two identified lane markers may further provide information about the vehicle's position relative to the lane markers. Furthermore, the shape of the roadway may be obtained by analyzing the lane markers' positions in multiple scanned lines of the image. Because the captured image is scanned line-by-line, only a small fraction of the image is needed during processing, and that fraction is predictable and deterministic (thus avoiding random access to memory). In one embodiment, images acquired at different times provide real-time information, such as the shape of the road and/or the distance between the vehicle and the lane markers. False detection of the lane markers may be reduced or eliminated based on properties of the lane-marker perspective geometry.
Accordingly, in one aspect, a method for detecting a lane marker includes: (i) receiving, from an image acquisition device, a first image including the lane marker; (ii) scanning, into a memory, a first substantially horizontal line across the first image; (iii) computing, using a processor, an intensity gradient of the first substantially horizontal line; and (iv) determining a first position of the lane marker by analyzing the intensity gradient. In one embodiment, analyzing the intensity gradient includes determining a left edge and a right edge of the lane marker in the first substantially horizontal line based at least in part on the intensity gradient. The substantially horizontal line may be a horizontal line. The method may further include determining a second position of a second lane marker by analyzing the intensity gradient and/or determining a position of a vehicle based on an angle between the first position of the road lane marker and the first substantially horizontal line.
The method may further include (i) scanning, into the memory, a plurality of additional substantially horizontal lines across the first image and (ii) determining positions of the lane marker in the plurality of additional substantially horizontal lines. A shape of a road may be determined based at least in part on the positions of the lane marker in the first substantially horizontal line and in the plurality of additional substantially horizontal lines.
A false detection of the lane marker may be eliminated in one of the substantially horizontal lines; eliminating the false detection of the lane marker may include (i) determining a width of the lane marker based at least in part on the intensity gradient and (ii) eliminating a false position of the lane marker having a width greater than a predetermined maximum threshold or less than a predetermined minimum threshold. Alternatively or in addition, eliminating the false detection of the lane marker may include (i) determining a vanishing point based at least in part on the positions of the lane markers in the plurality of scanned lines and (ii) eliminating a list of false positions having an associated line, wherein an extension of the associated line is outside of a detection region around the vanishing point.
The method for detecting a lane marker in a roadway may further include: (i) receiving, from an image acquisition device, a second image comprising the lane marker; (ii) scanning, into a memory, a second substantially horizontal line across the second image; (iii) computing, using a processor, a second intensity gradient from the second scanned line; and (iv) determining a second position of the lane marker by analyzing the second intensity gradient. A shape of a road may be determined based at least in part on the first position of the lane marker in the first image and the second position of the lane marker in the second image.
In another aspect, a system for detecting a lane marker in a roadway image includes: (i) an input port for receiving the roadway image; (ii) a main memory for storing the roadway image; (iii) a local memory for storing one substantially horizontal line of the roadway image; and (iv) a processor for computing an intensity gradient of the substantially horizontal line and determining a position of a lane marker in the substantially horizontal line. The processor, which may be a digital-signal processor, may be further configured for determining a position of a vehicle relative to the lane marker.
An output device may alert a user (via, for example, a user interface) if a distance between the vehicle and the lane marker is less than a threshold. An image-acquisition device may be used for acquiring the roadway image. The local memory of the system may be too small to store the roadway image; a link between the processor and the local memory in the system may be faster than a link between the processor and the main memory.
As used herein, the terms “approximately” or “substantially” means ±10% (e.g., by distance or by angle), and in some embodiments, ±5%. Reference throughout this specification to “one example,” “an example,” “one embodiment,” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example of the present technology. Thus, the occurrences of the phrases “in one example,” “in an example,” “one embodiment,” or “an embodiment” in various places throughout this specification are not necessarily all referring to the same example. Furthermore, the particular features, structures, routines, steps, or characteristics may be combined in any suitable manner in one or more examples of the technology. The headings provided herein are for convenience only and are not intended to limit or interpret the scope or meaning of the claimed technology.
These and other objects, along with advantages and features of the present invention herein disclosed, will become more apparent through reference to the following description, the accompanying drawings, and the claims. Furthermore, it is to be understood that the features of the various embodiments described herein are not mutually exclusive and can exist in various combinations and permutations.
In the following description, various embodiments of the present invention are described with reference to the following drawings, in which:
The system 200 may further include a user interface 260 (e.g., a WiFi link) for communicating with a user and/or an output device 270, such as an alarm. The local memory 230 may be disposed outside of the main processor 240 or located inside of the main processor 240. The main processor 240 may be implemented as part of a computer, a mobile device, a navigation system, or any other type of computing system. The user interface 260 may output and display results to a user and/or receive requests, such as commands and/or parameters from the user. The output device 270 may provide an audio or visual alert to the user when, for example, the vehicle drifts too close to the lane markers. In one embodiment, the processor 240 connects to the steering system of the vehicle. When the vehicle is too close to the lane markers, the steering system forcibly steers the vehicle back to the center of the road. If the automatic driving system is enabled, the steering system maintains the vehicle's position in the center of the road based on detected positions of the lane markers.
With reference to
An intensity gradient 340 may be created using the intensity map 330. In some embodiments, a discrete differentiation filter that can be implemented efficiently in hardware or software is used to compute an approximation of the image intensity gradient. For example, a modified Prewitt Filter:
may be used to obtain the intensity gradient map 340. The left edge of the lane marker 310 may be found by identifying a point at which the left side 342 of the intensity gradient 340 increases above a predetermined maximum threshold, +Th; the right edge of the lane marker 310 may be found by identifying a point at which the right side 344 of the intensity gradient 340 increases above a predetermined minimum threshold, −Th.
Detecting the lane markers based on the intensity gradient 340 may be performed under various lighting conditions, such as bright sun light or dim moon light. In various embodiments, +Th and −Th are adjusted to reflect the quality of the image contrast and/or brightness of the image. For example, +Th and −Th may have low absolute values when an image has poor contrast and high absolute values when the image has good contrast. The center 346 and the width w of the lane marker 310 may be determined based on the left 342 and right 344 edges thereof. Detecting positions of the lane markers is thereby very fast, occurring as soon as one horizontal line is scanned and the intensity gradient map thereof is analyzed. Embodiments of the current invention, therefore, may be implemented in a low-cost processor having limited memory.
In one embodiment, the lane marker 312 on the other, right-hand side of the road is detected based on the intensity gradients 352, 354, using the same approach as described above. The position of the vehicle relative to the lane markers may then be estimated using the detected centers 346, 356 of the lane markers. With reference to
More than one line in an image may be scanned, and additional information about the image may be derived from the two or more lines.
Some lane markers may be dashed (i.e., they may contain periodic breaks). In some embodiments, referring to
In various embodiments, a relative distance between the vehicle and the lane markers may be determined based on the angles between the detected central lines and the scanned horizontal lines. Referring to
In various embodiments, false detection of the lane markers is determined based on their width. In one embodiment, an assumption of approximately constant width of the lane marker is used to eliminate the false detection of the lane markers. For example, a detected lane marker having a width more than (or less than) 10% of a predetermined width size is considered a false detection; the detected center is then omitted in the current scanned line. The central line of the lane markers thus connects the centers detected from the previous scanned line and the next scanned line. The standard width size may vary in different countries and may be adjusted accordingly.
In another embodiment, an assumption that the left and right lane markers of a roadway vanish at a distant point is used to eliminate the false detection of the lane markers. This applies to both straight and curved lines.
In some embodiments, the lane marker detector receives a plurality of images taken at different points in time. The algorithms for lane marker detection and false detection elimination may be applied to each image and additional information may be extracted. For example, if it is detected that the vehicle is close to a lane marker but that the vehicle is moving back toward the center of the lane, a minor (or no) alarm may be sounded. If, on the other hand, the vehicle is close to the lane marker but is moving even closer to the lane marker, a louder or more noticeable alarm may be raised. Because the algorithms use only a fraction of each image to detect the lane markers therein, it is computationally fast to detect lane markers in this temporal series of images and thereby provides real-time information about, for example, the vehicle position relative to the lane markers and/or the shape of the roadway.
A method 700 for detecting lane markers in accordance with embodiments of the current invention is shown in
The terms and expressions employed herein are used as terms and expressions of description and not of limitation, and there is no intention, in the use of such terms and expressions, of excluding any equivalents of the features shown and described or portions thereof. In addition, having described certain embodiments of the invention, it will be apparent to those of ordinary skill in the art that other embodiments incorporating the concepts disclosed herein may be used without departing from the spirit and scope of the invention. Accordingly, the described embodiments are to be considered in all respects as only illustrative and not restrictive.
