This disclosure is directed generally to semi-autonomous and autonomous vehicles and, in particular, to the detection of vacant parking spots based on sensor input.
Autonomous or semi-autonomous vehicles rely on a host of sensors capable of detecting and interpreting the environment surrounding the vehicle. These systems may be utilized to provide driver assist functions, such as parking assist, autonomous parking, and autonomous valet parking. For example, having entered a parking facility the vehicle may enter a valet parking mode in which the passengers exit the vehicle and the vehicle proceeds to locate a parking spot. The process of locating a vacant parking spot requires an unoccluded view of the parking spot, which is complicated by the presence of other parked vehicles. Typically, this requires the vehicle to travel down the parking aisle to a position adjacent to a vacant spot to allow on-board sensors to detect the vacant spot. Logistically, this is problematic as it may require the vehicle to traverse a number of parking aisles in order to find a vacant spot. It would be beneficial to develop a system and method that allows the vehicle to detect vacant parking spots without requiring the vehicle to be adjacent to the detected parking spot (i.e., to allow the vehicle to determine from a position adjacent to the aisle whether the aisle is likely to include vacant parking spots).
According to some aspects, a method of identifying vacant parking locations may include receiving sensor data captured with respect to a parking aisle and analyzing the received sensor data to detect shadows cast by parked vehicles within the parking aisle. The method may further include detecting a vacant parking spot based on the detected shadows and generating an output indicating whether a parking spot vacancy has been detected within the parking aisle.
According to another aspect, a method of self-parking a vehicle within a parking lot may include positioning the vehicle at a first end of a first aisle and detecting shadows in the aisle using input received from one or more first sensors. The method may further include detecting parked vehicles based on the detected shadows, wherein if all parking spots are occupied the vehicle is instructed to proceed directly to the next aisle.
According to another aspect, a parking vacancy detection system for use in vehicles may include a first plurality of sensors configured to detect image data associated with a parking aisle located adjacent to the vehicle and a processor/controller configured to receive the image data detected by the first plurality of sensors. In some embodiments, the processor/controller is configured to detect shadows based on the received image data and to detect vacant parking spots based on detected gaps within the detected shadows, wherein the processor/controller generates an output identifying whether the parking aisle includes vacant parking spots.
According to some embodiments, a method of detecting vacant parking location within a parking lot includes receiving sensor data captured with respect to the parking lot (e.g., parking aisle). The sensor data—such as camera and/or infrared images—are utilized to detect the presence of shadows cast by the parked vehicles. For example, a variety of methods (e.g., color-based, texture-based, intensity-based, etc.) may be utilized to detect the presence of shadows within an image. Likewise, infrared images may be utilized to detect shadows based on differences in temperature resulting from the presence of shadows. Vacant parking spots are detected based on analysis of the detected shadows—in particular analysis of the gaps between adjacent shadows. If a vacant parking spot is detected, the vehicle may proceed down the aisle to the detected vacancy. If no vacant parking spot is detected, then the vehicle may proceed to the next aisle and repeat the process. A benefit of the system and method described herein is that the vehicle may be able to detect parking spot vacancies without being required to proceed down each aisle to a location adjacent to the vacancy.
Referring to
In the example shown in
In some embodiments, vehicle 106 includes one or more first sensors (not shown) utilized to detect the presence of objects (e.g., vehicles) and one or more second sensors utilized to detect the presence of or effects of shadows. For example, in some embodiments the one or more first sensors utilized to detect the presence of adjacent vehicles includes one or more of LiDAR sensors, radar sensors, and/or cameras. In some embodiments, the one or more second sensors utilized to detect the presence of or effects of shadows includes one or more of cameras and infrared sensors. In some embodiments, the same sensor utilized to detect the presence of vehicles may also be utilized to detect the presence or lack thereof of shadows. For example, a camera-based sensor may be utilized to both detect the presence of vehicles and detect the presence of shadows.
Referring to
According to some embodiments, the parking vacancy detection system 200 is deployed on a vehicle being parked autonomously or semi-autonomously and includes one or more sensors, including one or more of vehicle detection sensors 202, shadow detection sensors 204, and orientation sensors 206, and a processor/controller 208. In some embodiments, the vehicle detection sensor includes one or more of LiDAR sensors, radar sensors, cameras, and/or other sensors capable of detecting the presence of objects/vehicles within the vicinity of the vehicle being parked. In some embodiments, the shadow detection sensors 204 include one or more cameras and/or infrared sensors capable of detecting the presence of shadows. In some embodiments, an orientation sensor 206 provides input to the parking vacancy detection system 200 regarding the orientation of the vehicle to be parked, wherein the orientation sensor 206 includes one or more of an inertial measurement unit (IMU), accelerometers, etc. In some embodiments, some of the sensors utilized for vehicle detection are also utilized for shadow detection. For example, cameras may be utilized for both vehicle detection and shadow detection.
In some embodiments, processor/controller 208 receives inputs from one or more of the sensors and utilizes the received inputs to detect parking vacancies. In some embodiments, processor controller 208 may additionally receive inputs from other sources, such as map information provided with respect to a parking facility and/or inputs regarding the weather and/or location of the sun. As described in more detail below, in some embodiments these inputs are utilized to make determinations regarding whether shadows are present and if so, the likely direction in which shadows will be cast. For example, information regarding the location of artificial lights within indoor and outdoor (at least at night) parking facilities may be utilized to determine the likely location of shadows cast by parked vehicles. Likewise, the location/orientation of the sun may similarly be utilized to determine the likely location of shadows cast by parked vehicles.
In some embodiments, processor/controller 208 utilizes the received inputs to detect parking vacancies and to provide an output utilized by the semi-autonomous or autonomous vehicle to assist in self-park operations. For example, detection of a parking vacancy may result in the semi-autonomous or autonomous vehicle proceeding down the aisle to the parking vacancy detected. Alternatively, a determination that the aisle does not include any parking vacancies may result in the semi-autonomous or autonomous vehicle proceeding to the next aisle, wherein the process is repeated. In some embodiments, the determination that an aisle does not include any parking vacancies is not simply a determination that no vacancies were detected, but rather a determination than all parking spots are occupied. In the case that it can be determined that all spots are in fact occupied, then the autonomous vehicle may proceed to the next aisle, saving the cost associated with travel down the occupied aisle.
For example, the flowchart provided with respect to
At step 304, one or more first sensors are utilized to detect the presence of parked vehicles in the aisle. For example, in the embodiment shown in
At step 306, a determination is made whether shadows may be utilized to detect the presence of vacant parking spots. In some embodiments, this determination may be made based on one or more of a plurality of factors. For example, in some embodiments the time of day may be utilized to determine whether it is likely that shadows will be generated (e.g., if at night, shadows will not be cast by the sun). In some embodiments, the weather (e.g., cloudy, sunny) may be utilized to determine the likelihood of shadows being cast. In some embodiments, shadows cast by artificial light may be utilized in the event information is available regarding the presence of artificial light sources and assuming a time of day that would utilize artificial light sources.
In other embodiments, the determination of whether shadows are being cast may further include whether shadows are being cast in a way that is useful for detecting vacancies. This may include utilizing information regarding the location of a light source (e.g., sun, artificial sources, etc.) relative to a parking aisle and the resulting direction in which shadows will be cast. This may also require information regarding the orientation of the vehicle as provided by orientation sensor 206 shown in
If at step 306 a determination is made that shadows cannot be utilized to detect the presence of vacant parking spots, then the vehicle is instructed to traverse down the aisle at step 312 and to utilize one or more sensors to detect the presence of a vacant parking spot. If at step 306 a determination is made that shadows can be utilized to detect the presence of vacant parking spots, then the method proceeds to step 308.
At step 308, the one or more sensors are utilized to detect the presence of shadows. A number of methods may be utilized to detect the presence of shadows. For example, in some embodiments the input is an image such as those shown in
In other embodiments, other types of sensor inputs may be utilized, such as infrared inputs. Infrared sensors detect temperature and may be utilized in outdoor parking lots during the day in which the presence of shadows will result in a temperature differential capable of detection. In still other embodiments, other indications of the presence and/or absence of vehicles may be utilized. For example, the presence of dry pavement (adjacent to vehicles) and wet pavement (resulting from a lack of shielding from a vehicle) may be utilized to detect vacant parking spots.
At step 308, if no shadows are detected using one or more sensors—but from step 304 we know that vehicles are parked in the aisle—then shadows will not be available to detect the presence of vacancies and the vehicle will be required to drive down the parking aisle at step 312. If at step 308 one or more shadows are detected, then a determination is made that shadows can be utilized to detect the presence or absence of parking vacancies and the method proceeds to step 310.
At step 310 the vehicle shadows detected at step 308 are utilized to detect parking vacancies. In particular, gaps or spaces in the detected shadows of the appropriate size, shape, etc. are utilized to detect a parking vacancy (or conversely the lack of gaps or spaces are utilized to determine that all parking spots are occupied). For example, referring to the example shown in
If no missing shadows are detected at step 310, then an output is generated indicating that this aisle does not include any parking vacancies and at step 302 the vehicle is instructed to proceed to the next parking aisle and repeat the process. In some embodiments, the determinations at steps 304, 306, and 308 that the parking aisle includes parked vehicles (step 304), that shadows are available for detection (step 306), and are in fact detected (step 308), a determination at step 310 that no missing shadows are detected may also be considered a determination that all parking spots in the aisle are occupied. In particular, the determination at step 310 that all parking spots are occupied prevents the vehicle from having to drive down the parking aisle—and potentially back up the parking aisle if no spot exists. If a missing shadow indicative of a parking vacancy is detected at step 310, then an output is generated indicating the presence of a vacancy and the vehicle proceeds to travel down the parking aisle.
At step 314 one or more sensors are utilized to detect the vacancy identified by the missing shadows. For example, in some embodiments the one or more sensors are the same one or more sensors utilized to detect vehicles at step 304. In some embodiments, the one or more sensors include LiDAR sensors, radar sensors, ultrasonic sensors and/or cameras utilized to detect a parking spot vacancy. In the event no parking spot vacancy is detected (e.g., false positive detected based on the shadows) then an output is generated indicating the false event and the vehicle is instructed to proceed to the next aisle at step 302. In some embodiments, analysis may be performed to determine what caused the false positive event and may be provided in feedback to prevent the same root cause from causing future false positives.
If at step 314 a vacancy is detected based on the one or more sensors, then at step 316 outputs are generated to cause the autonomous or semi-autonomous vehicle to park within the vacancy and the process ends.
Referring now to
At step 403, one or more attributes associated with the detected shadows are measured. In some embodiments, measured attributes may include one or more of shadow length, shadow width, length-to-width ratio, and/or angle of the created shadows. In some embodiments, measured attributes are provided with respect to each shadow attributed to a particular vehicular shadow. For example, in the embodiment shown in
At step 404, the measured attributes are utilized to determine whether the detected shadows are indicative of parked vehicles. For example, in some embodiments based on knowledge of the location of the illumination source (e.g., sun, lights), and expected dimensions of vehicles, expected shadow lengths/widths, length-to-width ratios, etc. can be determined. If at step 404 a determination is made that the detected shadows are not cast by one or more vehicles, then at step 406 a determination is made that the shadows cannot be used to detect vacancies. In response, the vehicle may be required to travel down the aisle and detect vacancies using other sensors. If at step 404 a determination is made that the detected shadows are cast by one or more vehicles, this indicates that the shadows can be utilized to detect vacancies, and the method continues at step 408 to use the measured attributes to define thresholds for detecting vacancies.
At step 408 one or more thresholds are generated based on the one or more measured attributes to determine whether a gap between adjacent shadows is indicative of a vacancy. In some embodiments, a plurality of thresholds or dynamic thresholds are defined based on the location of the shadows. Once again, the threshold may be expressed in the form of pixels, distances, ratios or angle thresholds depending on how the detected shadows are expressed. In some embodiments, for shadows or gaps in shadows located close to the vehicle, the number of pixels associated with a vehicle shadow or gap may be larger than the number of pixels associated with a vehicle shadow or gap located further down the aisle. For example, this is illustrated in
At step 410, the threshold(s) generated at step 408 are compared to gaps between adjacent shadows to detect vacant parking spots. In some embodiments, parking vacancies are detected in response to the gap between adjacent shadows being greater than the defined threshold. For example, in the embodiment shown in
In this way, the present disclosure provides a system and method of detecting vacant parking spots based on detected shadows (i.e., detecting that each parking spot is occupied). A benefit of the system and method described herein is that the vehicle is able to detect in some cases if each parking spot in an aisle is occupied and therefore prevents the vehicle from having to traverse these aisles.
While the invention has been described with reference to an exemplary embodiment(s), it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment(s) disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.
The following are non-exclusive descriptions of possible embodiments of the present invention.
According to one aspect, a method of identifying vacant parking locations may include receiving sensor data captured with respect to a parking aisle and analyzing the received sensor data to detect shadows cast by parked vehicles within the parking aisle. Vacancies may be detected based on the detected shadows and an output generated indicating whether a parking spot vacancy has been detected.
The method of the preceding paragraph can optionally include, additionally and/or alternatively any, one or more of the following features, configurations and/or additional components.
According to some embodiments, the sensor data is image data captured by a camera.
According to some embodiments, the sensor data is thermal data captured by an infrared camera.
According to some embodiments, the method may further include measuring one or more attributes of detected shadows.
According to some embodiments, measuring one or more attributes of detected shadows includes measuring one or more of shadow length, shadow width, length-to-width ratio, and angle-of-shadow.
According to some embodiments, the method may further include generating a threshold(s) utilized to determine whether a gap between adjacent shadows in indicative of a parking spot vacancy based on the one or more measured attributes.
According to some embodiments, detecting a vacant parking spot based on the detected shadows may further include comparing a gap between adjacent shadows to the generated threshold(s).
According to another aspect, a method of self-parking a vehicle within a parking lot may include positioning the vehicle at a first end of a first aisle and detecting shadows in the aisle using input received from one or more first sensors. The method may further include detecting parked vehicles based on the detected shadows, wherein if all parking spots are occupied the vehicle is instructed to proceed directly to the next aisle.
The method of the preceding paragraph can optionally include, additionally and/or alternatively any, one or more of the following features, configurations and/or additional components.
According to some embodiments, the method may further include detecting a presence of parked vehicles using input received from one or more second sensors, wherein if no parked vehicles are detected the vehicle is instructed to proceed down the first aisle.
According to some embodiments, the method may further include determining based on one or more inputs whether shadows are available for detection, wherein the one or more inputs includes one or more of time of day, and weather, position/angle of the sun.
According to some embodiments, if shadows are not available for detection the vehicle is instructed to proceed down the first aisle.
According to some embodiments, the method may further include measuring one or more attributes of detected shadows, wherein the one or more measured attributes include one or more of shadow length, shadow width, length-to-width ratio, and angle-of-shadow.
According to some embodiments, the method may further include generating a threshold(s) utilized to determine whether a gap between adjacent shadows is indicative of a parking spot vacancy based on the one or more measured attributes.
According to some embodiments, detecting a vacant parking spot based on the detected shadows may include comparing a gap between adjacent shadows to the generated threshold(s).
According to another aspect, a parking vacancy detection system for use in vehicles may include a first plurality of sensors configured to detect image data associated with a parking aisle located adjacent to the vehicle and a processor/controller configured to receive the image data detected by the first plurality of sensors. In some embodiments, the processor/controller is configured to detect shadows based on the received image data and to detect vacant parking spots based on detected gaps within the detected shadows, wherein the processor/controller generates an output identifying whether the parking aisle includes vacant parking spots.
The system of the preceding paragraph can optionally include, additionally and/or alternatively any, one or more of the following features, configurations and/or additional components.
For example, in some embodiments the system may further include a second plurality of sensors, wherein the second plurality of sensors are configured to detect the presence of parked vehicles and wherein the processor/controller utilizes the detected presence of parked vehicles to determine whether shadows can be utilized to detect vacant parking spots.
According to some embodiments the processor/controller is configured to receive input from one or more external sources regarding the presence of and orientation of a light source, wherein the processor/controller utilizes the received input to determine whether shadows can be utilized to detect vacant parking spots.
According to some embodiments, the processor/controller is configured to receive orientation data from one or more orientation sensors, wherein the processor/controller utilizes the received orientation data in combination with input from one or more external sources regarding the presence of and orientation of a light source to determine a likely location of shadows cast in response to parked vehicles.
According to some embodiments, the processor/controller may measure one or more attributes of detected shadows.
According to some embodiments, the processor/controller may measure one or more of shadow length, shadow width, length-to-width ratio, and angle-of-shadow and generates a threshold utilized to determine whether a gap between adjacent shadows in indicative of a parking spot vacancy based on the one or more measured attributes.
This application is a continuation application and claims the benefit of U.S. patent application Ser. No. 16/824,882, filed Mar. 20, 2020, the entire disclosure of each of which is hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
Parent | 16824882 | Mar 2020 | US |
Child | 17739905 | US |