The present invention relates to assisting with the determination of a position or orientation of a vehicle, including using sensor data to improve the determining of a position or orientation of a vehicle.
Location-sensitive features of modern vehicles can enhance a vehicle operator's experience by providing information or making decisions that take into account the vehicle's position and orientation in the world. For example, onboard vehicle navigation systems can use a vehicle's current position to compute the shortest route to a target destination, or to suggest nearby products or services; and autonomous vehicles can use the vehicle's location and orientation to automate driving operations such as steering and parking. (As used herein, an autonomous vehicle can be one in which one or more driving operations traditionally performed by a human driver may be performed or enhanced by a computer system.) Location-sensitive features are only as accurate, useful, and reliable as the location data on which they rely. Existing location systems such as the Global Positioning System (GPS) and Global Navigation Satellite System (GNSS) are in wide use, but the limited precision of data received from those systems may limit the development of vehicle features that depend on those systems. Further, existing location systems are often dependent on satellite signals of sufficient strength, which may be unavailable or intermittently available in some situations. It is thus desirable to provide vehicles with more precise location data to improve the accuracy and usability of existing location-sensitive features, and to enable new such features. Further, it is desirable to provide such data while making use of existing location systems, such as GPS, in which there has been substantial investment and on which many existing systems currently rely. It is an intent of the present invention to augment existing vehicle location systems such as GPS using the systems and methods disclosed herein. It is a further intent of the present invention to enhance the behavior of autonomous driving systems with precise localization information.
An example of the present invention is directed to using sensor data representing information about a vehicle's surroundings, and/or one or more landmarks in the vicinity of the vehicle that are identified from the data, to determine a position or orientation of the vehicle relative to the one or more landmarks. In accordance with another aspect of the example, one or more of the landmarks can be identified from map data that relates landmarks to a world coordinate system, for example by associating point clouds of structures or terrain with satellite (e.g., GPS) coordinates. Using a position or orientation of the vehicle relative to one or more landmarks, and a position or orientation of the one or more landmarks relative to the world coordinate system, a position or orientation of the vehicle relative to the world coordinate system can be determined. The precision of such position or orientation potentially may exceed the precision or reliability of a position or orientation obtained from a single location system, such as GPS, alone.
In the following description of examples, reference is made to the accompanying drawings which form a part hereof, and in which it is shown by way of illustration specific examples that can be practiced. It is to be understood that other examples can be used and structural changes can be made without departing from the scope of the disclosed examples.
In some examples, the vehicle control system 100 can be connected or operatively coupled to (e.g., via controller 120) one or more actuator systems 130 in the vehicle and one or more indicator systems 140 in the vehicle. The one or more actuator systems 130 can include, but are not limited to, a motor 131 or engine 132, battery system 133, transmission gearing 134, suspension setup 135, brakes 136, steering system 137 and door system 138. The vehicle control system 100 can control, via controller 120, one or more of these actuator systems 130 during vehicle operation; for example, to open or close one or more of the doors of the vehicle using the door actuator system 138, or to control the vehicle during autonomous or semi-autonomous driving or parking operations, using the motor 131 or engine 132, battery system 133, transmission gearing 134, suspension setup 135, brakes 136 and/or steering system 137, etc. The one or more indicator systems 140 can include, but are not limited to, one or more speakers 141 in the vehicle (e.g., as part of an entertainment system in the vehicle), one or more lights 142 in the vehicle, one or more displays 143 in the vehicle (e.g., as part of a control or entertainment system in the vehicle) and one or more tactile actuators 144 in the vehicle (e.g., as part of a steering wheel or seat in the vehicle). The vehicle control system 100 can control, via controller 120, one or more of these indicator systems 140 to provide indications to a driver of the vehicle of one or more characteristics of the vehicle's surroundings that are determined using the onboard computer 110, such as a position or orientation of the vehicle relative to a world coordinate system.
It can be beneficial to use sensors 107 to determine a position or orientation of a vehicle relative to a world coordinate system. Examples of the disclosure are directed to using one or more sensors attached to a vehicle in conjunction with a location system, such as a GPS receiver, to identify such a position or orientation with a higher degree of precision than can be achieved using a GPS receiver alone. The disclosure is not limited to any particular type of world coordinate system (e.g. geodetic, earth-centered earth-fixed (ECEF)); nor is the disclosure limited to any particular type of location system (e.g. GPS), or even to the use of a location system at all; nor is the disclosure limited to any particular type of representation of a position or an orientation.
Examples of the disclosure are directed to using map data to determine a position or orientation of a vehicle relative to a world coordinate system. The disclosure is not limited to any particular type or format of map data; nor is the disclosure limited to map data stored or received in any particular manner. For example, the map data could be stored in local memory, streamed via the Internet, received via broadcast, etc.
Examples of the disclosure are directed to identifying landmarks. As used herein, a landmark is any point or region for which a position or orientation can be expressed as coordinates in a coordinate system. For example, a landmark could be a tree, a lamp post, a pedestrian, a building, a street sign, a road intersection, a city, a point on a two-dimensional map, a grain of sand, or many other things.
The usefulness of a landmark can be quantified according to how significant that landmark is in determining a vehicle's position and/or orientation. In some examples, landmarks may be identified if they are likely to meet or exceed a threshold value of usefulness, and rejected otherwise. In some examples, a landmark may influence determining a position or orientation of a vehicle with respect to a world coordinate system based on its expected usefulness.
Various techniques for identifying landmarks from sensor data are known to those skilled in the art. Such techniques vary based on the number and type of sensors employed. For example, where the sensor data includes an image presented by a camera, edge detection algorithms known in the art can be applied to the image to identify the shapes and boundaries of landmarks.
In some examples, landmark identification may be improved by utilizing sensor data from multiple sensors instead of a single sensor. For example, using techniques known in the art, LIDAR data and camera data can be combined to identify nearby objects with more accuracy than is possible with either LIDAR data or camera data acting alone.
Landmarks may be classified by their expected usefulness, as in some examples, using techniques known to those skilled in the art. For example, mobile objects such as pedestrians, which carry a low degree of expected usefulness, can be identified by comparing sensor data from multiple points in time to determine whether the landmarks have moved over time.
In
With knowledge of the position and/or orientation of a vehicle with respect to a landmark identified from both sensor data and map data, and knowledge of the landmark with respect to a world coordinate system, the position and/or orientation of the vehicle with respect to the world coordinate system is determined using techniques known in the art. As an example, if a position and orientation of the vehicle with respect to the landmark is represented as a matrix A, and a position and orientation of the landmark with respect to the world coordinate system is represented as a matrix B, then a matrix C representing a position and orientation of the vehicle with respect to the world coordinate system can be computed from matrix A and matrix B using standard matrix algebra.
Where the sensor data and map data is of sufficiently high accuracy and resolution, the determined position and/or orientation of the vehicle with respect to the world coordinate system may be more accurate than what can be estimated using GPS or another location system.
Further improvements are contemplated by the disclosure. In some examples, computational efficiencies can be achieved by storing computed values, such as position and/or orientation values for significant landmarks, in a shared repository. Other vehicles can then retrieve these values from the shared repository, using techniques known in the art, instead of recomputing them. For frequently traveled bridges and roadways, for example, position and orientation values could be pre-computed and stored in a shared repository for later retrieval by vehicles using those bridges and roadways. Additionally, data stored in a shared repository could be reviewed for analytic purposes.
Other examples use sensor data to supplement or correct map data, which may become out of date and inaccurate as changes are made to roads and other landmarks. As one example, if existing map data relating to a road sign indicates the road is open, and new sensor data relating to the road sign indicates the road is now closed, the sensor data indicates the map data is out of date; the map data in some examples is updated accordingly using the sensor data.
Other examples utilize machine learning techniques, known in the art, to enhance determining a position or orientation. In some examples, values including positions, orientations, and estimated locations are used to train a neural network, the neural network to be used according to techniques known in the art to improve the speed and accuracy of future position and orientation determinations.
In some examples in which a vehicle whose position or orientation is to be determined is a fully autonomous vehicle, the determined position or orientation is used by the vehicle to execute a driving operation. For example, a fully autonomous vehicle could automate parking behaviors or driving maneuvers in close quarters using a position or orientation determined with a high degree of precision.
In some examples in which the vehicle is not a fully autonomous vehicle, the determined position or orientation is used to enhance the execution of manual driving operations—for example, by automatically enabling anti-lock braking systems or traction control systems when the vehicle traverses locations with low-friction road surfaces.
Some examples of the disclosure are directed to a method of determining a position or orientation of a vehicle, the method comprising: identifying a landmark using sensor data presented by one or more sensors included with the vehicle; identifying the landmark using map data relating to an approximate location of the vehicle in a world coordinate system; determining a position or orientation of the vehicle relative to the landmark; determining a position or orientation of the landmark relative to the world coordinate system; and determining, using the determined position or orientation of the vehicle relative to the landmark and the determined position or orientation of the landmark relative to the world coordinate system, a position or orientation of the vehicle relative to the world coordinate system. Additionally or alternatively to one or more of the examples disclosed above, in some examples, the method further comprises the step of storing a position or orientation in a shared repository. Additionally or alternatively to one or more of the examples disclosed above, in some examples, the method further comprises the step of retrieving a position or orientation from a shared repository. Additionally or alternatively to one or more of the examples disclosed above, in some examples, the map data comprises one or more values retrieved from a shared repository. Additionally or alternatively to one or more of the examples disclosed above, in some examples, a position or orientation is determined using a neural network. Additionally or alternatively to one or more of the examples disclosed above, in some examples, the method further comprises determining a usefulness of a landmark. Additionally or alternatively to one or more of the examples disclosed above, in some examples, the step of determining, using the determined position or orientation of the vehicle relative to the landmark and the determined position or orientation of the landmark relative to the world coordinate system, a position or orientation of the vehicle relative to the world coordinate system further comprises using a determined usefulness of a landmark. Additionally or alternatively to one or more of the examples disclosed above, in some examples, the method further comprises the step of updating the map data using the sensor data. Additionally or alternatively to one or more of the examples disclosed above, in some examples in which the vehicle is an autonomous vehicle, the method further comprises the step of executing a driving operation using the determined position or orientation of the vehicle relative to the world coordinate system.
Some examples of the disclosure are directed to a system comprising: one or more sensors included with a vehicle, the one or more sensors configured to present sensor data; one or more processors coupled to the one or more sensors; and a memory including instructions, which when executed by the one or more processors, cause the one or more processors to perform a method comprising: identifying a landmark using the sensor data; identifying the landmark using map data relating to an approximate location of the vehicle in a world coordinate system; determining a position or orientation of the vehicle relative to the landmark; determining a position or orientation of the landmark relative to the world coordinate system; and determining, using the determined position or orientation of the vehicle relative to the landmark and the determined position or orientation of the landmark relative to the world coordinate system, a position or orientation of the vehicle relative to the world coordinate system.
Some examples of the disclosure are directed to a non-transitory machine-readable storage medium containing program instructions executable by a computer, the program instructions enabling the computer to perform: identifying a landmark using sensor data presented by one or more sensors included with a vehicle; identifying the landmark using map data relating to an approximate location of the vehicle in a world coordinate system; determining a position or orientation of the vehicle relative to the landmark; determining a position or orientation of the landmark relative to the world coordinate system; and determining, using the determined position or orientation of the vehicle relative to the landmark and the determined position or orientation of the landmark relative to the world coordinate system, a position or orientation of the vehicle relative to the world coordinate system.
Although examples of this disclosure have been fully described with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of examples of this disclosure as defined by the appended claims.
This application claims the benefit of U.S. Provisional Application No. 62/375,862, filed Aug. 16, 2016, the entirety of which is hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
62375862 | Aug 2016 | US |