Various embodiments of the present disclosure are generally directed to a method and apparatus for adaptively adjusting a resolution of a field of view (FoV) of a light detection and ranging (LiDAR) system.
Without limitation, some embodiments operate to generate an illumination profile identify a portion of a field of view (FoV) to which enhanced electromagnetic radiation is to be applied by an emitter of the LiDAR system. A scan profile is generated corresponding to the illumination profile, and the scan profile is applied to an output device of the emitter to produce the selected illumination profile upon targets in the FoV. The scan profile is generated in response to an external sensor that indicates a change in operational environment for the LiDAR system, such as a geopositioning sensor that detects a change in elevation or direction of a vehicle in which the LiDAR system is mounted. An observer and plant model can be incorporated into a servo control system to direct the scanning patterns.
These and other features and advantages of various embodiments can be understood from a review of the following detailed description in conjunction with the accompanying drawings.
Various embodiments of the present disclosure are generally directed to systems and methods for optimizing performance of an active light detection system.
Light Detection and Ranging (LiDAR) systems are useful in a number of applications in which range information (e.g., distance, etc.) associated with a target is determined by irradiating the target with electromagnetic radiation. One increasingly popular application for LiDAR is in the area of autonomously piloted or driver assisted vehicle guidance systems (e.g., self driving cars, autonomous drones, etc.). While not limiting, the light wavelengths used in a typical LiDAR system may range from ultraviolet to near infrared (e.g., 250 nanometers, nm to 1500 nm or more). Other wavelength ranges can be used.
One commonly employed form of LiDAR is sometimes referred to as coherent pulsed LiDAR, which generally uses coherent light and detects the range based on detecting phase differences in the reflected light. Such systems may use a dual (I/Q) channel detector with an I (in-phase) channel and a Q (quadrature) channel. Other forms of LiDAR systems can be used, however, including non-coherent light systems that may incorporate one or more detection channels. Further alternatives that can be incorporated into LiDAR systems include systems that sweep the emitted light using mechanical based systems such as rotatable mirrored polygons or micromirrors that utilize moveable mechanical elements, solid-state systems with no moving mechanical parts but instead use phase array mechanisms to sweep the emitted light in a direction toward the target, and so on.
While operable, these and other forms of LiDAR systems can have difficulty providing accurate detection resolution in all desired areas under all operating conditions. In particular, it may be desirable at times to provide enhanced detection to particular regions of interest within a particular field of view (FoV) of the system.
Various embodiments of the present disclosure are accordingly directed to a method and apparatus for providing enhanced detection capabilities in a LiDAR system. As explained below, some embodiments provide a closed loop servo control system that operates such that different areas of interest within a FoV of the system are adjusted to have different emitted beam point densities using different settings to account for different operational environments.
Operational environments that can be processed by various embodiments can include, but are not limited to, the following: (1) modeling vertical curvature of a road to set a vertical offset; (2) adjustments in vertical foveation based on limited point on area of interest; (3) selective focus once particular targets are identified (including object permanence, occupancy grid and other techniques); (4) horizontal resolution adjustments based on rotational velocity sensing (such as providing higher resolution to the path of a vehicle around a curve); (5) intersectional detection and enhanced sensing in side directions; (6) frame-to-frame foveation changes; (7) foveation changes based on road markings and other indicators; (8) minimum safe sample resolutions; (9) map integration operations; (10) vertical resolution changes based on system angle (such as during changes in elevation); and (11) setting different vertical (or other axial) spacing of LiDAR scanlines based on scan inputs and plant modeling. Other operational environments can be processed as well to provide optimal beam point densities for the system.
In further embodiments, the separation of beam scanlines is determined based on the scanning profile of an actuated mirror or other output directing device and thus the scanning profile directly impacts the resolution and its distribution for the LiDAR. The resolution across the scan can be optimized for regions of interest in the sensors field of view. Regions of interest may change position from frame to frame so it may be desirable for the servo system to quickly adjust to changing scanning profiles which may be non-linear.
The disclosed system and method make use of a feedforward system for tracking the profile using various servo techniques including plant inversion. The feedforward servo control system can include a plant model that models the response of the system and an observer module that observes and predicts the operation of the plant to provide inputs to obtain adjustments to provide suitable closed loop tracking of desired trajectories. In this way, system inputs, including inputs from external sensors, can be adaptively used to enact servo tracking modes automatically that provide the desired increases or decreases in system resolution during operation.
These and other features and advantages of various embodiments can be understood beginning with a review of
The LiDAR system 100 is configured to obtain range information regarding a target 102 that is located distal from the system 100. The information can be beneficial for a number of areas and applications including, but not limited to, topography, archeology, geology, surveying, geography, forestry, seismology, atmospheric physics, laser guidance, automated driving and guidance systems, closed-loop control systems, etc.
The LiDAR system 100 includes a controller 104 which provides top level control of the system. The controller 104 can take any number of desired configurations, including hardware and/or software. In some cases, the controller 104 can include the use of one or more programmable processors with associated programming (e.g., software, firmware) stored in a local memory which provides instructions that are executed by the programmable processor(s) during operation. Other forms of controllers can be used, including hardware based controllers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), system on chip (SOC) integrated circuits, application specific integrated circuits (ASICs), gate logic, reduced instruction set computers (RISCs), etc.
An energy source circuit 106, also sometimes referred to as an emitter or a transmitter, operates to direct electromagnetic radiation in the form of light pulses toward the target 102. A detector circuit 108, also sometimes referred to as a receiver or a sensor, senses reflected light pulses received back from the target 102. The controller 104 directs operation of the emitted light from the emitter 106, denoted by arrow 110, and decodes information from the reflected light obtained back from the target, as denoted by arrow 112.
Arrow 114 depicts the actual, true range information associated with the intervening distance (or other range parameter) between the LiDAR system 100 and the target 102. Depending on the configuration of the system, the range information can include the relative or absolute speed, velocity, acceleration, distance, size, location, reflectivity, color, surface features and/or other characteristics of the target 102 with respect to the system 100.
The decoded range information can be used to carry out any number of useful operations, such as controlling a motion, input or response of an autonomous vehicle, generating a topographical map, recording data into a data structure for further analysis and/or operations, etc. The controller 104 perform these operations directly, or can communicate the range information to an external system 116 for further processing and/or use.
In some cases, inputs supplied by the external system 116 can activate and configure the system to capture particular range information, which is then returned to the external system 116 by the controller 104. The external system can take any number of suitable forms, and may include a system controller (such as CPU 118), local memory 120, data storage device, etc. The external system may form a portion of a closed-loop control system and the range information output by the LiDAR system 100 can be used by the external system 116 to adjust the position of a moveable element.
The controller 104 can incorporate one or more programmable processors (CPU) 122 that execute program instructions in the form of software/firmware stored in a local memory 124, and which communicate with the external controller 118. External sensors 126 can provide further inputs used by the external system 116 and/or the LiDAR system 100.
The emitted light is processed by an output system 208 to issue a beam of emitted light 210. The light may be in the form of pulses, coherent light, non-coherent light, swept light, etc. The light can be issued in a rasterized pattern to provide frames of data. In some cases, the emitted and decoded light enable the controller to generate a three-dimensional (3D) point map or cloud representation of the surrounding environment within the field of view (FoV) of the system.
Regardless the configuration of the output system,
A low pass filter (LPF) 406 and an analog to digital converter (ADC) 408 can be used as desired to provide processing of the input pulses. A processing circuit 410 provides suitable signal processing operations to generate a useful output 412. Coherent and non-coherent detection strategies can be implemented as desired.
Various targets may be detected within the FoV 500 such as represented by targets 506, 508 and 510 (denoted as Targets 1-3). The targets are detected responsive to the reflected light provided by the emitter (e.g.,
In some cases, it may be desirable to not provide a consistent scanning density across the entirety of the FoV 500. Instead, it may be desirable to provide enhanced scans in different regions of interest. A number of different circumstances and system inputs can be provided to determine at such times when the focusing of the energy by the system is directed with greater intensity or other measure to portions of the FoV, while spending less of the overall available energy on other portions of the FoV.
For example, in the context of an automobile with LiDAR capabilities, it may make sense as the car goes down a hill to focus more of the expended electromagnetic radiation (e.g., scan lines, scan resolution, pulses, etc.) on targets that may be at a lower elevation than the car (as opposed to scanning the sky), on the basis that targets of interest will more likely be in lower portions of the FoV.
Similarly, if the same automobile is climbing a hill, it would make sense to focus more of the expended electromagnetic radiation on targets that are elevationally higher than the car on the basis that as the car climbs the hill, targets of interest will tend to be in the upper portions of the FoV. Other use cases can be envisioned that might make other regions of interest (for example, if the automobile is sensed as making a left-handed turn, targets to the left in the FoV may be of more interest, etc.).
As noted above, a variety of different mechanisms are envisioned in the present disclosure to compensate for these and other aspects. These can include modeling or other analytical aspects to determine appropriate scanning features of the light over the FoV. The operational modes that can be implemented under different circumstances to account for operational environments include but not limited to the following: (1) modeling vertical curvature of a road to set a vertical offset; (2) adjustments in vertical foveation based on limited point on area of interest; (3) selective focus once particular targets are identified (including object permanence, occupancy grid and other techniques); (4) horizontal resolution adjustments based on rotational velocity sensing (such as providing higher resolution to the path of a vehicle around a curve); (5) intersectional detection and enhanced sensing in side directions; (6) frame-to-frame foveation changes; (7) foveation changes based on road markings and other indicators; (8) minimum safe sample resolutions; (9) map integration operations; (10) vertical resolution changes based on system angle (such as during changes in elevation); and (11) setting different vertical (or other axial) spacing of LiDAR scanlines based on scan inputs and plant modeling.
Other operational environments can be processed as well to provide optimal beam point densities for the system. Some embodiments can account for all of these listed operational environments, while other embodiments may focus on only a single one or a subset of these (or other) environments.
The first region (Region 1) 802 is scanned using beam points 806 that are arranged (rasterized) along orthogonal x (horizontal) and y (vertical) axes as indicated by rows 808 and columns 810. Region 1 generally occupies the upper half of the overall FoV 800, although the respective sizes of the respective regions can vary as desired (e.g., Region 1 can occupy the top quarter of the FoV, the top two-thirds of the FoV, etc.).
The beams 806 are continuously and repetitively scanned using a suitable rasterization pattern over each frame so that all of the area of the region is covered by the scan points 806 in each frame. In some cases, a horizontal pattern is used (e.g. each row 806 is scanned in turn); in other cases, a vertical pattern is used (e.g., each column 810 is scanned in turn); in still other cases, a serpentine or other scan pattern is used. As will be recognized, the scanning of each frame within Region 1 is provided many times per unit of time (e.g., many thousands or millions of frames per second or more) to detect and track targets (see
The second region 804, denoted as Region 2, similarly receives a rasterized scanning pattern of beam points 816. The beams 816 may be nominally the same as the beams 806, or may have different waveform characteristics. As before, the beams 816 are rasterized along orthogonal x-y axes indicated by rows 818 and columns 820. In some cases, one rasterizing pattern is applied to Region 1 (e.g., on a per-row basis) and a different rasterizing pattern is applied to Region 2 (e.g., on a per-column basis). The overall density of the scanning pattern of Region 2 is lower than the density of the scanning pattern of region 1. This change in density can be accomplished in a number of ways, including by the use of a lower frame rate, a different number of beam points per row/column, a lower amplitude of the respective pulses, etc. as desired.
Regardless, it will be understood from
The basis for the division between Region 1 and Region 2 in
The Region 1 area 852 is rasterized using beam points 856 which, as before, are arranged along orthogonal axes such as x-y Cartesian axes provided by rows 858 and columns 860. The Region 2 area 854 is similarly rasterized using beam points 866 arranged along rows 868 and columns 870. The rows and columns in the respective regions may be aligned or may be offset as required. As before, Region 1 has a significantly greater beam density as compared to Region 2.
While
A closed loop servo control circuit 906 operates in at least some embodiments using a plant model 906 to provide inputs to an actuator 910, which in turn electrically and/or mechanically operates upon the beam output device 904 to direct the output beam scan pattern for the various FoV regions such as represented in
In some cases, the system can be arranged to provide a baseline scan operation over an overall FoV. Based on detected environmental conditions, an illumination profile can be identified to identify an area in which enhanced density scanning should be applied (e.g., such as the respective Region 1 areas in
In response to this illumination profile, the servo control circuit can operate to generate and output a corresponding scan profile, which is then used to direct the beam to provide the enhanced scan resolution to the region of interest.
Reflected from the target is a received set of pulses 1012 including pulses 1014 (pulse P1) and 1016 (pulse P2). The time of flight (TOF) value for pulse P1 is denoted at 1018. Similar TOF values are provided for each pulse in turn.
The received P1 pulse 1014 will likely undergo frequency doppler shifting and other distortions as compared to the emitted P1 pulse 1004. The same is generally true for each successive sets of transmitted and received pulses such as the P2 pulses 1006, 1016. Nevertheless, the frequencies, phase and amplitudes of the received pulses 1014, 1016 will be processed as described above to enable the detector circuit to correctly match the respective pulses and obtain accurate distance and other range information.
In some cases, the emitted/received pulses such as P1 can represent the higher resolution pulses submitted to a first field (e.g., Region 1 in
A LiDAR system such as 100 in
Thereafter the system commences with normal operation at block 1106. Light pulses are transmitted to illuminate various targets within the FoV as described above using the emitters as variously described above. Reflected pulses from various targets within the baseline FoV are detected at block 1108 using a detector system as provided including at
An area of interest within the baseline FoV is next selected at 1010. This can be carried out based on a number of inputs, including range information obtained from 1108, external sensor information, user input, etc. Regardless, a particular field of interest is identified to receive enhanced scanning resolution. In response, the servo system (e.g.,
As noted above, the system can cycle to provide different scanning patterns for different areas as required.
The manager circuit 1202 uses a number of inputs including system configuration information, measured distance for various targets, various other sensed parameters from the system (including external sensors 126), history data accumulated during prior operation, and user selectable inputs. Other inputs can be used as desired.
The manager circuit 1202 uses these and other inputs to provide various outputs including accumulated history data 1204 and various profiles 1206, both of which can be stored in local memory such as 124 for future reference. The history data 1204 can be arranged as a data structure providing relevant history and system configuration information. The profiles 1206 can describe different pulse set configurations with different numbers of pulses at various frequencies and other configuration settings, as well as other appropriate gain levels, ranges and slopes for different sizes, types, distances and velocities of detected targets.
The manager circuit 1202 further operates to direct various control information to an emitter (transmitter Tx) 1208 and a detector (receiver Rx) 1210 to implement these respective profiles. It will be understood that the Tx and Rx 1208, 1210 correspond to the various emitters and detectors described above. Without limitation, the inputs to the Tx 1208 can alter the pulses being emitted in the area of interest (including actuation signals to selectively switch in the specially configured lens or other optical element), and the inputs to the Rx 1210 can include gain, timing and other information to equip the detector to properly decode the pulses from the enhanced resolution area of interest.
The adaptive scan window manager 1202 can include an observer 1212 that provides observed or estimated positions of the scanning element which is modeled by model 1214. In this way, closed loop servo control can be adaptively and effectively implemented to accommodate rapidly changing detected environmental conditions.
As described previously, different gain ranges can be selected and used for different targets within the same FoV. Closer targets within the point cloud can be provided with one range with a lower slope and magnitude values to obtain optimal resolution of the closer targets, while at the same time farther targets within the point cloud can be provided with one or more different gain ranges with higher slopes and/or different magnitude values to obtain optimal resolution of the farther targets.
While coherent, I/Q based systems have been contemplated as a basic environment in which various embodiments can be practiced, such are not necessarily required. Any number of different types of systems can be employed, including solid state, mechanical, micromirror technology, etc.
It is to be understood that even though numerous characteristics and advantages of various embodiments of the present disclosure have been set forth in the foregoing description, together with details of the structure and function of various embodiments of the disclosure, this detailed description is illustrative only, and changes may be made in detail, especially in matters of structure and arrangements of parts within the principles of the present disclosure to the full extent indicated by the broad general meaning of the terms in which the appended claims are expressed.
The present application makes a claim of domestic priority to U.S. Provisional Patent Application No. 63/217,995 filed Jul. 2, 2021, the contents of which are hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
63217995 | Jul 2021 | US |