Patent Application
20240426986
Light detection and ranging (lidar) is a technology that can be used to measure distances to remote targets. Typically, a lidar system includes a light source and an optical receiver. The light source can include, for example, a laser which emits light having a particular operating wavelength. The operating wavelength of a lidar system may lie, for example, in the infrared, visible, or ultraviolet portions of the electromagnetic spectrum. The light source emits light toward a target which scatters the light, and some of the scattered light is received back at the receiver. The system determines the distance to the target based on one or more characteristics associated with the received light. For example, the lidar system may determine the distance to the target based on the time of flight for a pulse of light emitted by the light source to travel to the target and back to the lidar system.
Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
A lidar system may have multiple problems caused by temperature drifts and other external factors. Both the laser power and the laser pulse delay (or optical delay) may drift with temperature and other external factors. The present application discloses laser power monitoring (LPM) techniques and laser pulse delay compensation techniques to compensate for the drifts caused by temperature or other external factors for improved performance of the lidar system. The disclosed laser power monitoring techniques and laser pulse delay compensation techniques utilize an internal target (IT).
An internal (reference) target has multiple benefits for an automotive pulsed lidar. This target is typically positioned at the end of the scan and is imaged periodically to provide reference range and pulse energy information that can be used in various compensation procedures. For example, large temperature variations may cause drifts in the reflectance and range estimates that will corrupt the radiometry of a point cloud. These drifts can be detected and measured on the reference target. In a multi-beam lidar, variations in energy and pulse delays between the beams can also be detected and corrected. In a variable-power lidar, where certain segments of the scene are imaged under a lower power, while selected regions of interest are imaged at a high power, the laser power can also be monitored on the internal target, provided that the various regions intersect the reference target. However, due to its proximity and placement, extracting accurate and robust information from the reference target is challenging. The present application describes sampling and signal processing techniques to extract accurate and precise pulse energy and range information from the lidar returns originating from an internally installed reference target.
In the present application, a lidar system is disclosed. The system comprises a light source configured to emit light pulses. The system comprises a scanner configured to scan the emitted light pulses across an internal reference target internal to the system. The system comprises a detector configured to detect light that is at least a portion of light scattered by the internal reference target from at least a portion of the emitted light pulses. The system comprises a processor configured to selectively gather detected optical property values of the detected light corresponding to a selective portion of the emitted light pulses scanned across the internal reference target and use the selectively gathered detected optical property values to determine one or more calibration values.
In various embodiments, a lidar system comprises at least a light source and a scanner. The light source is configured to emit light, and the scanner is configured to scan the emitted light. For example, a light source such as a laser can emit light of a particular operating wavelength that is scanned by the scanner towards a particular scan region, such as an area in front of a vehicle and/or along the sides or behind the vehicle. The emitted light passes through a light sensor window before reaching any downrange objects. In some embodiments, the sensor window is used at least in part to protect the lidar system, for example, from environmental elements such as road debris and weather. In various embodiments, the lidar system includes multiple detectors, for example, as part of a receiver module. The detectors can utilize different types of technology to detect scattered light. For example, one or more of the detectors can utilize avalanche photodiodes (APDs), one or more single-photon avalanche diodes (SPADs), one or more PN photodiodes, or one or more PIN photodiodes, etc. In some embodiments, a first detector of the lidar system is configured to detect at least a portion of the emitted light scattered by a target located downrange from the system. For example, the positioning of the first detector may be optimized to detect objects that are a certain distance from the lidar system and/or the application environment to which the lidar system is mounted, such as an automobile. In some embodiments, the first detector is optimized to detect objects that are up to 50 meters, 200 meters, or another distance away from the lidar system. In some embodiments, a second detector of the lidar system is configured to detect at least a portion of the emitted light scattered by an internal reference target to provide reference range and pulse energy information that can be used in various compensation procedures. In various embodiments, the first and second detectors are different detectors and are configured to detect different scatter patterns. One or more processors of the lidar system can be configured to analyze detected information from the detectors.
A lidar system may operate in a vehicle and includes multiple “eyes,” each of which has its own field of regard, or an angular range over which the eye scans targets using pulses of light in accordance with a scan pattern. The fields of regard can combine along a certain dimension (e.g., horizontally) to define the overall field of regard of the lidar system. The lidar system then can use data received via both eyes to generate a point cloud or otherwise process the received data. In a two-eye configuration of the lidar system, the two eyes can be housed together and scan the respective fields of regard via a shared window or separate windows, or the eyes can be housed separately. In the latter case, an assembly referred to as a “sensor head” can include a scanner, a receiver, and an optical element such as a collimator or a laser diode to generate or convey a beam of light.
Depending on the implementation, each eye of a lidar system can include a separate scanner. For example, each eye can be equipped with a pivotable scan mirror to scan the field of regard vertically and another pivotable scan mirror to scan the field of regard horizontally. Each eye of the lidar system can include a partially shared scanner. For example, each eye can be equipped with a pivotable scan mirror to scan the field of regard vertically, and a shared polygon mirror can scan the corresponding fields of regard horizontally, using different reflective surfaces. Each eye of the lidar system can include a fully shared scanner. For example, a pivotable planar mirror can scan the fields of regard vertically by reflecting incident beams at different regions on the reflective surface, and a shared polygon mirror can scan the corresponding fields of regard horizontally, using different reflective surfaces.
Different hardware configurations allow the lidar system to operate the eyes more independently of each other, as is the case with separate scanners, or less independently, as is the case with a fully shared scanner. For example, the two or more eyes may scan the respective fields of regard using similar or different scan patterns. In one implementation, the two eyes trace out the same pattern, but with a certain time differential to maintain angular separation between light-source fields of view and thereby reduce the probability of cross-talk events between the sensor heads. In another implementation, the two eyes scan the corresponding fields of regard according to different scan patterns, at least in some operational states (e.g., when the vehicle is turning right or left). Further, according to one approach, two eyes of a lidar system are arranged so that the fields of regard of the eyes are adjacent and non-overlapping. For example, each field of regard can span approximately 60 degrees horizontally and 30 degrees vertically, so that the combined field of regard of the lidar system spans approximately 120 degrees horizontally and 30 degrees vertically. The corresponding scanners (or paths within a shared scanner) can point away from each other at a certain angle, for example, so that the respective fields of regard abut approximately at an axis corresponding to the forward-facing direction of the vehicle.
Alternatively, the lidar system can operate in a “cross-eyed” configuration to create an area of overlap between the fields of regard. The area of overlap can be approximately centered along the forward-facing direction or another direction, which in some implementations a controller can determine dynamically. In this implementation, the two sensor heads can yield a higher density of scan in the area that generally is more important. In some implementations, the fields of regard in a cross-eyed two-eye configuration are offset from each other by a half-pixel value, so that the area of overlap has twice as many pixels. In general, the fields of regard can overlap angularly or translationally. To reduce the probability of cross-talk events (e.g., the situation when a pulse emitted by the light source associated with the first eye is received by the receiver of the second eye), the lidar system can use output beams with different wavelengths.
Once the output beam 125 reaches the downrange target 130, the target may scatter or reflect at least a portion of light from the output beam 125, and some of the scattered or reflected light may return toward the lidar system 100. In the example of
In particular embodiments, output beam 125 may include or may be referred to as an optical signal, output optical signal, emitted optical signal, output light, emitted pulse of light, laser beam, light beam, optical beam, emitted beam, emitted light, or beam. In particular embodiments, input beam 135 may include or may be referred to as a received optical signal, received pulse of light, input pulse of light, input optical signal, return beam, received beam, return light, received light, input light, scattered light, or reflected light. As used herein, scattered light may refer to light that is scattered or reflected by a target 130. As an example, an input beam 135 may include: light from the output beam 125 that is scattered by target 130; light from the output beam 125 that is reflected by target 130; or a combination of scattered and reflected light from target 130.
In various embodiments, lidar system 100 includes sensor window 157 through which the beams 125 and 135 pass. In some embodiments, one or more relatively high reflectivity regions (e.g., reflectivity of 1% to 5%) are located on sensor window 157 and can be utilized for calibration purposes (e.g., pulse energy and optical axis drift). The reflectivity regions can be of various sizes (e.g., small dots to large zones) depending on the type of calibration. Output beam 125 can be directed toward the reflectivity regions to perform calibration. In many scenarios, the reflectivity regions are not encountered while scanning a typical field of regard using output beam 125. Stated alternatively, the reflectivity regions do not interfere with normal operation of lidar system 100 when calibration using the reflectivity regions is not desired. When the reflectivity regions are utilized for calibration, the output beam of light source 110 may scatter or reflect at least a portion of light from the output beam 125. Some of the scattered or reflected light may return toward the lidar system 100. The light scattered or reflected can take a path similar to input beam 135. The light may pass through scanner 120 and be reflected by mirror 115 and directed to receiver 140.
In particular embodiments, receiver 140 may receive or detect photons from input beam 135 and produce one or more representative signals. For example, the receiver 140 may produce an output electrical signal 145 that is representative of the input beam 135, and the electrical signal 145 may be sent to controller 150. In particular embodiments, receiver 140 or controller 150 may include a processor, computing system (e.g., an ASIC or FPGA), or other suitable circuitry. A controller 150 may be configured to analyze one or more characteristics of the electrical signal 145 from the receiver 140 to determine one or more characteristics of the target 130, such as its distance downrange from the lidar system 100. This may be done, for example, by analyzing a time of flight or a frequency or phase of a transmitted beam of light 125 or a received beam of light 135. If lidar system 100 measures a time of flight of T (e.g., T represents a round-trip time of flight for an emitted pulse of light to travel from the lidar system 100 to the target 130 and back to the lidar system 100), then the distance D from the target 130 to the lidar system 100 may be expressed as D=c. T/2, where c is the speed of light (approximately 3.0×108 m/s). As an example, if a time of flight is measured to be T=300 ns, then the distance from the target 130 to the lidar system 100 may be determined to be approximately D=45.0 m. As another example, if a time of flight is measured to be T=1.33 μs, then the distance from the target 130 to the lidar system 100 may be determined to be approximately D=199.5 m. In particular embodiments, a distance D from lidar system 100 to a target 130 may be referred to as a distance, depth, or range of target 130. As used herein, the speed of light c refers to the speed of light in any suitable medium, such as for example in air, water, or vacuum. As an example, the speed of light in vacuum is approximately 2.9979×108 m/s, and the speed of light in air (which has a refractive index of approximately 1.0003) is approximately 2.9970×108 m/s.
In particular embodiments, light source 110 may include a pulsed or CW laser. As an example, light source 110 may be a pulsed laser configured to produce or emit pulses of light with a pulse duration or pulse width of approximately 10 picoseconds (ps) to 100 nanoseconds (ns). The pulses may have a pulse duration of approximately 100 ps, 200 ps, 400 ps, 1 ns, 2 ns, 5 ns, 10 ns, 20 ns, 50 ns, 100 ns, or any other suitable pulse duration. As another example, light source 110 may be a pulsed laser that produces pulses with a pulse duration of approximately 1-5 ns. As another example, light source 110 may be a pulsed laser that produces pulses at a pulse repetition frequency of approximately 100 kHz to 10 MHz or a pulse period (e.g., a time between consecutive pulses) of approximately 100 ns to 10 μs. In particular embodiments, light source 110 may have a substantially constant pulse repetition frequency, or light source 110 may have a variable or adjustable pulse repetition frequency. As an example, light source 110 may be a pulsed laser that produces pulses at a substantially constant pulse repetition frequency of approximately 640 kHz (e.g., 640,000 pulses per second), corresponding to a pulse period of approximately 1.56 μs. As another example, light source 110 may have a pulse repetition frequency (which may be referred to as a repetition rate) that can be varied from approximately 200 kHz to 3 MHz. As used herein, a pulse of light may be referred to as an optical pulse, a light pulse, or a pulse.
In particular embodiments, light source 110 may include a pulsed or CW laser that produces a free-space output beam 125 having any suitable average optical power. As an example, output beam 125 may have an average power of approximately 1 milliwatt (mW), 10 mW, 100 mW, 1 watt (W), 10 W, or any other suitable average power. In particular embodiments, output beam 125 may include optical pulses with any suitable pulse energy or peak optical power. As an example, output beam 125 may include pulses with a pulse energy of approximately 0.01 μJ, 0.1 μJ, 0.5 μJ, 1 μJ, 2 μJ, 10 μJ, 100 μJ, 1 mJ, or any other suitable pulse energy. As another example, output beam 125 may include pulses with a peak power of approximately 10 W, 100 W, 1 KW, 5 KW, 10 KW, or any other suitable peak power. The peak power (Ppeak) of a pulse of light can be related to the pulse energy (I) by the expression E=Ppeak·Δt, where Δt is the duration of the pulse, and the duration of a pulse may be defined as the full width at half maximum duration of the pulse. For example, an optical pulse with a duration of 1 ns and a pulse energy of 1 μJ has a peak power of approximately 1 kW. The average power (Pav) of an output beam 125 can be related to the pulse repetition frequency (PRF) and pulse energy by the expression Pav=PRF·E. For example, if the pulse repetition frequency is 500 kHz, then the average power of an output beam 125 with 1-μJ pulses is approximately 0.5 W.
In particular embodiments, light source 110 may include a laser diode, such as for example, a Fabry-Perot laser diode, a quantum well laser, a distributed Bragg reflector (DBR) laser, a distributed feedback (DFB) laser, a vertical-cavity surface-emitting laser (VCSEL), a quantum dot laser diode, a grating-coupled surface-emitting laser (GCSEL), a slab-coupled optical waveguide laser (SCOWL), a single-transverse-mode laser diode, a multi-mode broad area laser diode, a laser-diode bar, a laser-diode stack, or a tapered-stripe laser diode. As an example, light source 110 may include an aluminum-gallium-arsenide (AlGaAs) laser diode, an indium-gallium-arsenide (InGaAs) laser diode, an indium-gallium-arsenide-phosphide (InGaAsP) laser diode, or a laser diode that includes any suitable combination of aluminum (Al), indium (In), gallium (Ga), arsenic (As), phosphorous (P), or any other suitable material. In particular embodiments, light source 110 may include a pulsed or CW laser diode with a peak emission wavelength between 1200 nm and 1600 nm. As an example, light source 110 may include a current-modulated InGaAsP DFB laser diode that produces optical pulses at a wavelength of approximately 1550 nm. As another example, light source 110 may include a laser diode that emits light at a wavelength between 1500 nm and 1510 nm.
In particular embodiments, light source 110 may include a pulsed or CW laser diode followed by one or more optical-amplification stages. For example, a seed laser diode may produce a seed optical signal, and an optical amplifier may amplify the seed optical signal to produce an amplified optical signal that is emitted by the light source 110. In particular embodiments, an optical amplifier may include a fiber-optic amplifier or a semiconductor optical amplifier (SOA). For example, a pulsed laser diode may produce relatively low-power optical seed pulses which are amplified by a fiber-optic amplifier. As another example, a light source 110 may include a fiber-laser module that includes a current-modulated laser diode with an operating wavelength of approximately 1550 nm followed by a single-stage or a multi-stage erbium-doped fiber amplifier (EDFA) or erbium-ytterbium-doped fiber amplifier (EYDFA) that amplifies the seed pulses from the laser diode. As another example, light source 110 may include a continuous-wave (CW) or quasi-CW laser diode followed by an external optical modulator (e.g., an electro-optic amplitude modulator). The optical modulator may modulate the CW light from the laser diode to produce optical pulses which are sent to a fiber-optic amplifier or SOA. As another example, light source 110 may include a pulsed or CW seed laser diode followed by a semiconductor optical amplifier (SOA). The SOA may include an active optical waveguide configured to receive light from the seed laser diode and amplify the light as it propagates through the waveguide. The optical gain of the SOA may be provided by pulsed or direct-current (DC) electrical current supplied to the SOA. The SOA may be integrated on the same chip as the seed laser diode, or the SOA may be a separate device with an anti-reflection coating on its input facet or output facet. As another example, light source 110 may include a seed laser diode followed by an SOA, which in turn is followed by a fiber-optic amplifier. For example, the seed laser diode may produce relatively low-power seed pulses which are amplified by the SOA, and the fiber-optic amplifier may further amplify the optical pulses.
In particular embodiments, light source 110 may include a direct-emitter laser diode. A direct-emitter laser diode (which may be referred to as a direct emitter) may include a laser diode which produces light that is not subsequently amplified by an optical amplifier. A light source 110 that includes a direct-emitter laser diode may not include an optical amplifier, and the output light produced by a direct emitter may not be amplified after it is emitted by the laser diode. The light produced by a direct-emitter laser diode (e.g., optical pulses, CW light, or frequency-modulated light) may be emitted directly as a free-space output beam 125 without being amplified. A direct-emitter laser diode may be driven by an electrical power source that supplies current pulses to the laser diode, and each current pulse may result in the emission of an output optical pulse.
In particular embodiments, light source 110 may include a diode-pumped solid-state (DPSS) laser. A DPSS laser (which may be referred to as a solid-state laser) may refer to a laser that includes a solid-state, glass, ceramic, or crystal-based gain medium that is pumped by one or more pump laser diodes. The gain medium may include a host material that is doped with rare-earth ions (e.g., neodymium, erbium, ytterbium, or praseodymium). For example, a gain medium may include a yttrium aluminum garnet (YAG) crystal that is doped with neodymium (Nd) ions, and the gain medium may be referred to as a Nd:YAG crystal. A DPSS laser with a Nd:YAG gain medium may produce light at a wavelength between approximately 1300 nm and approximately 1400 nm, and the Nd:YAG gain medium may be pumped by one or more pump laser diodes with an operating wavelength between approximately 730 nm and approximately 900 nm. A DPSS laser may be a passively Q-switched laser that includes a saturable absorber (e.g., a vanadium-doped crystal that acts as a saturable absorber). Alternatively, a DPSS laser may be an actively Q-switched laser that includes an active Q-switch (e.g., an acousto-optic modulator or an electro-optic modulator). A passively or actively Q-switched DPSS laser may produce output optical pulses that form an output beam 125 of a lidar system 100.
In particular embodiments, an output beam of light 125 emitted by light source 110 may be a collimated optical beam having any suitable beam divergence, such as for example, a full-angle beam divergence of approximately 0.5 to 10 milliradians (mrad). A divergence of output beam 125 may refer to an angular measure of an increase in beam size (e.g., a beam radius or beam diameter) as output beam 125 travels away from light source 110 or lidar system 100. In particular embodiments, output beam 125 may have a substantially circular cross section with a beam divergence characterized by a single divergence value. As an example, an output beam 125 with a circular cross section and a full-angle beam divergence of 2 mrad may have a beam diameter or spot size of approximately 20 cm at a distance of 100 m from lidar system 100. In particular embodiments, output beam 125 may have a substantially elliptical cross section characterized by two divergence values. As an example, output beam 125 may have a fast axis and a slow axis, where the fast-axis divergence is greater than the slow-axis divergence. As another example, output beam 125 may be an elliptical beam with a fast-axis divergence of 4 mrad and a slow-axis divergence of 2 mrad.
In particular embodiments, an output beam of light 125 emitted by light source 110 may be unpolarized or randomly polarized, may have no specific or fixed polarization (e.g., the polarization may vary with time), or may have a particular polarization (e.g., output beam 125 may be linearly polarized, elliptically polarized, or circularly polarized). As an example, light source 110 may produce light with no specific polarization or may produce light that is linearly polarized.
In particular embodiments, lidar system 100 may include one or more optical components configured to reflect, focus, filter, shape, modify, steer, or direct light within the lidar system 100 or light produced or received by the lidar system 100 (e.g., output beam 125 or input beam 135). As an example, lidar system 100 may include one or more lenses, mirrors, filters (e.g., band-pass or interference filters), beam splitters, optical splitters, polarizers, polarizing beam splitters, wave plates (e.g., half-wave or quarter-wave plates), diffractive elements, holographic elements, isolators, couplers, detectors, beam combiners, or collimators. The optical components in a lidar system 100 may be free-space optical components, fiber-coupled optical components, or a combination of free-space and fiber-coupled optical components.
In particular embodiments, lidar system 100 may include a telescope, one or more lenses, or one or more mirrors configured to expand, focus, or collimate the output beam 125 or the input beam 135 to a desired beam diameter or divergence. As an example, the lidar system 100 may include one or more lenses to focus the input beam 135 onto a photodetector of receiver 140. As another example, the lidar system 100 may include one or more flat mirrors or curved mirrors (e.g., concave, convex, or parabolic mirrors) to steer or focus the output beam 125 or the input beam 135. For example, the lidar system 100 may include an off-axis parabolic mirror to focus the input beam 135 onto a photodetector of receiver 140. As illustrated in
In particular embodiments, mirror 115 may provide for output beam 125 and input beam 135 to be substantially coaxial so that the two beams travel along approximately the same optical path (albeit in opposite directions). The input and output beams being substantially coaxial may refer to the beams being at least partially overlapped or sharing a common propagation axis so that input beam 135 and output beam 125 travel along substantially the same optical path (albeit in opposite directions). As an example, output beam 125 and input beam 135 may be parallel to each other to within less than 10 mrad, 5 mrad, 2 mrad, 1 mrad, 0.5 mrad, or 0.1 mrad. As output beam 125 is scanned across a field of regard, the input beam 135 may follow along with the output beam 125 so that the coaxial relationship between the two beams is maintained.
In particular embodiments, lidar system 100 may include a scanner 120 configured to scan an output beam 125 across a field of regard of the lidar system 100. As an example, scanner 120 may include one or more scanning mirrors configured to pivot, rotate, oscillate, or move in an angular manner about one or more rotation axes. The output beam 125 may be reflected by a scanning mirror, and as the scanning mirror pivots or rotates, the reflected output beam 125 may be scanned in a corresponding angular manner. As an example, a scanning mirror may be configured to periodically pivot back and forth over a 30-degree range, which results in the output beam 125 scanning back and forth across a 60-degree range (e.g., a 0-degree rotation by a scanning mirror results in a 20-degree angular scan of output beam 125).
In particular embodiments, a scanning mirror (which may be referred to as a scan mirror) may be attached to or mechanically driven by a scanner actuator or mechanism which pivots or rotates the mirror over a particular angular range (e.g., over a 5° angular range, 30° angular range, 60° angular range, 120° angular range, 360° angular range, or any other suitable angular range). A scanner actuator or mechanism configured to pivot or rotate a mirror may include a galvanometer scanner, a resonant scanner, a piezoelectric actuator, a voice coil motor, an electric motor (e.g., a DC motor, a brushless DC motor, a synchronous electric motor, or a stepper motor), a microelectromechanical systems (MEMS) device, or any other suitable actuator or mechanism. As an example, a scanner 120 may include a scanning mirror attached to a galvanometer scanner configured to pivot back and forth over a 1° to 30° angular range. As another example, a scanner 120 may include a scanning mirror that is attached to or is part of a MEMS device configured to scan over a 1° to 30° angular range. As another example, a scanner 120 may include a polygon mirror configured to rotate continuously in the same direction (e.g., rather than pivoting back and forth, the polygon mirror continuously rotates 360 degrees in a clockwise or counterclockwise direction). The polygon mirror may be coupled or attached to a synchronous motor configured to rotate the polygon mirror at a substantially fixed rotational frequency (e.g., a rotational frequency of approximately 1 Hz, 10 Hz, 50 Hz, 100 Hz, 500 Hz, or 1,000 Hz).
In particular embodiments, scanner 120 may be configured to scan the output beam 125 (which may include at least a portion of the light emitted by light source 110) across a field of regard of the lidar system 100. A field of regard (FOR) of a lidar system 100 may refer to an area, region, or angular range over which the lidar system 100 may be configured to scan or capture distance information. As an example, a lidar system 100 with an output beam 125 with a 30-degree scanning range may be referred to as having a 30-degree angular field of regard. As another example, a lidar system 100 with a scanning mirror that rotates over a 30-degree range may produce an output beam 125 that scans across a 60-degree range (e.g., a 60-degree fOR). In particular embodiments, lidar system 100 may have a FOR of approximately 10°, 20°, 40°, 60°, 120°, 360°, or any other suitable FOR.
In particular embodiments, scanner 120 may be configured to scan the output beam 125 horizontally and vertically, and lidar system 100 may have a particular FOR along the horizontal direction and another particular FOR along the vertical direction. As an example, lidar system 100 may have a horizontal FOR of 10° to 120° and a vertical FOR of 2° to 45°. In particular embodiments, scanner 120 may include a first scan mirror and a second scan mirror, where the first scan mirror directs the output beam 125 toward the second scan mirror, and the second scan mirror directs the output beam 125 downrange from the lidar system 100. As an example, the first scan mirror may scan the output beam 125 along a first direction, and the second scan mirror may scan the output beam 125 along a second direction that is substantially orthogonal to the first direction. As another example, the first scan mirror may scan the output beam 125 along a substantially horizontal direction, and the second scan mirror may scan the output beam 125 along a substantially vertical direction (or vice versa). As another example, the first and second scan mirrors may each be driven by galvanometer scanners. As another example, the first or second scan mirror may include a polygon mirror driven by an electric motor. In particular embodiments, scanner 120 may be referred to as a beam scanner, optical scanner, or laser scanner.
In particular embodiments, one or more scanning mirrors may be communicatively coupled to controller 150 which may control the scanning mirror(s) so as to guide the output beam 125 in a desired direction downrange or along a desired scan pattern. In particular embodiments, a scan pattern may refer to a pattern or path along which the output beam 125 is directed. As an example, scanner 120 may include two scanning mirrors configured to scan the output beam 125 across a 60° horizontal FOR and a 20° vertical FOR. The two scanner mirrors may be controlled to follow a scan path that substantially covers the 60°×20° FOR. As an example, the scan path may result in a point cloud with pixels that substantially cover the 60°×20° FOR. The pixels may be approximately evenly distributed across the 60°×20° FOR. Alternatively, the pixels may have a particular nonuniform distribution (e.g., the pixels may be distributed across all or a portion of the 60°×20° FOR, and the pixels may have a higher density in one or more particular regions of the 60°×20° FOR).
In particular embodiments, a lidar system 100 may include a scanner 120 with a solid-state scanning device. A solid-state scanning device may refer to a scanner 120 that scans an output beam 125 without the use of moving parts (e.g., without the use of a mechanical scanner, such as a mirror that rotates or pivots). For example, a solid-state scanner 120 may include one or more of the following: an optical phased array scanning device; a liquid-crystal scanning device; or a liquid lens scanning device. A solid-state scanner 120 may be an electrically addressable device that scans an output beam 125 along one axis (e.g., horizontally) or along two axes (e.g., horizontally and vertically). In particular embodiments, a scanner 120 may include a solid-state scanner and a mechanical scanner. For example, a scanner 120 may include an optical phased array scanner configured to scan an output beam 125 in one direction and a galvanometer scanner that scans the output beam 125 in an orthogonal direction. The optical phased array scanner may scan the output beam relatively rapidly in a horizontal direction across the field of regard (e.g., at a scan rate of 50 to 1,000 scan lines per second), and the galvanometer may pivot a mirror at a rate of 1 30 Hz to scan the output beam 125 vertically.
In particular embodiments, a lidar system 100 may include a light source 110 configured to emit pulses of light and a scanner 120 configured to scan at least a portion of the emitted pulses of light across a field of regard of the lidar system 100. One or more of the emitted pulses of light may be scattered by a target 130 located downrange from the lidar system 100, and a receiver 140 may detect at least a portion of the pulses of light scattered by the target 130. A receiver 140 may be referred to as a photoreceiver, optical receiver, optical sensor, detector, photodetector, or optical detector. In particular embodiments, lidar system 100 may include a receiver 140 that receives or detects at least a portion of input beam 135 and produces an electrical signal that corresponds to input beam 135. As an example, if input beam 135 includes an optical pulse, then receiver 140 may produce an electrical current or voltage pulse that corresponds to the optical pulse detected by receiver 140. As another example, receiver 140 may include one or more avalanche photodiodes (APDs) or one or more single-photon avalanche diodes (SPADs). As another example, receiver 140 may include one or more PN photodiodes (e.g., a photodiode structure formed by a p-type semiconductor and an n-type semiconductor, where the PN acronym refers to the structure having p-doped and n-doped regions) or one or more PIN photodiodes (e.g., a photodiode structure formed by an undoped intrinsic semiconductor region located between p-type and n-type regions, where the PIN acronym refers to the structure having p-doped, intrinsic, and n-doped regions). An APD, SPAD, PN photodiode, or PIN photodiode may each be referred to as a detector, photodetector, or photodiode. A detector may have an active region or an avalanche-multiplication region that includes silicon, germanium, silicon germanium, InGaAs, InAsSb (indium arsenide antimonide), AlAsSb (aluminum arsenide antimonide), or AlInAsSb (aluminum indium arsenide antimonide). The active region may refer to an area over which a detector may receive or detect input light. An active region may have any suitable size or diameter, such as for example, a diameter of approximately 10 μm, 25 μm, 50 μm, 80 μm, 100 μm, 200 μm, 500 μm, 1 mm, 2 mm, or 5 mm.
In particular embodiments, receiver 140 may include electronic circuitry that performs signal amplification, sampling, filtering, signal conditioning, analog-to-digital conversion, time-to-digital conversion, pulse detection, threshold detection, rising-edge detection, or falling-edge detection. As an example, receiver 140 may include a transimpedance amplifier that converts a received photocurrent (e.g., a current produced by an APD in response to a received optical signal) into a voltage signal. The voltage signal may be sent to pulse-detection circuitry that produces an analog or digital output signal 145 that corresponds to one or more optical characteristics (e.g., rising edge, falling edge, amplitude, duration, or energy) of a received optical pulse. As an example, the pulse-detection circuitry may perform a time-to-digital conversion to produce a digital output signal 145. The electrical output signal 145 may be sent to controller 150 for processing or analysis (e.g., to determine a time-of-flight value corresponding to a received optical pulse).
In particular embodiments, a controller 150 (which may include or may be referred to as a processor, an FPGA, an ASIC, a computer, or a computing system) may be located within a lidar system 100 or outside of a lidar system 100. Alternatively, one or more parts of a controller 150 may be located within a lidar system 100, and one or more other parts of a controller 150 may be located outside a lidar system 100. In particular embodiments, one or more parts of a controller 150 may be located within a receiver 140 of a lidar system 100, and one or more other parts of a controller 150 may be located in other parts of the lidar system 100. For example, a receiver 140 may include an FPGA or ASIC configured to process an output electrical signal from the receiver 140, and the processed signal may be sent to a computing system located elsewhere within the lidar system 100 or outside the lidar system 100. In particular embodiments, a controller 150 may include any suitable arrangement or combination of logic circuitry, analog circuitry, or digital circuitry.
In particular embodiments, controller 150 may be electrically coupled or communicatively coupled to light source 110, scanner 120, or receiver 140. As an example, controller 150 may receive electrical trigger pulses or edges from light source 110, where each pulse or edge corresponds to the emission of an optical pulse by light source 110. As another example, controller 150 may provide instructions, a control signal, or a trigger signal to light source 110 indicating when light source 110 should produce optical pulses. Controller 150 may send an electrical trigger signal that includes electrical pulses, where each electrical pulse results in the emission of an optical pulse by light source 110. In particular embodiments, the frequency, period, duration, pulse energy, peak power, average power, or wavelength of the optical pulses produced by light source 110 may be adjusted based on instructions, a control signal, or trigger pulses provided by controller 150. In particular embodiments, controller 150 may be coupled to light source 110 and receiver 140, and controller 150 may determine a time-of-flight value for an optical pulse based on timing information associated with when the pulse was emitted by light source 110 and when a portion of the pulse (e.g., input beam 135) was detected or received by receiver 140. In particular embodiments, controller 150 may include circuitry that performs signal amplification, sampling, filtering, signal conditioning, analog-to-digital conversion, time-to-digital conversion, pulse detection, threshold detection, rising-edge detection, or falling-edge detection.
In particular embodiments, lidar system 100 may include one or more processors (e.g., a controller 150) configured to determine a distance D from the lidar system 100 to a target 130 based at least in part on a round-trip time of flight for an emitted pulse of light to travel from the lidar system 100 to the target 130 and back to the lidar system 100. The target 130 may be at least partially contained within a field of regard of the lidar system 100 and located a distance D from the lidar system 100 that is less than or equal to an operating range (ROP) of the lidar system 100. In particular embodiments, an operating range (which may be referred to as an operating distance) of a lidar system 100 may refer to a distance over which the lidar system 100 is configured to sense or identify targets 130 located within a field of regard of the lidar system 100. The operating range of lidar system 100 may be any suitable distance, such as for example, 25 m, 50 m, 100 m, 200 m, 250 m, 500 m, or 1 km. As an example, a lidar system 100 with a 200-m operating range may be configured to sense or identify various targets 130 located up to 200 m away from the lidar system 100. The operating range ROP of a lidar system 100 may be related to the time t between the emission of successive optical signals by the expression ROP=c·τ/2. For a lidar system 100 with a 200-m operating range (ROP=200 m), the time t between successive pulses (which may be referred to as a pulse period, a pulse repetition interval (PRI), or a time period between pulses) is approximately 2· ROP/c≅1.33 μs. The pulse period τ may also correspond to the time of flight for a pulse to travel to and from a target 130 located a distance ROP from the lidar system 100. Additionally, the pulse period t may be related to the pulse repetition frequency (PRF) by the expression τ=1/PRF. For example, a pulse period of 1.33 μs corresponds to a PRF of approximately 752 kHz.
In particular embodiments, a lidar system 100 may be used to determine the distance to one or more downrange targets 130. By scanning the lidar system 100 across a field of regard, the system may be used to map the distance to a number of points within the field of regard. Each of these depth-mapped points may be referred to as a pixel or a voxel. A collection of pixels captured in succession (which may be referred to as a depth map, a point cloud, or a frame) may be rendered as an image or may be analyzed to identify or detect objects or to determine a shape or distance of objects within the FOR. As an example, a point cloud may cover a field of regard that extends 60° horizontally and 15° vertically, and the point cloud may include a frame of 100-2000 pixels in the horizontal direction by 4-400 pixels in the vertical direction.
In particular embodiments, lidar system 100 may be configured to repeatedly capture or generate point clouds of a field of regard at any suitable frame rate between approximately 0.1 frames per second (FPS) and approximately 1,000 FPS. As an example, lidar system 100 may generate point clouds at a frame rate of approximately 0.1 FPS, 0.5 FPS, 1 FPS, 2 FPS, 5 FPS, 10 FPS, 20 FPS, 100 FPS, 500 FPS, or 1,000 FPS. As another example, lidar system 100 may be configured to produce optical pulses at a rate of 5×105 pulses/second (e.g., the system may determine 500,000 pixel distances per second) and scan a frame of 1000×50 pixels (e.g., 50,000 pixels/frame), which corresponds to a point-cloud frame rate of 10 frames per second (e.g., 10 point clouds per second). In particular embodiments, a point-cloud frame rate may be substantially fixed, or a point-cloud frame rate may be dynamically adjustable. As an example, a lidar system 100 may capture one or more point clouds at a particular frame rate (e.g., 1 Hz) and then switch to capture one or more point clouds at a different frame rate (e.g., 10 Hz). A slower frame rate (e.g., 1 Hz) may be used to capture one or more high-resolution point clouds, and a faster frame rate (e.g., 10 Hz) may be used to rapidly capture multiple lower-resolution point clouds.
In particular embodiments, a lidar system 100 may be configured to sense, identify, or determine distances to one or more targets 130 within a field of regard. As an example, a lidar system 100 may determine a distance to a target 130, where all or part of the target 130 is contained within a field of regard of the lidar system 100. All or part of a target 130 being contained within a FOR of the lidar system 100 may refer to the FOR overlapping, encompassing, or enclosing at least a portion of the target 130. In particular embodiments, target 130 may include all or part of an object that is moving or stationary relative to lidar system 100. As an example, target 130 may include all or a portion of a person, vehicle, motorcycle, truck, train, bicycle, wheelchair, pedestrian, animal, road sign, traffic light, lane marking, road-surface marking, parking space, pylon, guard rail, traffic barrier, pothole, railroad crossing, obstacle in or near a road, curb, stopped vehicle on or beside a road, utility pole, house, building, trash can, mailbox, tree, any other suitable object, or any suitable combination of all or part of two or more objects. In particular embodiments, a target may be referred to as an object.
In particular embodiments, light source 110, scanner 120, and receiver 140 may be packaged together within a single housing, where a housing may refer to a box, case, or enclosure that includes sensor window 157 and holds or contains all or part of a lidar system 100. As an example, a lidar-system enclosure may contain a light source 110, mirror 115, scanner 120, and receiver 140 of a lidar system 100. Additionally, the lidar-system enclosure may include a controller 150. The lidar-system enclosure may also include one or more electrical connections for conveying electrical power or electrical signals to or from the enclosure. In particular embodiments, one or more components of a lidar system 100 may be located remotely from a lidar-system enclosure. As an example, all or part of light source 110 may be located remotely from a lidar-system enclosure, and pulses of light produced by the light source 110 may be conveyed to the enclosure via optical fiber. As another example, all or part of a controller 150 may be located remotely from a lidar-system enclosure.
In particular embodiments, light source 110 may include an eye-safe laser, or lidar system 100 may be classified as an eye-safe laser system or laser product. An eye-safe laser, laser system, or laser product may refer to a system that includes a laser with an emission wavelength, average power, peak power, peak intensity, pulse energy, beam size, beam divergence, exposure time, or scanned output beam such that emitted light from the system presents little or no possibility of causing damage to a person's eyes. As an example, light source 110 or lidar system 100 may be classified as a Class 1 laser product (as specified by the 60825 1:2014 standard of the International Electrotechnical Commission (IEC)) or a Class I laser product (as specified by Title 21, Section 1040.10 of the United States Code of Federal Regulations (CFR)) that is safe under all conditions of normal use. In particular embodiments, lidar system 100 may be an eye-safe laser product (e.g., with a Class 1 or Class I classification) configured to operate at any suitable wavelength between approximately 900 nm and approximately 2100 nm. As an example, lidar system 100 may include a laser with an operating wavelength between approximately 1200 nm and approximately 1400 nm or between approximately 1400 nm and approximately 1600 nm, and the laser or the lidar system 100 may be operated in an eye-safe manner. As another example, lidar system 100 may be an eye-safe laser product that includes a scanned laser with an operating wavelength between approximately 900 nm and approximately 1700 nm. As another example, lidar system 100 may be a Class 1 or Class I laser product that includes a laser diode, fiber laser, or solid-state laser with an operating wavelength between approximately 1200 nm and approximately 1600 nm. As another example, lidar system 100 may have an operating wavelength between approximately 1500 nm and approximately 1510 nm.
In particular embodiments, one or more lidar systems 100 may be integrated into a vehicle. As an example, multiple lidar systems 100 may be integrated into a car to provide a complete 360-degree horizontal FOR around the car. As another example, 2-10 lidar systems 100, each system having a 45-degree to 180-degree horizontal FOR, may be combined together to form a sensing system that provides a point cloud covering a 360-degree horizontal FOR. The lidar systems 100 may be oriented so that adjacent FORs have an amount of spatial or angular overlap to allow data from the multiple lidar systems 100 to be combined or stitched together to form a single or continuous 360-degree point cloud. As an example, the FOR of each lidar system 100 may have approximately 1-30 degrees of overlap with an adjacent FOR. In particular embodiments, a vehicle may refer to a mobile machine configured to transport people or cargo. For example, a vehicle may include, may take the form of, or may be referred to as a car, automobile, motor vehicle, truck, bus, van, trailer, off-road vehicle, farm vehicle, lawn mower, construction equipment, forklift, robot, golf cart, motorhome, taxi, motorcycle, scooter, bicycle, skateboard, train, snowmobile, watercraft (e.g., a ship or boat), aircraft (e.g., a fixed-wing aircraft, helicopter, or dirigible), unmanned aerial vehicle (e.g., drone), or spacecraft. In particular embodiments, a vehicle may include an internal combustion engine or an electric motor that provides propulsion for the vehicle.
In particular embodiments, one or more lidar systems 100 may be included in a vehicle as part of an advanced driver assistance system (ADAS) to assist a driver of the vehicle in operating the vehicle. For example, a lidar system 100 may be part of an ADAS that provides information (e.g., about the surrounding environment) or feedback to a driver (e.g., to alert the driver to potential problems or hazards) or that automatically takes control of part of a vehicle (e.g., a braking system or a steering system) to avoid collisions or accidents. A lidar system 100 may be part of a vehicle ADAS that provides adaptive cruise control, automated braking, automated parking, collision avoidance, alerts the driver to hazards or other vehicles, maintains the vehicle in the correct lane, or provides a warning if an object or another vehicle is in a blind spot.
In particular embodiments, one or more lidar systems 100 may be integrated into a vehicle as part of an autonomous-vehicle driving system. As an example, a lidar system 100 may provide information about the surrounding environment to a driving system of an autonomous vehicle. An autonomous-vehicle driving system may be configured to guide the autonomous vehicle through an environment surrounding the vehicle and toward a destination. An autonomous-vehicle driving system may include one or more computing systems that receive information from a lidar system 100 about the surrounding environment, analyze the received information, and provide control signals to the vehicle's driving systems (e.g., steering mechanism, accelerator, brakes, lights, or turn signals). As an example, a lidar system 100 integrated into an autonomous vehicle may provide an autonomous-vehicle driving system with a point cloud every 0.1 seconds (e.g., the point cloud has a 10 Hz update rate, representing 10 frames per second). The autonomous-vehicle driving system may analyze the received point clouds to sense or identify targets 130 and their respective locations, distances, or speeds, and the autonomous-vehicle driving system may update control signals based on this information. As an example, if lidar system 100 detects a vehicle ahead that is slowing down or stopping, the autonomous-vehicle driving system may send instructions to release the accelerator and apply the brakes.
In particular embodiments, an autonomous vehicle may be referred to as an autonomous car, driverless car, self-driving car, robotic car, or unmanned vehicle. In particular embodiments, an autonomous vehicle may refer to a vehicle configured to sense its environment and navigate or drive with little or no human input. As an example, an autonomous vehicle may be configured to drive to any suitable location and control or perform all safety-critical functions (e.g., driving, steering, braking, parking) for the entire trip, with the driver not expected to control the vehicle at any time. As another example, an autonomous vehicle may allow a driver to safely turn their attention away from driving tasks in particular environments (e.g., on freeways), or an autonomous vehicle may provide control of a vehicle in all but a few environments, requiring little or no input or attention from the driver.
In particular embodiments, an autonomous vehicle may be configured to drive with a driver present in the vehicle, or an autonomous vehicle may be configured to operate the vehicle with no driver present. As an example, an autonomous vehicle may include a driver's seat with associated controls (e.g., steering wheel, accelerator pedal, and brake pedal), and the vehicle may be configured to drive with no one seated in the driver's seat or with little or no input from a person seated in the driver's seat. As another example, an autonomous vehicle may not include any driver's seat or associated driver's controls, and the vehicle may perform substantially all driving functions (e.g., driving, steering, braking, parking, and navigating) without human input. As another example, an autonomous vehicle may be configured to operate without a driver (e.g., the vehicle may be configured to transport human passengers or cargo without a driver present in the vehicle). As another example, an autonomous vehicle may be configured to operate without any human passengers (e.g., the vehicle may be configured for transportation of cargo without having any human passengers onboard the vehicle).
In particular embodiments, an optical signal (which may be referred to as a light signal, a light waveform, an optical waveform, an output beam, an emitted optical signal, or emitted light) may include pulses of light, CW light, amplitude-modulated light, frequency-modulated (FM) light, or any suitable combination thereof. Although this disclosure describes or illustrates example embodiments of lidar systems 100 or light sources 110 that produce optical signals that include pulses of light, the embodiments described or illustrated herein may also be applied, where appropriate, to other types of optical signals, including continuous-wave (CW) light, amplitude-modulated optical signals, or frequency-modulated optical signals. For example, a lidar system 100 as described or illustrated herein may be a pulsed lidar system and may include a light source 110 that produces pulses of light. Alternatively, a lidar system 100 may be configured to operate as a frequency-modulated continuous-wave (FMCW) lidar system and may include a light source 110 that produces CW light or a frequency-modulated optical signal.
In particular embodiments, a lidar system 100 may be an FMCW lidar system where the emitted light from the light source 110 (e.g., output beam 125 in
A light source 110 for a FMCW lidar system may include (i) a direct-emitter laser diode, (ii) a seed laser diode followed by an SOA, (iii) a seed laser diode followed by a fiber-optic amplifier, or (iv) a seed laser diode followed by an SOA and then a fiber-optic amplifier. A seed laser diode or a direct-emitter laser diode may be operated in a CW manner (e.g., by driving the laser diode with a substantially constant DC current), and a frequency modulation may be provided by an external modulator (e.g., an electro-optic phase modulator may apply a frequency modulation to seed-laser light). Alternatively, a frequency modulation may be produced by applying a current modulation to a seed laser diode or a direct-emitter laser diode. The current modulation (which may be provided along with a DC bias current) may produce a corresponding refractive-index modulation in the laser diode, which results in a frequency modulation of the light emitted by the laser diode. The current-modulation component (and the corresponding frequency modulation) may have any suitable frequency or shape (e.g., piecewise linear, sinusoidal, triangle-wave, or sawtooth). For example, the current-modulation component (and the resulting frequency modulation of the emitted light) may increase or decrease monotonically over a particular time interval. As another example, the current-modulation component may include a triangle or sawtooth wave with an electrical current that increases or decreases linearly over a particular time interval, and the light emitted by the laser diode may include a corresponding frequency modulation in which the optical frequency increases or decreases approximately linearly over the particular time interval. For example, a light source 110 that emits light with a linear frequency change of 200 MHz over a 2-μs time interval may be referred to as having a frequency modulation m of 1014 Hz/s (or, 100 MHz/μs).
In addition to producing frequency-modulated emitted light, a light source 110 may also produce frequency-modulated local-oscillator (LO) light. The LO light may be coherent with the emitted light, and the frequency modulation of the LO light may match that of the emitted light. The LO light may be produced by splitting off a portion of the emitted light prior to the emitted light exiting the lidar system. Alternatively, the LO light may be produced by a seed laser diode or a direct-emitter laser diode that is part of the light source 110. For example, the LO light may be emitted from the back facet of a seed laser diode or a direct-emitter laser diode, or the LO light may be split off from the seed light emitted from the front facet of a seed laser diode. The received light (e.g., emitted light that is scattered by a target 130) and the LO light may each be frequency modulated, with a frequency difference or offset that corresponds to the distance to the target 130. For a linearly chirped light source (e.g., a frequency modulation that produces a linear change in frequency with time), the larger the frequency difference is between the received light and the LO light, the farther away the target 130 is located.
A frequency difference between received light and LO light may be determined by mixing the received light with the LO light (e.g., by coupling the two beams onto a detector so they are coherently mixed together at the detector) and determining the resulting beat frequency. For example, a photocurrent signal produced by an APD may include a beat signal resulting from the coherent mixing of the received light and the LO light, and a frequency of the beat signal may correspond to the frequency difference between the received light and the LO light. The photocurrent signal from an APD (or a voltage signal that corresponds to the photocurrent signal) may be analyzed using a frequency-analysis technique (e.g., a fast Fourier transform (FFT) technique) to determine the frequency of the beat signal. If a linear frequency modulation m (e.g., in units of Hz/s) is applied to a CW laser, then the round-trip time T may be related to the frequency difference Δf between the received scattered light and the LO light by the expression T=Δf/m. Additionally, the distance/) from the target 130 to the lidar system 100 may be expressed as I)=(Δf/m)·c/2, where c is the speed of light. For example, for a light source 110 with a linear frequency modulation of 1014 Hz/s, if a frequency difference (between the received scattered light and the LO light) of 33 MHz is measured, then this corresponds to a round-trip time of approximately 330 ns and a distance to the target of approximately 50 meters. As another example, a frequency difference of 133 MHz corresponds to a round-trip time of approximately 1.33 μs and a distance to the target of approximately 200 meters. A receiver or processor of an FMCW lidar system may determine a frequency difference between received scattered light and LO light, and the distance to a target may be determined based on the frequency difference. The frequency difference Δf between received scattered light and LO light corresponds to the round-trip time T (e.g., through the relationship T=Δf/m), and determining the frequency difference may correspond to or may be referred to as determining the round-trip time.
In the example of
In particular embodiments, a scan pattern 200 may include multiple pixels 210, and each pixel 210 may be associated with one or more laser pulses or one or more distance measurements. Additionally, a scan pattern 200 may include multiple scan lines 230, where each scan line represents one scan across at least part of a field of regard, and each scan line 230 may include multiple pixels 210. In
In particular embodiments, a pixel 210 may refer to a data element that includes (i) distance information (e.g., a distance from a lidar system 100 to a target 130 from which an associated pulse of light was scattered) or (ii) an elevation angle and an azimuth angle associated with the pixel (e.g., the elevation and azimuth angles along which the associated pulse of light was emitted). Each pixel 210 may be associated with a distance (e.g., a distance to a portion of a target 130 from which an associated laser pulse was scattered) or one or more angular values. As an example, a pixel 210 may be associated with a distance value and two angular values (e.g., an azimuth and altitude) that represent the angular location of the pixel 210 with respect to the lidar system 100. A distance to a portion of target 130 may be determined based at least in part on a time-of-flight measurement for a corresponding pulse. An angular value (e.g., an azimuth or altitude) may correspond to an angle (e.g., relative to reference line 220) of output beam 125 (e.g., when a corresponding pulse is emitted from lidar system 100) or an angle of input beam 135 (e.g., when an input signal is received by lidar system 100). In particular embodiments, an angular value may be determined based at least in part on a position of a component of scanner 120. As an example, an azimuth or altitude value associated with a pixel 210 may be determined from an angular position of one or more corresponding scanning mirrors of scanner 120.
In particular embodiments, a polygon mirror 301 may be configured to rotate along a Θx or Θy direction and scan output beam 125 along a substantially horizontal or vertical direction, respectively. A rotation along a Θx direction may refer to a rotational motion of mirror 301 that results in output beam 125 scanning along a substantially horizontal direction. Similarly, a rotation along a Θy direction may refer to a rotational motion that results in output beam 125 scanning along a substantially vertical direction. In
In particular embodiments, a polygon mirror 301 may refer to a multi-sided object having reflective surfaces 320 on two or more of its sides or faces. As an example, a polygon mirror may include any suitable number of reflective faces (e.g., 2, 3, 4, 5, 6, 7, 8, or 10 faces), where each face includes a reflective surface 320. A polygon mirror 301 may have a cross-sectional shape of any suitable polygon, such as for example, a triangle (with three reflecting surfaces 320), square (with four reflecting surfaces 320), pentagon (with five reflecting surfaces 320), hexagon (with six reflecting surfaces 320), heptagon (with seven reflecting surfaces 320), or octagon (with eight reflecting surfaces 320). In
In particular embodiments, a polygon mirror 301 may be continuously rotated in a clockwise or counter-clockwise rotation direction about a rotation axis of the polygon mirror 301. The rotation axis may correspond to a line that is perpendicular to the plane of rotation of the polygon mirror 301 and that passes through the center of mass of the polygon mirror 301. In
In particular embodiments, output beam 125 may be reflected sequentially from the reflective surfaces 320A, 320B, 320C, and 320D as the polygon mirror 301 is rotated. This results in the output beam 125 being scanned along a particular scan axis (e.g., a horizontal or vertical scan axis) to produce a sequence of scan lines, where each scan line corresponds to a reflection of the output beam 125 from one of the reflective surfaces of the polygon mirror 301. In
In particular embodiments, scanner 120 may be configured to scan both a light-source field of view and a receiver field of view across a field of regard of the lidar system 100. Multiple pulses of light may be emitted and detected as the scanner 120 scans the FOVL and FOVR across the field of regard of the lidar system 100 while tracing out a scan pattern 200. In particular embodiments, the light-source field of view and the receiver field of view may be scanned synchronously with respect to one another, so that as the FOVL is scanned across a scan pattern 200, the FOVR follows substantially the same path at the same scanning speed. Additionally, the FOVL and FOVR may maintain the same relative position to one another as they are scanned across the field of regard. As an example, the FOVL may be substantially overlapped with or centered inside the FOVR (as illustrated in
In particular embodiments, the FOVL may have an angular size or extent OL that is substantially the same as or that corresponds to the divergence of the output beam 125, and the FOVR may have an angular size or extent OR that corresponds to an angle over which the receiver 140 may receive and detect light. In particular embodiments, the receiver field of view may be any suitable size relative to the light-source field of view. As an example, the receiver field of view may be smaller than, substantially the same size as, or larger than the angular extent of the light-source field of view. In particular embodiments, the light-source field of view may have an angular extent of less than or equal to 50 milliradians, and the receiver field of view may have an angular extent of less than or equal to 50 milliradians. The FOVL may have any suitable angular extent OL, such as for example, approximately 0.1 mrad, 0.2 mrad, 0.5 mrad, 1 mrad, 1.5 mrad, 2 mrad, 3 mrad, 5 mrad, 10 mrad, 20 mrad, 40 mrad, or 50 mrad. Similarly, the FOVR may have any suitable angular extent OR, such as for example, approximately 0.1 mrad, 0.2 mrad, 0.5 mrad, 1 mrad, 1.5 mrad, 2 mrad, 3 mrad, 5 mrad, 10 mrad, 20 mrad, 40 mrad, or 50 mrad. In particular embodiments, the light-source field of view and the receiver field of view may have approximately equal angular extents. As an example, ΘL and ΘR may both be approximately equal to 1 mrad, 2 mrad, or 4 mrad. In particular embodiments, the receiver field of view may be larger than the light-source field of view, or the light-source field of view may be larger than the receiver field of view. As an example, ΘL may be approximately equal to 3 mrad, and ΘR may be approximately equal to 4 mrad. As another example, ΘR may be approximately L times larger than ΘL, where L is any suitable factor, such as for example, 1.1, 1.2, 1.5, 2, 3, 5, or 10.
In particular embodiments, a pixel 210 may represent or may correspond to a light-source field of view or a receiver field of view. As the output beam 125 propagates from the light source 110, the diameter of the output beam 125 (as well as the size of the corresponding pixel 210) may increase according to the beam divergence ΘL. As an example, if the output beam 125 has a ΘL of 2 mrad, then at a distance of 100 m from the lidar system 100, the output beam 125 may have a size or diameter of approximately 20 cm, and a corresponding pixel 210 may also have a corresponding size or diameter of approximately 20 cm. At a distance of 200 m from the lidar system 100, the output beam 125 and the corresponding pixel 210 may each have a diameter of approximately 40 cm.
In particular embodiments, a unidirectional scan pattern 200 may be produced by a scanner 120 that includes a polygon mirror (e.g., polygon mirror 301 of
An internal reference target may be positioned at the end of the scan and is imaged periodically to provide reference range and pulse energy information that can be used in various compensation procedures. For example, large temperature variations may cause drifts in the reflectance and range estimates that will corrupt the radiometry of a point cloud. These drifts can be detected and measured on the reference target. In a multi-beam lidar, variations in energy and pulse delays between the beams can also be detected and corrected. In a variable-power lidar, where certain segments of the scene are imaged under a lower power, while selected regions of interest are imaged at a high power, the laser power can also be monitored on the internal target, provided that the various regions intersect the reference target.
An azimuth (which may be referred to as an azimuth angle) may represent a horizontal angle with respect to reference line 220 in
Although internal target 660 is shown in
Internal target 660 may have different sizes, shapes, dimensions, thicknesses, volumes, and geometries. In some embodiments, the internal target may have an irregular shape, a non-negligible thickness, one or more mounting holes, connectors, screws, attachments, or other features, all of which may contribute to significant noise in the return signal, particular in energy.
In some embodiments, scanner 120 includes two scanning mirrors: (1) a polygon mirror 301 that rotates along the Θx direction and (2) a scanning mirror 302 that oscillates back and forth along the Θy direction. The output beam 625 from light source 110, which passes alongside mirror 115, is reflected by reflecting surface 321 of scan mirror 302, is then reflected by a reflecting surface (e.g., surface 320A, 320B, 320C, or 320D) of polygon mirror 301, and then hits internal target 660. Scattered light from internal target 660 reflects from polygon mirror 301, scan mirror 302, and mirror 115, which directs the scattered light through focusing lens 330 and to the detector 340 of receiver 140. The detector 340 may be a PN photodiode, a PIN photodiode, an APD, an SPAD, or any other suitable detector.
In some embodiments, lidar system 100 may include additional components (not shown in
An internal reference target can provide the means to maintain stable radiometric characteristics in an automotive lidar point cloud. However, producing accurate and precise measurements from the internal reference target is very challenging for a number of reasons. One challenge is that the internal reference target is in close proximity to the detector. Since the internal target is placed inside the lidar system, the distance between the internal target and the detector is short compared to the typical operating range of a lidar system, which may be in meters or kilometers. Another challenge is that the internal target may be installed under an acute angle of incidence, which causes accurate measurements from the internal reference target to be very challenging. Another challenge is that the returns from the internal target may have intense speckle noise. For example, the internal target may have Lambertian surfaces, which may produce diffuse reflections and therefore speckle noise.
In lidars with parallax, the close proximity of the internal target causes the imaging of the internal target using regular point cloud detectors to be very challenging. In some embodiments, special detectors for detecting returns from targets that are positioned at close proximity may be utilized. In some embodiments, the special detectors may be blockage detectors. Blockage detectors are designed to capture returns from any foreign material or contaminants deposited on the window that block the photons from passing through the window.
In some embodiments, lidar system 100 may include multiple receivers.
In some embodiments, receiver 140 may include additional one or more auxiliary detectors (not shown in the figure) to detect returns from internal target 660. In some embodiments, the one or more auxiliary detectors and detector 340 share the same circuitry of the receiver. For example, a second detector of the lidar system is configured to detect at least a portion of the emitted light scattered by internal reference target 660 to provide reference range and pulse energy information that can be used in various compensation procedures. In various embodiments, the first and second detectors are different detectors and are configured to detect different scatter patterns.
The internal target being installed under an acute angle of incidence causes a multitude of artifacts. First, the returned energy is minimized by possible specular reflections that will divert photons away from the receiver. Second, even in the case of perfect Lambertian reflection, the return energy is diminished by the cosine of the angle of incidence. Finally, due to its beam divergence, the laser may illuminate the target under a very large spot, in comparison to the area of the target. The spot may reach the periphery of the target or illuminate the raised edge of the target and create a ‘hot’ edge that will corrupt the radiometric estimates derived from it. Returns from this target will exhibit severe noise, including high-frequency noise from speckle, and low-frequency modulations due to the variations of the angle of incidence.
In general, imaging a reference target in any system will face similar challenges, because 1) the internal target is outside the scanning area and therefore has challenging imaged geometries, and 2) the estimates extracted from this target's returns must have great accuracy and robustness in order to drive compensation. The required techniques should be computationally inexpensive and can accurately and rapidly detect both slow and rapid variations in laser power and delay. The techniques disclosed in the present application may be used for imaging challenging targets (internal or external), including targets with steep angles of incidence, which produce very noisy returns.
The present application further discloses techniques that are capable of filtering out this noise with the following features and advantages:
The proposed techniques employ a sequential state machine to sample the returns from the internal target, followed by an accumulator, and an IIR filter. The state machine is designed so that it selects and accumulates only a number (which is a power of 2 number) of returns from the center of the target.
Scan patterns may be configured for different environments. For example, there is a scan pattern for urban traffic and there is a scan pattern for highways, where a car may need to look further away. In addition, using horizon tracking, the scan pattern is adjusted dynamically to keep the high-density area centered on the horizon line. The state machine of the process 1300 in
At 701, one or more beams of light are emitted. The light source is configured to emit light pulses that are scanned across any downrange objects or a reference reflectivity material internal to the system. For example, an output light beam is emitted from the lidar system that passes through a sensor window. The sensor window can act as a protective barrier for the lidar system while still allowing light to pass in both directions. In some embodiments, the lidar system emits multiple output beams, each potentially scanning a different field of regard. The output beam can reach downrange objects and can be scattered and/or reflected by the downrange objects. The output beam may periodically reach internal target 660 and can be scattered and/or reflected by internal target 660.
At 703, scattered light is received. For example, light scattered and/or reflected by an object is received at the lidar sensor. The corresponding object can be a downrange object as well as internal target 660. In some embodiments, the transmit and receive axes of the emitted light and received light are offset, for example, by a parallax distance configured for the lidar system.
At 705, the received scattered light is analyzed. For example, the received scattered light is directed to a receiver component of the lidar system where sensor data can be captured and analyzed. In some embodiments, the receiver component includes a receive lens for focusing the received scattered light onto a readout integrated circuit (ROIC) of the receiver where detector site locations of the detector plane are located. The detector site locations can be used to detect scatter patterns associated with the received scattered light. In various embodiments, different detector site locations are utilized to detect light scattered from a downrange object compared to light scattered from internal target 660. In some embodiments, different detector site locations are utilized to differentiate light scattered from different emitted output beams. Using detectors positioned at the detector site locations, sensor data can be captured and analyzed for sensor readings. In some embodiments, the analysis is performed at least in part by the readout integrated circuit (ROIC) and/or processor of the receiver and/or a controller of the lidar system. The processing can include determining an output signal corresponding to the detected scatter pattern. In some embodiments, the output signal is a sensor reading that corresponds to a measurement of the detected scattered light, such as an intensity reading or another measured sensor reading.
At 707, a sensor reading is provided. For example, one or more sensor readings are provided in response to the analysis of the received scattered light performed at 705. The sensor reading can be an intensity reading and/or another measurement or measurements. In some embodiments, the sensor reading includes location and/or distance data. For example, sensor readings can correspond to a point cloud based on objects detected in the fields of regard scanned by the lidar system. The output signal can include a relative intensity to serve as a pulse energy calibration reading.
The laser power monitoring (LPM) techniques monitor the total laser power (TLPM) and the ratio between the laser powers in output beam A and output beam B (RLPM-Relative Laser Power Monitoring), respectively, for slow variations due to temperature drifts or other external factors. This feature is not designed to provide a pulse-by-pulse estimation, but a long term mean estimate. The laser pulse delay compensation techniques monitor the optical delay, which can also drift with temperature and other external factors, and the techniques further perform a compensation. Both of these techniques are based on illuminating the internal target and computing radiometry estimates from the return signal detected by the obstruction detector PIN0. The laser power monitoring techniques are based on monitoring the return energy from the internal target, whereas the laser pulse delay compensation techniques are based on range.
In some embodiments, because laser power monitoring techniques and laser pulse delay compensation techniques share at least some of the logic, they may be implemented in a single pipeline, which has two main modules as depicted in
The input of metric estimation module 802, denoted as Xk, may include the detected signals from the lidar returns originating from the internally installed reference target (e.g., internal target 660). For example, Xk may be voltage signals from receiver 140, including output electrical signal 145 in
Compensation module 804 receives the pulse energy and range metrics Yk from metric estimation module 802. Compensation module 804 further collects and analyzes the statistics of the pulse energy and range metrics Yk over time, and uses the collected statistics to calculate a set of calibration correction values, Zk, for compensating the errors of the lidar system due to temperature drift and other external factors, thereby improving the performance of the lidar system. In some embodiments, stable and robust metrics (e.g., using a long-time mean) are derived from the energy and range of the internal target returns, which are then compared against their corresponding reference values. In some embodiments, differences from these reference values that are greater than a set of predetermined thresholds will trigger compensations of the energy/range of the point cloud. The compensations of the energy or range are used as calibration values. For example, the compensations of the energy or range may be added or subtracted offset values that are determined based on the corresponding differences, such as scaled differences.
In this example, in order to scan internal target 660 (see
As shown in
In some embodiments, ROI 902 is dynamically determined based on different factors. In some embodiments, ROI 902 is dynamically determined based on the scan patterns of lidar system 100. Scan patterns may be configured for different environments. For example, there is a scan pattern for urban traffic and there is a scan pattern for highways, where a car may need to look further away. In some embodiments, the scan pattern (and therefore ROI 902) may be dynamically determined based on horizon tracking. The horizon line is a boundary line defined between a sky and non-sky region. It is a line that represents the eye level of the viewer. For example, if lidar system 100 operates in a vehicle, the horizon line may be a line that represents the eye level of the driver of the vehicle. In some embodiments, using horizon tracking, the scan pattern may be adjusted dynamically to keep the high-density area centered on the horizon line. The state machine of the process 1300 in
In some embodiments, ROI 902 may be dynamically determined based on horizon tracking. For example, ROI 902 may be centered at the elevation corresponding to the horizon line. In some embodiments, the state machine of the process 1300 in
With continued reference to
The range measurements of the returns from internal target 660 for each beam (output beam A or output beam B) are measured by a comparator attached to a PIN photodiode detector, PIN0, in receiver 140. Both comparators have effective thresholds set at 5 DAC (digital-to-analog converter) counts above the baseline. The ranges captured by both comparators are very close to each other.
In some embodiments, in order to avoid statistical bias due to different actors, such as the angle of incidence or proximity to the edges of the internal target, a minimal number of pixels are acquired in an area of interest positioned at certain coordinates on the internal target (denoted by R and not to be confused with the scan ROI). This area of interest is centered on the surface of the internal target and is subtended by certain azimuth and elevation intervals, which may be specific to the utilized scan pattern. In some embodiments, for computing efficiency, a predetermined number of pixels (NP) within the region of interest are collected. In one example, a 1 Hz scan pattern is used, and NP=64 pixels.
In some embodiments, the energy and range measurements are averaged over the predetermined number of pixels (NP), and therefore the set of NP pixels collected during a frame should be in-sync with the frame. In other words, the set of NP pixels belongs to a single frame and does not straddle over multiple frames. The set of NP pixels should not shift with respect to the center of the surface of internal target 660 from frame to frame.
At step 1302, the coordinates of a region of interest, R 1402, are determined. This area of interest is centered on the surface of internal target 1400 and is subtended by certain azimuth and elevation intervals, which may be specific to the utilized scan pattern. In other words, the region of interest is positioned at certain coordinates (e.g., defined by the azimuth and elevation intervals) on internal target 1400.
The region of interest R where the pixels from the internal target are collected needs to be parameterized so that it can be adapted to the scan pattern. This adaptiveness ensures that the collected energy and range metrics are well-behaved and form a periodic pattern with the same period as the frame rate. In some embodiments, the region of interest is dynamically determined based on different factors. In some embodiments, the region of interest is dynamically determined based on the scan patterns of lidar system 100. Scan patterns may be configured for different environments. For example, there is a scan pattern for urban traffic and there is a scan pattern for highways, where a car may need to look further away. In some embodiments, the scan pattern (and therefore the region of interest) may be dynamically determined based on horizon tracking. In some embodiments, using horizon tracking, the scan pattern may be adjusted dynamically to keep the high-density area centered on the horizon line. The region of interest sampling area is dynamically determined based on the scan pattern utilized and the possible dynamic adjustments to the scan pattern (e.g., due to horizon tracking or environment changes). In some embodiments, the region of interest may be dynamically determined based on horizon tracking. For example, the region of interest may be centered at the elevation corresponding to the horizon line. In some embodiments, horizon line information from the horizon tracker is received, and the coordinates and the angles corresponding to the region of interest sampling area are changed. At this point, the state machine is in the INIT state 1404.
At step 1304, the region of interest is being detected as pixels are captured along a scan path. In some embodiments, the coordinates of the region of interest (determined at step 1302) are compared with those corresponding to the current pixel in the current frame. Suppose that (az, el) are the azimuth (az) and elevation (el) coordinates of the current pixel in the frame. If (az, el) falls within the region of interest (i.e., (az, el) € R), then the process has entered the region of interest, and the state machine has entered the FOUND_ROI state 1406. After the state machine has entered FOUND_ROI state 1406, process 1300 enters the ACCUMULATE 1408 state when the process starts to collect the pixels and accumulate their energy and range values.
At step 1306, pixels within the region of interest are collected and their energy and range values are accumulated. A predetermined number of pixels (NP) within the region of interest are collected. The energy and range values are accumulated over NP in one frame. A counter may be used to keep track of the number of pixels that have been collected so far. As shown in
At step 1308, the energy and range metrics are determined based on the accumulated values. For example, the accumulated energy value may be divided by NP to obtain a corresponding average energy metric. The accumulated range value may be divided by NP to obtain a corresponding average range metric. For example, the energy metric is a batch average of the energy collected during the current frame over the region of interest on the internal target, and it is computed by dividing the accumulated value by the number of samples collected. For most scan patterns, the number of samples can be chosen to be a power of 2. The advantage is that the division by a power of 2 is a shift operation, which is computationally efficient. For example, if NP is equal to 256, the division by 256 is performed by shifting 8 bits.
At step 1310, the average energy metric and the average range metric are filtered. Each average metric may be fed into a filter to be smoothed further. For example, the filter may be a single pole IIR filter. In another example, the filter may be a second order section IIR with two poles and two zeros. The filtered metrics may be used by compensation module 804 of
At step 1312, scanning is continued. When the filtering is performed, the state is transitioned to FAST_FORWARD 1414 state. During this state, the scanning is continued within R, but the accumulation has stopped. When the elevation coordinate of the current ray is outside the interval subtended by R, the state is transitioned back to INIT 1404 state.
Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
This application claims priority to U.S. Provisional Patent Application No. 63/522,970 entitled PROCESSING REFLECTED LIGHT SIGNALS FROM INTERNAL REFERENCE TARGET filed Jun. 23, 2023 which is incorporated herein by reference for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 63522970 | Jun 2023 | US |
