Patent Application
20220146680
Autonomous, self-driving, and semi-autonomous automobiles use a combination of different sensors and technologies such as radar, image-recognition cameras, and sonar for detection and location of surrounding objects. These sensors enable a host of improvements in driver safety including collision warning, automatic-emergency braking, lane-departure warning, lane-keeping assistance, adaptive cruise control, and piloted driving. Among these sensor technologies, light detection and ranging (LiDAR) systems take a critical role, enabling real-time, high-resolution 3D mapping of the surrounding environment.
Most current LiDAR systems used for autonomous vehicles today utilize a small number of lasers, combined with some method of mechanically scanning the environment. Some state-of-the-art LiDAR systems use two-dimensional Vertical Cavity Surface Emitting Lasers (VCSEL) arrays as the illumination source and various types of solid-state detector arrays in the receiver. It is highly desired that future autonomous cars utilize solid-state semiconductor-based LiDAR systems with high reliability and wide environmental operating ranges. These solid-state LiDAR systems are advantageous because they use solid state technology that has no moving parts. However, currently state-of-the-art LiDAR systems have many practical limitations and new systems and methods are needed to improve performance.
The present teaching, in accordance with preferred and exemplary embodiments, together with further advantages thereof, is more particularly described in the following detailed description, taken in conjunction with the accompanying drawings. The skilled person in the art will understand that the drawings, described below, are for illustration purposes only. The drawings are not necessarily to scale; emphasis instead generally being placed upon illustrating principles of the teaching. The drawings are not intended to limit the scope of the Applicant's teaching in any way.
The present teaching will now be described in more detail with reference to exemplary embodiments thereof as shown in the accompanying drawings. While the present teaching is described in conjunction with various embodiments and examples, it is not intended that the present teaching be limited to such embodiments. On the contrary, the present teaching encompasses various alternatives, modifications and equivalents, as will be appreciated by those of skill in the art. Those of ordinary skill in the art having access to the teaching herein will recognize additional implementations, modifications, and embodiments, as well as other fields of use, which are within the scope of the present disclosure as described herein.
Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the teaching. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.
It should be understood that the individual steps of the method of the present teaching can be performed in any order and/or simultaneously as long as the teaching remains operable. Furthermore, it should be understood that the apparatus and method of the present teaching can include any number or all of the described embodiments as long as the teaching remains operable.
The present teaching relates generally to Light Detection and Ranging (LiDAR), which is a remote sensing method that uses laser light to measure distances (ranges) to objects. LiDAR systems generally measure distances to various objects or targets that reflect and/or scatter light. Autonomous vehicles make use of LiDAR systems to generate a highly accurate 3D map of the surrounding environment with fine resolution. The systems and methods described herein are directed towards providing a solid-state, pulsed time-of-flight (TOF) LiDAR system with high levels of reliability, while also maintaining long measurement range as well as low cost.
Some embodiments of LiDAR systems according to the present teaching use a laser transmitter that includes a laser array. In some specific embodiments, the laser array comprises Vertical Cavity Surface Emitting Laser (VCSEL) devices. These may include top-emitting VCSELs, bottom-emitting VCSELs, and various types of high-power VCSELs. The VCSEL arrays may be monolithic. The laser emitters may all share a common substrate, including semiconductor substrates or ceramic substrates.
In various embodiments, individual lasers and/or groups of lasers using one or more transmitter arrays can be individually controlled. Each individual emitter in the transmitter array can be fired independently. The optical beam emitted by each laser emitter corresponds to a 3D projection angle subtending only a portion of the total system field-of-view. One example of such a LiDAR system is described in U.S. Patent Publication No. 2017/0307736 A1. U.S. Patent Publication No. 2017/0307736 A1 is assigned to the present assignee and is incorporated herein by reference. In addition, the number of pulses fired by an individual laser, or group of lasers, can be controlled based on a desired performance objective of the LiDAR system. The duration and timing of this sequence can also be controlled to achieve various performance goals.
Some embodiments of LiDAR systems according to the present teaching use detectors and/or groups of detectors in a detector array that can also be individually controlled. See, for example, U.S. Provisional Application No. 62/859,349, entitled “Eye-Safe Long-Range Solid-State LiDAR System”. U.S. Provisional Application No. 62/859,349 is assigned to the present assignee and is incorporated herein by reference. This independent control over the individual lasers and/or groups of lasers in the transmitter array and/or over the detectors and/or groups of detectors in a detector array provide for various desirable operating features including control of the system field-of-view, optical power levels, and scanning pattern.
The optical power level(s) emitted by the LiDAR system transmitter is an important parameter that factors into numerous performance metrics of the LiDAR system. This includes, for example, distance, field-of-view, resolution, speed and frame rate and eye safety, among other performance metrics. As such, systems and methods that monitor transmit power for LiDAR systems are desirable. It is desirable that the monitor systems be compact, low-cost and produce a desired precision and accuracy of the monitored parameter.
One feature of the LiDAR systems of the present teaching is the inclusion of optical performance monitoring directly in the LiDAR transmit module. Optical performance monitoring within a LiDAR module may be important for a variety of reasons. For example, incorporating optical power monitoring inside the illuminator assembly can improve calibration, performance, and reliability monitoring. Lasers degrade with lifetime and so it can be useful to monitor the laser output power within the projector assembly itself. For example, by monitoring the light as is exiting the projector, rather than just relying on the received optical signal after the light has been reflected from an external object, it is possible to monitor generated power more accurately and quickly. Also, it is possible to monitor the temperature proximate to the VCSEL lasers. Such a feature is useful to improve the reliability and performance. The optional monitoring of both the temperature and the power can be used not only for diagnostics, but also for controlling the lasers during operation to improve performance and/or lifetime of the system.
Another feature of the present teaching is that the power monitoring elements are configured to monitor light reflected off of, or directed from, various optical devices within the LiDAR transmitter that are providing other functions. The reflected or directed light detected within the transmitter can be used not just for passive monitoring purposes, but also to provide additional active control of the lasers and detectors in the transmitter.
Some embodiments of the performance monitor for LiDAR system transmitters of the present teaching monitor one or more parameters of the light generated by the LiDAR system transmitter itself. For example, the light generated by the transmitters can be monitored for laser wavelength, optical power, pulse timing, and pulse frequency. The wavelength of the generated light can be detected by using a power monitor including a receiver that is not simply a photodiode, but instead comprises a more complicated set of optics that allows detection of wavelength as well as optical power.
In a LiDAR design where multiple wavelengths are used, particularly if the wavelengths are close in absolute value, it may be desired to monitor their absolute or relative values in order to ensure that the system parameters are as intended. There are various known methods of monitoring either absolute wavelength of light generated by the laser, or the relative offset between light generated by the lasers of different wavelength. For example, an etalon-based device could be used as a wavelength monitor.
Embodiments of the systems and method of the present teaching that use multi-wavelength power monitoring also can improve the system robustness for detecting whether a fault is caused by laser degradation or shifts in optical performance metrics. Multi-wavelength power monitoring can also provide redundancy if one set of wavelengths should fail in the transmitter. A partial or full failure in operation of one set of wavelengths in the transmitter would still allow the ability for partial operation of the system using the other set of wavelengths in the transmitter if the optical monitoring for each wavelength is independent.
Another feature of the method and system of the present teaching is that multi-element, multi-wavelength optical power monitoring can be realized. One or more directing elements can be positioned to direct light to one or more monitors so that collective and individual powers from the one or more laser elements at one or more wavelengths can be monitored. For example, in one embodiment, the multiple reflection elements can be partial mirrors. In other embodiments, the multiple reflection elements can be configured to project the beam. In some embodiments, the monitors include photodetectors which are each sensitive to only one particular wavelength band of light. This configuration allows monitoring optical power of one or more wavelengths independently, which improves the system capabilities. Multi-wavelength power monitoring according to the present teaching can be configured to monitor for multiple parameters, such as laser wavelength, including absolute wavelength and/or relative wavelength, optical power, pulse timing, and pulse frequency.
Multi-wavelength power monitoring according to the present teaching also improves the LiDAR system's robustness for detecting whether a fault is caused by laser degradation or shifts in optical performance. Multi-wavelength power monitoring is useful, for example, to provide redundancy if one set of wavelengths generated by the transmitter should fail. A partial or full failure in the transmitter's generation of one set of wavelengths would still allow the ability for partial operation of the LiDAR system using the other set of wavelengths if the system is configured so that the optical monitoring for each wavelength band is independent.
It is known that degradation in the performance of the optical transmitter can be determined by monitoring the optical transmitter's laser output power, and then comparing the measured optical power to an expected reference value. The degradation in the performance of the optical transmitter can be caused by either or both of the laser itself or caused by various aspects of the opto-mechanical assembly. The degradation in the performance of the optical transmitter can then be analyzed. For example, U.S. Patent Application Publication No. US 2016/0025842 A1, entitled “System and Method for Monitoring Optical Subsystem Performance in Cloud LiDAR Systems” describes the benefits of laser output power monitoring for a LiDAR system designed for cloud measurements.
Measurements of the optical signal generated by the LiDAR transmitter can also be used in a passive monitoring system. In addition, the optical signal from the LiDAR transmitter can be used for active control of the laser bias current driving the semiconductor laser. A laser diode operates over a range of operating bias currents. Many types of semiconductor laser diode systems are operated in closed loop fashion where a received photodiode current from a monitor photodiode is used as an input to a bias control feedback loop. By monitoring and maintaining the monitor photodiode current at a constant value, which is a largely linear function of the incident power, the output power of the semiconductor laser can be maintained at a near constant value. This condition enables the system to react to environmental changes, such as temperature and mechanical movements, to achieve improved output power stability. Also, monitoring the optical power, and controlling the laser bias in response to the monitored optical power, can be used to accommodate degradation of the laser efficiency over its lifetime, without loss of optical power at system level.
In various embodiments, the transmitter optical signal can be monitored for numerous parameters, including, for example, laser wavelength, optical power, pulse timing, pulse frequency, and pulse duration among other parameters. The laser wavelength can be detected using a power monitor, which is not simply a photodiode or other optical detector, but instead, it is an optical system that allows detection of wavelength as well as optical power. In a LiDAR system design where multiple wavelengths are used, particularly if the wavelengths are close in absolute value, it may be desired to monitor their absolute or relative values in order to ensure that the system parameters are as intended. Various methods of monitoring either absolute wavelength, or the relative offset between lasers of different wavelength, are known within the art. For example, an etalon-based device could be used as a wavelength monitor.
It is desirable for some LiDAR systems to perform a calibration at the beginning of life (BOL) to provide a reference during the lifetime operation of the system. By calibration, we mean the characterization of the initial laser bias, temperature, and output power of the device, and then the subsequent tuning of laser bias and output power as a function of temperature to meet the required performance specifications with suitable margins at BOL. Often this process is performed as part of the manufacturing process for the LiDAR system. The performance parameters, such as the laser bias and the measured optical power, obtained during the calibration process, will often be stored as a function of temperature in the LiDAR system memory as a reference to be used for various operations. Various monitors of the present teaching can provide measured optical power for some of these calibration processes.
During operation of LiDAR systems, the actual temperature can be monitored and used in conjunction with the reference values stored in memory in some form of a look-up table to determine the optical laser bias set point. Alternatively, in combination with an optical power monitor, the actual values of output power, laser bias, and temperature during operation can be compared to the reference values in a look-up table to identify any significant change or degradation in the system which can indicate a potential reliability issue. In various implementations of practical systems, a LiDAR system detecting such changes could then communicate with the overall monitoring system in an automotive other vehicle to identify a need for potential service or repair.
The monitored transmitter 100 is described using lenses 112, 114 to project the light in the optical beams generated by the VCSEL array 102. However, one skilled in the art will appreciate that numerous other projecting optical elements can also be used. Numerous known implementations of projecting elements can be used in the monitored transmitter 100 of the present teaching. In various embodiments, these projecting elements serve to shape and project the optical beams toward a target to achieve, for example, a desired field-of-view, target range, resolution, etc. These projecting elements can be configured to produce a common point for the optical beams within the transmitter footprint that allows the directing element to generate an illumination region at the monitor plane such that light from each of the optical beams can be monitored.
It will be understood to those of skill in the art that some embodiments of the present teaching do not require all of the generated optical beams from the VCSEL array 102 to appear in the illumination region. Instead, in some embodiments, only a subset of the optical beams that share the common point for the optical beams are provided in the illumination region and sampled by the monitor. The relative positions of the projecting elements, VCSEL array 102 and directing element serve to determine a desired subset of optical beams that share the common point and therefore are provided in the illumination region.
The directing element 110 of the monitored transmitter 100 embodiment of
In some embodiments, the optical components including one or more of the first lens 112, second lens 114, and directing element 110, have an optical coating on a surface that is designed to control reflectance. In embodiments, where no monitoring is needed, the optical reflectance is often set as low as possible to maximize the output power from the transmitter 100 in the direction of transmission 104. In some embodiments, an optical coating on the directing element 110 might have a reflectance of up to 5%. In some embodiments, the directing element 110 is not a separate discrete element but instead is an optical coating or set of coatings on one or more of the optical lens surfaces, where the reflectance of the coating has been selected to optimize the measured optical signal.
A feature of the present teaching is that the directing element 110 can be placed within the path of the optical beams being generated by the laser at a common point so that every laser within the transmitter array has some portion of light that can be reflected. See, for example, U.S. Pat. No. 10,514,444, which describes a LiDAR transmitter that includes multiple lenses to achieve a small angular divergence of the transmitted optical beam. U.S. Pat. No. 10,514,444 is assigned to the present assignee and is incorporated herein by reference.
In some embodiments, the directing element 110 is a diffractive optic element. In other embodiments, the directing element 110 is both a partially reflective mirror and an optical filter which blocks a portion of the visible spectrum. In one specific embodiments, the directing element is a prism. In some embodiments, the directing element is a holographic element. Numerous known lens configurations can be utilized to achieve the desired lens power, size and relative positions of the first lens 112 and second lens 114 and laser array 102.
In some embodiments, the monitor photodiode 106 is manufactured monolithically with the VCSEL array 102. The monitor photodiode 106 can be fabricated in numerous configurations. For example, in some embodiments, the monitor photodiode 106 is a single photodiode, which receives light from all the individual lasers within the 2D array. In other embodiments, more than one photodiode is used where the outputs of the multiple photodiodes are combined in some fashion to provide a common signal output at some point in the signal chain.
In some embodiments, the monitor photodiode 106 includes an optical element positioned so that an input receives the light propagating from the monitor photodiode 106. For example, the monitor photodiode 106 can include a light guide or a prism positioned to receive light and configured to direct that collected light to a photodiode. The optical element can take all the light in the monitoring aperture, or only a portion of the light from different spatial regions of the monitoring aperture, and then direct that light to one or more photodiodes. For example, the optical element may be a lightpipe as described herein.
In some configurations, it is not possible to place the monitor photodiode 106 immediately adjacent to the VCSEL array 102. Instead, for easier manufacturing the monitor photodiode 106 may need to be some distance away from the VCSEL array 102, yet still close enough that it falls within the illuminated area of the reflected optical beams. Thus, the inclination and position of the directing element 110 must be properly chosen such that light from multiple elements in the array 102 overlaps at the monitor photodiode 106.
In this embodiment, the monitor 304 includes the lightpipe 358. A lightpipe is a device that acts as a waveguide that retains the light internally through total internal reflection as the light propagates along the length of the lightpipe. The lightpipe 358 can be made of glass or plastic, or can be a hollow waveguide with internal mirrored surfaces. Typically, the dimensions of the cross-section of the lightpipe 358 perpendicular to the axis of propagation are much smaller than the propagation distance. The lightpipe 358 can be constructed with fixed or flexible bends to direct the light as desired provided that the bend radius is sufficiently large to maintain total internal reflection. A common example of a lightpipe is a fiber optic cable.
In a lightpipe configuration, instead of the reflected light being measured directly by a photodiode located within the illuminated area as in the embodiment of
Embodiments of the monitored transmitter that use a lightpipe do have an additional component, which can increase cost. For example, the lightpipe might incorporate various optical elements including lenses, and mirrors. However, lightpipe embodiments have several potential advantages compared to other embodiments that do not use a lightpipe. One advantage of a lightpipe is that the size of the monitor photodiode for the active area is often restricted in physical size, which limits the optical signal that can be produced. Particularly, if the actual pulse shape, that is often nanoseconds in duration, needs to be measured, the photodiode active area should be small to have good dynamic response. In this case, a lightpipe can be designed with a larger collection area that allows measured light to be condensed/focused onto a smaller photodiode active area, thereby improving the SNR. Because the lightpipe is purely a passive optic component, it can be positioned on top of elements which may be physically close to the VCSEL array 302, such as the electrical drive components. Closer placement of the lightpipe to the VCSEL can also improve the optical efficiency of the monitoring function.
Another potential advantage of using a lightpipe is the ability to locate the monitor photodiode relatively far from the VCSEL array and the VCSEL drive circuits. This feature is important because the drive currents/voltages used to operate the VCSEL can electrically couple and be a source of noise within the optical monitor circuit when the monitor diode is placed physically close to those components. It is highly desirable for the noise level be kept as low as possible in the optical monitor circuit in order to provide high SNR and any false readings.
The micro-prism of the monitor 408 is shown attached to the directing element 410, which is a mounting plate that in some embodiments is a transparent optical window, and in other embodiments is an optical filter. In some configurations, the directing element 410 is an optical element in the LiDAR transmitter that would be required even without the monitor 408. Thus, the directing element 410 provides two functions, one of which is securing the micro-prism of the monitor 408 and the other is optical in nature. In some configurations, the directing element 410 protects the LiDAR system from the outside environment. The micro-prism in the monitor 410 is shown coupling the light into an optical fiber 412 that has a core that is large enough to maintain the required optical signal level. The optical fiber 412 then directs the light to a monitor photodiode (not shown in the diagram).
The embodiment of a monitored transmitter with sampling prism shown in
The embodiment of a monitored transmitter with optical fiber 412 shown in
Another feature of the monitored transmitter of the present teaching is that the common point where the light is directed to from the illumination region can be positioned at various points in the optical transmitter. The common point can be determined by the positions of the projecting elements and the laser device. For example, the common point can be positioned before the last optical lens surface in the transmitter optical system. Alternatively, or in addition, the common point can be positioned after the last optical lens surface in the transmitter optical system. In some embodiments, the optical monitor can be attached to an optical window which is the last optical element in the transmitter path and protects the LiDAR system from the outside environment. In other embodiments, the optical monitor can be attached to one of the optical lens surfaces. In yet other embodiments, the optical monitor can be attached to an optical filter element which blocks some portion of the visible spectrum.
It should be understood that the monitored transmitter of the present teaching has been described in connection with particular configurations. These embodiments are only illustrative and not intended to limit the scope of the present teaching. It should be understood that various aspects of the different embodiments can be used in different combinations to achieve the advantages of the method and system of the present teaching.
The monitor 512 is connected to one or both of the controller and interface electronics 502 and the transmit electronics including laser driver 504. The monitor 512 provides information on the detected signal power, and in combination with processing in one or both of the controller and interface electronics 502 and the transmit electronics including laser driver 504, provides information about laser wavelength, optical power, pulse timing, pulse frequency, and/or pulse duration among other parameters. In some embodiments, the controller and interface electronics 502 directly controls and gets information from the monitor 512. The embodiment of
In some embodiments, the lasers in the array 506 are operated in closed loop configuration using one or both of the controller and interface electronics 502 and the transmit electronics including laser driver 504 that respond to a received photodiode current from a monitor photodiode that serves as an input to a bias control loop. This configuration can allow the transmitter optical power including the power from some or all of the optical beams generated by the laser array 506 to be maintained at a near constant value. This allows the system to be more stable against temperature and/or mechanical shifts. Also, using a control loop via one or both of the controller and interface electronics 502 and transmit electronics including laser driver 504 that includes monitoring of the optical power and control of the laser bias can accommodate some amount of degradation of the laser efficiency over its lifetime, without loss of optical power at output of the LiDAR system 500.
In some embodiments, the controller and interface electronics 502 calculates an object reflectivity using an optical power reading generated by the monitor 512. The monitored optical power can be used as a reference and then, based on actual photon counting/intensity and pre-calibration, improved reflectivity data can be achieved. This improved reflectivity data can be utilized in systems that are used for various known LiDAR applications that relate to perception.
Another feature of the method and apparatus of the present teaching is that these monitor photodiode implementations can address functional safety of the LiDAR system itself. For example, a control loop via the controller and interface electronics 502 and/or the transmit electronics including laser driver 504, including the power monitor 512 can be used to indicate that the LiDAR transmitter is faulting, for example if the measured optical power is below or above a certain threshold. For example, a control loop using one or both of the controller and interface electronics 502 and transmit electronics including laser driver 504 and including the power monitor 512 can be used to indicate the LiDAR transmitter is operating beyond an eye safety threshold if optical power is above a certain threshold.
The monitored transmitter for a LiDAR system of the present teaching has been described in connection with various embodiments that use a single VCSEL array. It should be understood that the present teaching can be extended to LiDAR transmitters that include more than one VCSEL array. In these embodiments including more than one VCSEL array, the VCSEL arrays are positioned such that the illumination region that covers the monitor has a distinct region for each VCSEL array. In these embodiments, the monitor photodiode can be configured such that it includes more than one photodiode and a separate monitor photodiode can be used for each of the separate illumination regions.
While the Applicant's teaching is described in conjunction with various embodiments, it is not intended that the Applicant's teaching be limited to such embodiments. On the contrary, the Applicant's teaching encompasses various alternatives, modifications, and equivalents, as will be appreciated by those of skill in the art, which may be made therein without departing from the spirit and scope of the teaching.
The present application is a non-provisional application of U.S. Provisional Patent Application No. 63/112,735, filed on Nov. 12, 2020, entitled “LiDAR System with Transmit Optical Power Monitor”. The entire contents of U.S. Provisional Patent Application No. 63/112,735 are herein incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63112735 | Nov 2020 | US |
